Posts tagged geocoding
In the previous post I mentioned that the current build of Depiction was not Geo-coding starting locations.
We have fixed the problem and created a new build which you can find here. This build 12614 will geo-code starting locations.
Our initial geocoding of a new location at the Welcome screen is currently not working. This will mainly effect new users. Long time users that have geo-coded their location before won’t run into this because the geo-coding info will be in their cache. We hope to have this resolved soon.
If you are running a version of Depiction before 1.4.2 or a version of 1.4.2 with a build number of 12450 or earlier then you are probably getting can’t geo-code errors when you try to start a new Depiction. Our geo-coding service changed their API and so prior versions of the software no longer will geo-code. You can find out what version you are running by going to the help menu and choosing about. You can still start new depictions by entering a starting latitude and longitude.
A new build (12522) is available on the website and can be found here.
Problems seem to happen in threes. First our geo-server went down, then USGS changed the location of their elevation data and now Yahoo has changed their API which is what we used for geo-coding the starting location for new Depictions. Existing long time users may not notice either of these problems if they are creating new depictions in their home areas because the data may still be in their cache. However, new users and trial users will definitely notice this as will new DepictionPrep users. We have a fix for both problems and the current download on the website is an interim build 12522 which fixes these issues. We will be coming out with an official build next week but if you are experiencing these issues then you can download the new version from the website. We have noticed that the new USGS elevation source is very slow. Apparently they are now returning 1 degree tiles that are 250mb in size and if your boundary happens to cover more than one tile then you could have to download multiple tiles. We hope to have our geo-server up and running next week and so we can have reasonable elevation download times.
It appears that all of our data sources are changing. Now the initial location geo-coding has ceased to function. We use Yahoo and they recently changed their API (application program interface) which caused our malfunction. We hope to have a fix for this in the next day or two. We have actually fixed the problem and are in the process of testing the build to make sure it is OK. We have also fixed the Elevation Quickstart data source. Unfortunately they changed how they are serving out the data and give it out in 250mb chunks and so it takes much longer for it to download. Existing users won’t notice much of a change if your elevation data is in your cache for your area but new users will certainly experience this. We are working to find a solution to this problem as well. We are sorry for any inconvenience these problems cause you.
In our recent Depiction 101 Q&A webinar (http://www.depiction.com/101/QA/Jan12)one of our customers asked the question about why one would want to export data to a CSV file if they had already imported it. It was a good question and I came up with at least 5 reasons one might want to do that:
- Backup your data in a pre-geocoded format.
- If you moved elements to their geographically correct location, to save them in case you have to reload the elements at some point.
- To more easily make mass changes to the elements.
- To backup a subset of a large shape file.
- To change a shape file property to EID in order to merge data with it.
I wanted to elaborate a bit more on those reasons.
1. Backup your data in a pre-geocoded format.
When you import a CSV file with addresses. Depiction goes out to the USC geo-coder and gets the latitude and longitude for each address and places the appropriate icon at that location. Once you have done this, if you export those elements to a CSV file then they will include the lat/long of each element. This way if you ever have to reload those elements or want to share them with somebody else, then the file won’t have to be geo-coded the next time. If you have a large CSV file this can save some time. It helps us too because every time you geo-code an address it costs us a few cents.
2. If you moved elements to their geographically correct location, to save them in case you have to reload the elements at some point.
This second item is similar to the first but with a little twist. When the geo-coder geo-codes an address it will usually get it to the right block but not necessarily on the right lot in the block. The reason for this is that if you an address of lets say 2025 main st, the geo-coder thinks that the address range for that block is 2000 to 2099. It calculates that 25 is one fourth of 99 and so it places the icon 1/4 of the way down that block. Where in reality the real address range may be 2000 to 2032 and so the icon really should be at 9/10’s of the way down the block. So if it is important to you to have the icons in the exact location you will have to go in and move each one. Now if for some reason you have to reload that original file then you are going to have to move all of those icons again. Unless you had first exported it to a CSV file, in which case the program would have saved the new lat/long for each element. So if you have to move a lot of icons to their geographically correct locations then backing them up to a CSV file is a really good idea.
3. To more easily make mass changes to the elements.
If you are using Depiction as a data base manager and are storing a lot of data along with each element, sometimes it is easier to make changes to the data in a spreadsheet program. So if you export them to a CSV file you can edit them in the spreadsheet program and then reload them.
4. To backup a subset of a large shape file.
A fairly new feature to Depiction in the 1.3 series is the ability to export shape files to a csv file. So to give a specific example of when you might use this. I was working with a community that was attempting to do a community wide map your neighborhood exercise. One person had the entire community and a shape file was loaded that had all of the parcel boundaries for the community. Using the shape drawing tool, we drew the neighborhood boundaries for all of the neighborhoods. Then using the select tool we selected all of parcel shapes that were in a particular neighborhood. Then using the export to CSV function we exported the selected shapes to a CSV file. Then we could start a new Depiction story and import that CSV file and we would have a depiction for just that neighborhood. That file could then be given to the neighborhood coordinator who could use DepictionPrep to load the file and then make the appropriate changes and maps for their neighborhood.
5. To change a shape file property to EID in order to merge data with it.
This is really a neat feature. Before we could export shapes to a CSV file this particular task was fairly complicated.
So lets say you have a shape file of zip codes and you also have some tabular data based on zip codes. In this tabular data file you have the zip code and lets say median income, population, etc. You want to be able to colorize your zip codes based on this numeric data. But how do you get the tabular data into the zip code shape elements. Here is a step by step process:
- Import your shape file
- Delete unwanted shapes (if necessary)
- Export your shapes to a CSV file
- Open the CSV file
- Change the property name of your key field i.e. zip code to EID and save the file
- In your depiction delete the shapes
- Re-import your CSV file of shapes
- Open your spreadsheet of data that you wish to merge.
- In our example we are saying the key field is Zip Code so in this file change the name of the zip code field to EID.
- Delete any other properties that aren’t of interest and then save that file
- In your Depiction chose Add by File and chose your csv file
- Select import by EID and then select Import – the data in this file will now be merged with the data in the shape file based on zip code
- Now if you open a shape element you should see the data fields you just imported and you can now colorize the shapes based on those fields.
Here are view other tips when dealing with CSV files.
TIP #1 – When you export elements it is best to export just one kind of element to a CSV file. If you export multiple element types at once then you will get all of the properties for all of the elements in your resulting file, which is OK. But when you go to re-import that file all of the elements will have all of the properties of all of the elements which is probably not what you wanted.
TIP #2 - When you export to CSV it includes all of the default properties, any properties you have added and a bunch of depiction descriptive fields. If you want can delete all of the descriptive fields to make the file easier to work with. We are looking to add a feature in a later release that would allow you to not have these fields show up on the export.
TIP #3 - If you are exporting a shape file and the shape is a fairly complex polygon it is possible that the number of points in that polygon will exceed the total amount of characters allowed in a single cell. In this particular situation we don’t have a solution to this and that shape may not be able to be re-imported properly.
I hope you find this information useful and feel free to send me any questions you have on any of it.
My local volunteer fire department purchased a copy of Depiction asked me to help them with a project. The fire chief had several things he wanted to do with the software but the first one was to create a map of all of the parcels in the district and then add to it the ownership information. He had received two files from the local county GIS department: a shape file of all of the parcel boundaries and a database file of the ownership information from the assessor.
The challenge was to get the shape file data loaded and then geo-code the address data.
The first challenge was to get the shape file loaded. The main issue here was because of the shape of the fire district in order to get the entire district in one story also meant getting non-district territory included. This wouldn’t have been a problem but the file from the county was of the entire county and not just the district, so when the shape file was loaded it included the shapes for the non-district territory as well. Unfortunately this added about 2000 more shapes to the file which of course would have had an performance impact. So I ended up cutting the district in half and having two files. I still had non-district shapes but an acceptable number.