Converting Shapfiles to GeoJSON

For a recent project I needed county boundaries in GeoJSON format, which I did not have.  I did, however, have the shapefiles.  Luckily for me the ogr2org command easily handles this conversion:

ogr2ogr -f "GeoJSON" c:\california.json "c:\co51_d00.shp" co51_d00

Note: ogr2ogr, at least the Windows version, is bundled with Quantum GIS.

Google Fusion Tables and KML Polygons

Google Fusion Tables (Fusion) makes it easy to visualize relatively simple data on a Google Map, assuming some portion of your data is or can be geocoded.

Recently, I needed to visualize California county level data on a Google Map.  My requirements were pretty simple:

1. Outline the counties
2. Fill county polygons with a color representative of the data
3. Provide an informational pop-up when a county is clicked

Since the requirements were pretty simple I decided to give Fusion a shot.  Fusion allows you to import a Google Docs Spreadsheet as a datasource, so that's the route I took.  

In order to outline the California counties I turned to Fusion's ability to process KML.  Using the KML polygon I was able to provide standard polygon coordinates (derived from Census data), which ended up being my "location" source for my data.

Fusion offers several options for formatting polygons on the map.  In my case I used a number in my import file to color the counties/polygons creating a semi choropleth map.

Finally, Fusion allows you to format your informational popups using a combination of data from your datasource and HTML.

If interested, I've created a sample Google Docs spreadsheet that contains polygons for California's 58 counties.  Note: The county_name in the sample_data sheet is being used to lookup the kml_polygon in the ca_counties sheet. 

Visualizing County Level Data

A few weeks ago I picked up a copy of Visual This at the bookstore.  Overall I thought the book was great and was particularly interested in an example that used FIPS codes to create a choropleth map using a public domain map from WikiMedia.

The hardest part was creating an appropriate dataset.  The map itself is an SVG image, which is nothing more than an XML formatted text file.  Each county has an associated id which just happens to be its FIPS code.  So if you're able to match up a piece of your data, in my case the zipcode, to a FIPS code you're almost there.

I'm pretty happy with the results.  I ended up using Inkscape to tweak the SVG file before sharing it -- not represented here.

Update: Here's a link to a tutorial from the book's author.  Wish I would have found it first!

Parsing Files with Python

Recently, I've had to work with a lot of data stored in various text files: comma and tab delimited, fixed width, etc... So I decided, based on a little research, Python would be my tool of choice to parse and manipulate these files prior to inserting them into an Oracle database.

My most recent task: extract several columns from a fixed width file with over 22 million lines, remove leading zeros from two of the columns (making them a proper number) and write the results to a new CSV file.

Here's what I came up with -- it's quick and dirty, but works.

Python code:

import re

i = open("zipcty10")
o = open("10.csv","w")

line = i.readline()

while line:
    print >>o,line[0:5] + "," + re.sub(r"^[0]*","",line[15:19].strip()) + "," + re.sub(r"^[0]*","",line[19:23].strip()) + "," + line[23:25] + "," + line[25:28] + "," + line[28:53].strip()

    line = i.readline()

i.close()
o.close()

2011 Chicago Marathon

I've been contemplating running the Chicago Marathon for a while, and today I finally decided to do it!  The marathon is Sunday, October 9, 2011.

I've decided to register/run under a charity/non-profit entry for the United Parent Support for Down Syndrome (UPS for DownS for short).  This mean I need to raise $950 that will go to the Katie MacDonald Literacy Project.

This give me about 5 months to train and raise the needed $950.  So if you are feeling charitable, please make a donation using my Active.com fundraising page.  100% of your donation goes to the Katie MacDonald Literacy Project.

My goal is to keep this site up to date with my training and fundraising progress, so please sign up for email updates.

Thanks for your support.

Mike