This blog post discusses the application of crowdsourced GPS-based mobility datasets for cybersecurity incident investigations.
What neighborhood related datasets are openly available, how good are they, and how can I use them? These are essential questions for any project looking to use or understand neighborhoods. This first of a 4-part series delves into the geometries of neighborhoods. The following three parts of this series will focus on demographics, housing and experiences. The resources below are not comprehensive and several are urban-centric or U.S.-centric, but they offer a quick start to data driven neighborhood work.
The list below shows neighborhood shape data from newest to oldest. They come in two common flavors: ESRI Shapefile and geoJSON. This linked OGRE web client offers an effective solution to convert between the two if you find one easier to work with than the other. All the data sources included are open source and have links to find the raw data.
- Mapzen (2016 - geoJSON) Mapzen's open source project Who's On First aims to create a full gazette of neighborhoods and other locations worldwide. At the moment it aggregates shapefiles from Quattroshapes and Zetashapes. Overall, the project seems to be proceeding with strong momentum and could become more and more valuable in the future.
- U.S. Census Tracts (2015 - ESRI Shapefile) The geometries found from the U.S. Census are not neighborhoods per se. Census tracts are small, relatively permanent statistical subdivisions of counties that are updated by local participants before each Census. I include them here because they are well-defined and easily tied to relevant U.S. Census demographic data. Further, census tracts are roughly the size of typical neighborhoods work as neighborhood approximations. It's all available as state by state downloads.
- Zetashapes (2014 - geoJSON) The shapefiles found here began with Flickr as well but use crowd-sourcing to obtain low-level granularity that focuses strictly on U.S. neighborhood polygons. At times, this hyper-local fragmentation is useful, though you can miss the forest for the trees here. The data has an advantage of relative recency and it's all available as city by city downloads.
Quattroshapes (2013 - ESRI Shapefile) This data is the most recent of a lineage of Flickr inspired shapefiles stretching back to 2008 (see below). It contains info on not only the U.S., but also major cities worldwide (Beijing, Paris, Rio de Janeiro). This latest iteration was built on Flickr data with Foursquare check-ins, Natural Earth and open government data. It is more highly fragmented than Zillow, but more cohesive than Zetashapes. It's all available as a single unified file download.
- Betashapes (2012 - geoJSON)
- Flickr 2.0 (2011 - geoJSON)
- Flickr 1.0 (Alphashapes) (2008 - ESRI Shapefile)
- Zillow (2008 - ESRI Shapefile) This data comes from Zillow's work in compiling and understanding the shapes of U.S. neighborhoods. It is relatively complete (~1% of 7000 neighborhoods have questionable geometries), and the neighborhoods are well formed, but the data is largely out of date. It's all available as state by state downloads.
If you have other useful neighborhood related sources, let everyone know in the comments below!