Merge pull request #2 from AccessMap/develop

Develop

Author: nbolten

IDEAS.md

@@ -0,0 +1,84 @@
+## Alternative strategies
+
+### Select all sidewalks within X meters of the intersection
+
+1. Select all sidewalks within X meters of the intersection
+2.
+
+### By studying 'eye test' method
+
+Humans can figure out where to draw crossings pretty easily (even unmarked
+ones). How do they do it? Here's some guesses.
+
+1. Locate sidewalk corners. This might be just one sidewalk approaching the
+street, it may be a legit corner with 2 sidewalks meeting.
+
+2. If you could reasonably cross from corner to corner, draw a crossing.
+
+3. The location where the crossing connects to the sidewalk is somewhat
+subjective and up to the style of the mapper (for now). A natural place is
+along the line of the incoming sidewalk, if there are 2. Having them meet at
+just one location is *fine* for now.
+
+So, the real strategy in code:
+
+1. Go up the street from the intersection X meters / half way up (whichever
+comes first). X could be ~10-15 meters. Return this point for each outgoing
+road.
+
+2.
+
+### Original-ish
+
+1. Go up each street in small increments (~half meter?), generating long
+orthogonal lines.
+
+2. Restrict to sidewalks within X meters of the street (something moderately
+big like 30 meters)
+
+3. Discard lines that don't intersect sidewalks on both the left and right
+sides. Can be done by taking the intersection and asking the left/right question
+
+4. Subset remainder to line segment that intersects the street (trim by the
+sidewalks)
+
+5. Remove crossings that intersect any other streets (i.e. crossing multiple
+streets). This should be done very carefully given that some streets are
+divided and have multiple intersections. osmnx tries to group these intersections
+but I don't know what happens to the lines. i.e. we don't want to discard all
+boulevards. Actual question is more complex? FIXME: revisit this step
+
+6. The first crossing remaining is probably legit.
+
+FIXME: the problem with this approach is that it will miss the situation where
+there's only one sidewalk at the corner (one street doesn't have sidewalks,
+e.g.).
+
+### Original-ish 2
+
+1. Identify sidewalks to the left of street, to the right of street
+2. Find point on the street where an orthogonal line first intersects the
+sidewalk on the right and does not intersect any other streets.
+3. Repeat for the left side
+4. Whichever is farther down is the primary candidate.
+5. Attempt to extend this orthogonal line such that it spans from right sidewalk
+to left sidewalk.
+6. If this fails, find the closest point on the 'other' sidewalk to the point
+on the street. Draw a line from sidewalk to sidewalk.
+
+### Original-ish 3: ding ding ding!
+
+1. Walk down the street in X increments.
+2. Find a line between the street point and the nearest sidewalk to the right
+that does not intersect another street. One method for doing this is to
+split the sidewalks up by block beforehand. Alternatively, use the 'block graph'
+method used in sidewalkify to group sidewalks. Cyclic subgraphs = block
+polygons, acyclic sugraphs = tricky
+3. Repeat for the left side.
+4. Compare the lines: if they're roughly parallel, keep the crossing endpoints
+and draw a line between them. This is the crossing
+5. If the crossing is too long (40 meters?), delete it.
+
+Note: rather than incrementing by small amounts and then stopping, this
+strategy could use a binary search such that an arbitrary parallel-ness could
+be found.

README.md

@@ -1,8 +1,16 @@
 # Crossify
 
-`crossifyify` is a Python library and command line application for drawing
+`crossify` is a Python library and command line application for drawing
 street crossing lines from street centerline and sidewalk centerline data.
 
+`crossify` has two usage modes:
+
+- It can automatically fetch sidewalk data from OpenStreetMap
+- It can read sidewalk data from a GIS file (e.g. GeoJSON or shapefile)
+
+In both modes, it fetches street data from OpenStreetMap and uses the sidewalk
+and street data to construct likely crossing locations.
+
 ## Introduction
 
 Pedestrian transportation network data are often missing. `crossify` works in
@@ -28,20 +36,33 @@ to Python 2.
 Once installed, `crossify` is available both as a command line application
 and a Python library.
 
-#### CLI
+### Sidewalks should be fetched from OpenStreetMap
+
+To fetch sidewalk data from OpenStreetMap, use the `from_bbox` command:
+
+    crossify from_bbox -- <west> <south> <east> <north> <output file>
+
+The values of west, south, east, and north define a rectangular bounding box
+for your query, and should be in terms of latitude (south, north) and longitude
+(west, east). The use of a double dash is necessary for the use of negative
+coordinates not getting parsed as command line options (see the example below).
 
 Example:
 
-    crossify <streets.shp> <sidewalks.shp> <output.shp>
+    crossify from_bbox -- -122.31846 47.65458 -122.31004 47.65783
+    test/output/crossings.geojson
+
+### A sidewalks file is provided
 
-##### Arguments
+If you want to provide your own sidewalks layer, use the `from_file` command:
 
-The input file can be any file type readable by `geopandas.read_file`, which
-should be anything readable by `fiona`, i.e. GDAL.
+    crossify from_file <sidewalks file> <output file>
+
+Example:
 
-For example, you could also use a GeoJSON input file:
+    crossify from_file test/input/sidewalks_udistrict.geojson
+    test/output/crossings.geojson
 
-    crossify <streets.geojson> <sidewalks.geojson> <output.geojson>
 
 #### Python Library
 

STRATEGY.md

@@ -0,0 +1,69 @@
+# Strategy
+
+This document describes the overall strategy behind `crossify`'s algorithms.
+
+At a glance, `crossify` generates lines that connect sidewalks across street
+intersections. Some of these crossings may be crosswalks, or entirely unmarked:
+such information is treated as metadata to be added later as attributes of
+the crossings generated by `crossify`.
+
+Given a streets dataset and sidewalks dataset, crossify attempts to draw one
+crossing per street in an intersection, given that there are appropriate
+sidewalks on either side. It accomplishes this in an N-step process:
+
+1. Streets are split when they share a node (point) and are at the same
+z-level. Streets are also joined end-to-end at all other locations.
+Note: this step is not actually implemented yet, so datasets are
+assumed to have been appropriately split in advance.
+
+2. Intersections are identified as shared nodes with more than 2 streets from
+step one.
+
+3. Intersections are grouped based on proximity, as some may be complex
+(e.g. 6-way intersections, boulevards). The street segment(s) connecting each
+intersection node are ignored. All further logic happens on a
+per-intersection group basis.
+
+4. Create a buffer of 10 meters (by default) around the intersection's node(s)
+and do an intersection operation on the streets/sidewalks: all streets and
+sidewalks are clipped at 10 meters from an intersection node. In most cases,
+this will mean that all streets and sidewalks are cut off by a 30-meter radius
+circle. If necessary, streets are temporarily extended to be 30 meters long.
+Idea: possibly use polygonized street data?
+Idea: use voronoi polygons?
+
+5. Select all incoming streets and split the buffer by them. This aids the
+algorithm in ensure a connection across the street, guaranteeing each end of a
+crossing connects to left or right.
+
+6. Extend 'up' each street in 1-meter increments, finding the
+closest sidewalk on each side. Only extend at most half-way down the street.
+
+7. If each sidewalk side differs too much in length, the entire crossing is
+discarded. The problem this is attempting to solve is one where incoming
+streets have a very acute angle, and we don't want to select the sidewalk on
+the opposite side.
+
+9. Choose the first 1-meter increment node that's relatively straight. Can
+abort generating any further 1-meter increment nodes for this street.
+
+9. Make that crossing truly straight - connect each side with a straight line.
+
+10. If the option is set, the first and last ~1 meter of the crossing is
+converted to a short line of 'sidewalk'. The point where a 1-meter segment of
+new sidewalk meets a crossing is where a curb interface should eventually be
+annotated.
+
+## Future work
+
+There are many ways that these strategies could be improved. Here's a list of
+ideas:
+
+- Generate crossings automatically for long sidewalk stretches so that the
+pedestrian network remains well-connected even with missing data.
+
+- Generate half-crossings where sidewalks end, so they are more appropriately
+connected to the street network.
+
+- Add option to generate crossing lines from street node metadata, e.g.
+OpenStreetMap's highway=crossing data.

cache/.gitignore

@@ -0,0 +1,2 @@
+*
+!.gitignore

crossify/__init__.py

@@ -1,4 +1 @@
-from . import draw, graph
-
-
 __version__ = '0.1.0'

crossify/__main__.py

@@ -1,37 +1,112 @@
 import click
-import geopandas as gpd
+from os import path
+import osmnx as ox
 
-from . import populate, constrain, schema
+from . import crossings, intersections, io
 
 
-@click.command()
+# TODO: See if there's a more proper way to find the project root dir
+# TODO: use a temporary dir?
+ox.utils.config(cache_folder=path.join(path.dirname(__file__), '../cache'),
+                use_cache=True)
+
+# Groups:
+#   - Download all data from OSM bounding box, produce OSM file
+#   - Download all data from OSM bounding box, produce GeoJSON file
+#   - Provide own sidewalks data, produce OSM file
+#   - Provide own sidewalks data, produce GeoJSON file
+
+# So, the arguments are:
+#   - Where is the info coming from? A file or a bounding box in OSM?
+#   - What is the output format?
+
+# crossify from_bbox [bbox] output.extension
+# crossify from_file sidewalks.geojson output.extension
+
+
+@click.group()
+def crossify():
+    pass
+
+
+@crossify.command()
 @click.argument('sidewalks_in')
-@click.argument('streets_in')
 @click.argument('outfile')
-def crossify(sidewalks_in, streets_in, outfile):
-    # FIXME: these should be turned into configuration options
-    intersections_only = True
-    osm_schema = True
+def from_file(sidewalks_in, outfile):
+    #
+    # Read, fetch, and standardize data
+    #
+
+    # Note: all are converted to WGS84 by default
+    sidewalks = io.read_sidewalks(sidewalks_in)
+    core(sidewalks, outfile)
+
+
+@crossify.command()
+@click.argument('west')
+@click.argument('south')
+@click.argument('east')
+@click.argument('north')
+@click.argument('outfile')
+@click.option('--debug', is_flag=True)
+def from_bbox(west, south, east, north, outfile, debug):
+    #
+    # Read, fetch, and standardize data
+    #
+
+    # Note: all are converted to WGS84 by default
+    sidewalks = io.fetch_sidewalks(west, south, east, north)
+    core(sidewalks, outfile, debug)
+
+
+def core(sidewalks, outfile, debug=False):
+    #
+    # Read, fetch, and standardize data
+    #
+
+    # Note: all are converted to WGS84 by default
+    click.echo('Fetching street network from OpenStreetMap...')
+    G_streets = io.fetch_street_graph(sidewalks)
+
+    # Work in UTM
+    click.echo('Generating street graph...')
+    G_streets_u = ox.projection.project_graph(G_streets)
+    sidewalks_u = ox.projection.project_gdf(sidewalks)
 
-    sidewalks = gpd.read_file(sidewalks_in)
-    streets = gpd.read_file(streets_in)
+    # Get the undirected street graph
+    G_undirected_u = ox.save_load.get_undirected(G_streets_u)
 
-    # Ensure we're working in the same CRS as the sidewalks dataset
-    crs = sidewalks.crs
-    streets = streets.to_crs(crs)
+    # Extract streets from streets graph
+    click.echo('Extracting geospatial data from street graph...')
+    streets = ox.save_load.graph_to_gdfs(G_undirected_u, nodes=False,
+                                         edges=True)
+    streets.crs = sidewalks_u.crs
 
-    # FIXME: this is where we'd create the crossings
-    dense_crossings = populate.populate(sidewalks, streets)
+    #
+    # Isolate intersections that need crossings (degree > 3), group with
+    # their streets (all pointing out from the intersection)
+    #
+    click.echo('Isolating street intersections...')
+    ixns = intersections.group_intersections(G_streets_u)
 
-    if intersections_only:
-        crossings = constrain.constrain(dense_crossings)
-    else:
-        crossings = dense_crossings
+    #
+    # Draw crossings using the intersection + street + sidewalk info
+    #
+    click.echo('Drawing crossings...')
+    st_crossings = crossings.make_crossings(ixns, sidewalks_u, debug=debug)
+    if debug:
+        st_crossings, street_segments = st_crossings
 
-    if osm_schema:
-        crossings = schema.apply_schema(crossings)
+    #
+    # Write to file
+    #
+    click.echo('Writing to file...')
+    io.write_crossings(st_crossings, outfile)
+    if debug:
+        base, ext = path.splitext(outfile)
+        debug_outfile = '{}_debug{}'.format(base, ext)
+        io.write_debug(street_segments, debug_outfile)
 
-    crossings.to_file(outfile)
 
 if __name__ == '__main__':
     crossify()