Blogs.OpenStreetMap.org

August 28, 2014

"OpenStreetMap.org User's Diaries"

Water in OpenStreetMap

This article is supposed to give a brief introduction about water in OpenStreetMap. It will show you typical errors made by mappers and how they can be fixed with the help of MapRoulette.

The Water Network

Mapping water in OSM can be divided into two categories. On the one hand there are area types of water like ponds, lakes, reservoirs and even whole river areas (waterway=riverbank) and as a special case the sea (natural=costline). On the other hand there are simple ways, the waterways: These are rivers, streams, canals, ditches and drains.

The water network is thus built by the waterways. This also means that a broad river mapped as riverbank additionally contains a waterway=river in between. Thereby the direction of the way also denotes the direction of the water flow. If you follow a river or stream, you should at some point reach a big lake or the see. How to identify the direction of a way is explained on this wiki page.

Unfortunately the water network in OSM is broken at many places. As a result, some rivers appear to flow uphill or spring from the sea.

SRTM and MapRoulette

SRTM provides height data for mostly the whole planet for free. It makes it possible to determine for any waterway in OSM if it flows downhill or uphill. Though the data is unprecise, many waterways are quite long which makes it quite reliable to identify the direction of flow for many rivers and streams. However, one can't advise to start an automatic mass edit based on SRTM. MapRoulette Having said that, there's a neat project called MapRoulette, helping to solve such problems by letting the OSM community decide on small tasks. Thus there is a new challenge in MapRoulette to check on the waterway's direction. Each task is one waterway that supposedly flows uphill and its flow direction has to be turned.

The difficulty of the tasks will grow over time. If the difference of height is some hundred of meters it's simple in most cases. For smaller differences, e.g. 10 meters, it might well be a error in SRTM. However, the challenge will start with the easy tasks. And still there will be extra help: Each task's instruction contains the height difference from the start to the end of the way as an additional help.

False Positives

It's not always as easy as one might think. There are some cases that need special attention that are listed below. * A canal might be tagged as a river or stream. In contrast to rivers or streams a canal may "flow" uphill. Particularly: What's the direction of a canal anyway? * Not every fork is a confluence. There are cases where a river splits up into two ways. * There are rivers and streams that dry out or drain away. Thus they "disappear" from the map. But you may not conclude that this is a spring.

Errors and Error Detection

The following paragraph will show some scenes with errors in the water network. It should give you an impression, how to determine (and repair) common mistakes.

All screenshots showing imagery are proprietary and based on GEOIMAGE-AUSTRIA® (c.f. wiki)

Mountains

In mountainous regions you typically get a large difference in height. An example of this can be seen in the following JOSM screenshots: The complete stream, the start and the end of the stream all end start As one can clearly see, the stream originates high up in the mountains and finally leads in a larger river in the valley. Downloading the data in the surrounding area one can additionaly check, if the stream and river are connected or not. The right image below shows the result after correcting the direction of the way. data fixed

Forks

If you check a waterway, it is also wise to investigate the complete way for forks. It happens quite often that more than one stream flows uphill as it was tagged by the same author. This might look like this: multiple

Water Mouth

Quite easy cases are rivers and streams that end in the sea. This can be detected, as the waterway is connected to the coastline. Thus, the way's direction has to lead to the sea. coastline

Defective Parts of the Way

In some cases only parts of a waterway are directed in incorrect. In these cases one part has to be wrong, but one also has to be cautious to correctly determine which part is wrong. wrong

Additional Notes

The network of waterways should be fully linked. Thus, it is advisable to also verify that ever way's end is connected to the next waterway. Especially ways should not end at riverbanks but at a river or stream. The same applies for lakes: If a lake has an inflowing and outflowing waterway, these should be connected.

Further typical errors or missing notes are welcome. I'd add them here

Any changeset's source tag should at least contain SRTM as a source.

by Peda at August 28, 2014 12:17 AM

Wasser in OpenStreetMap

Im Folgenden soll ein kleiner Überblick über Wasser in OpenStreetMap gegeben werden, die typischen Fehler erklärt werden und mit MapRoulette ein Weg aufgezeigt werden, diese Fehler einfach und schnell zu beseitigen.

Das Wassernetzwerk in OpenStreetMap

Das Mapping von Wasser in OSM unterteilt sich in 2 große Kategorien. Zum einen gibt es Wasserflächen die gemappt werden. Dies sind z.B. Weiher, Seen, Wasserreservoirs aber auch eine genaue flächenmäßige Erfassung von Flüssen (waterway=riverbank) und als Spezialfall das Meer. Zum anderen sind dies Wasserwege. Das sind die Flüsse (river), Bäche (stream), Kanäle (canal), Bewässerungsgräben (ditch) und Abwasserkanäle (drain).

Das eigentliche Wassernetzwerk wird dabei ausschließlich durch die Wasserwege aufgespannt, so dass ein breiter Fluss der als riverbank gemappt ist stets zusätzlich den eigentlichen Wasserweg (waterway=river) enthält. Die Richtung des Weges gibt dabei gleichzeitig die Flussrichtung des Wassers an. Verfolgt man also einen Bach oder Fluss immer weiter, sollte man irgendwann in einem großen See oder im Meer enden. Wie man die Richtung des Flusses (des Weges) erkennen kann, erklärt dieser Wikiartikel.

Leider ist das Wassernetzwerk an vielen Stellen in OpenStreetMap unvollständig bzw. kaputt, so dass Flüsse schon mal bergauf fließen oder im Meer entstehen statt dort zu enden.

SRTM und MapRoulette

Mit SRTM gibt es eine nahezu weltweite Abdeckung mit frei verfügbaren Höhendaten. Dadurch ist es einfach möglich, für Wasserwege in OSM zu prüfen, ob diese bergauf oder bergab fließen. Zwar sind die Höhendaten an vielen Stellen relativ ungenau, durch die typischerweise sehr langen Wasserwege kann man aber für viele Flüsse und Bäche recht zuverlässig bestimmen, ob diese richtig gemappt sind. Dennoch, eine automatische Korrektur auf Basis dieser Daten ist nicht zu empfehlen. MapRoulette Mit MapRoulette gibt es aber ein nettes Projekt, dieses Problem zu lösen: Man teil das Problem (Challenge) in kleine Aufgaben (Tasks) und lässt diese von der OSM-Community lösen. Es gibt daher eine neue Challenge, Wasserwege zu korrigieren und somit das Wassernetz zu verbessern. Jeder Task ist dabei ein Wasserweg der vermeintlich bergaufwärts läuft und umgedreht werden muss.

Der Schwierigkeitslevel wird dabei mit der Zeit steigen. Bei einer Höhendifferenz von mehreren Hundert Metern ist der Fall meist leicht, bei einer Höhendifferenz von 10 Metern kann es sich aber auch schnell um Fehler in den SRTM-Daten handeln. Beginnen wird die Challenge aber anfangs mit den eher eindeutigen Fällen. Dennoch ist als zusätzliche Hilfe für jeden Task die Höhendifferenz von Weganfang zu Wegende angegeben.

Falsepositiv

Aber nicht immer ist der Fall so einfach wie man anfangs denkt. Daher gibt es hier eine kurze Liste von Dingen, die man beachten sollte bzw. warum es eben doch vorkommen kann, dass der Wasserweg bereits korrekt ist. * Durch eine falsche Klassifikation kann ein Kanal als Bach bzw. Fluss getaggt worden sein. Kanäle können aber im Gegensatz zu Bächen und Flüssen durchaus bergauf "fließen" und besitzen insbesondere meist keine Richtung im eigentlichen Sinn. * Nicht immer ist eine Kreuzungsstelle von Wasserwegen ein Zusammenfluss. Es gibt durchaus Bäche und Flüsse, die sich teilen und in zwei unterschiedliche Richtungen weiterfließen. * Es gibt Bäche und Flüsse die austrocknen/versickern oder einfach nicht mehr sichtbar sind und somit von der Karte "verschwinden". Man darf daher nicht daraus schließen, dass es sich um eine Quelle handelt.

Fehler und deren Erkennung

Im Folgenden werden einige typische Szenarien gezeigt, in denen das Wassernetz einen Fehler enthält. Es soll grob vermitteln, wie man diese Fehler identifizieren (und reparieren) kann.

Die Screenshots im Folgenden zeigen zum Teil Luftbilder. Diese unterliegen dem Markenschutz von GEOIMAGE-AUSTRIA® (siehe hier ).

Gebirge

Vor allem bei großen Höhenunterschieden hat man es typischerweise mit Gebirge zu tun. Die folgenden drei Bilder zeigen einen solchen Fall in JOSM mit den jeweiligen Ausschnitten von Bachanfang und -ende. all end start Wie man recht klar erkennt, entspringt der Bach weit oben am Berg und mündet im Tal in einen größeren Fluss. Dies erkennt man noch besser, wenn man sich auch noch die Daten aus der Umgebung herunterlädt. So kann man auch gleich noch prüfen, ob der Bach auch mit dem Fluss verbunden ist. Im rechten Bild ist dann das Ergebnis, nach dem Ändern der Flussrichtung sichtbar und man kann sein Ergebnis hochladen. data fixed

Verzweigungen

Bei der Korrektur von einzelnen Wasserwegen solle man zudem prüfen, ob der Bach unter Umständen Verzweigungen aufweist. Dies sind oft Stellen, wo gleich mehrere Wasserwege die falsche Richtung aufweisen, da sie vom selben Autor gemappt wurden. multiple

Meeresmündung

Sehr einfach erkennbar sind Flüsse und Bäche, die im Meer enden. Der Wasserweg schließt dabei an die Küstenlinie (natural=coastline) an, die Flußrichtung muss entsprechend im Meer enden. coastline

Falsche Teilstücke

In einigen Fällen passiert es auch, dass ein angeschlossener Wasserweg bzw. ein Teilstück des Wasserwegs eine falsche Richtung aufweist. Hier ist etwas Vorsicht geboten, da man noch prüfen muss, welcher Teil des Wasserweges den Fehler aufweist. wrong

Weitere Hinweise

Da das Wassernetz vollständig vernetzt sein sollte, ist es ratsam, bei Korrekturen zu prüfen, ob der Wasserweg korrekt an den nächsten Wasserweg angeschlossen wurde. So sollte der Wasserweg insbesondere nicht an der riverbank sondern am Fluss selbst enden. Gleiches gilt für Seen: Hat der See sowohl Zulauf als auch Abfluss, sollten die beiden Wasserwege entsprechend verknüpft werden und nicht am Rand des Sees enden.

Beim Hochladen der Korrekturen sollte im source-Tag des Changesets unter anderem SRTM als Quelle genannt werden.

Viel Spaß

by Peda at August 28, 2014 12:15 AM

August 27, 2014

OSMBlog (German)

Wochennotiz Nr. 214

19.08. – 25.08.2014

gaza

Gaza-Karte auf OSM und bei einem Marktbegleiter [1]

Talk, Forum, Wiki & Blog

Open Data & ODbL

Wochenaufruf

Humanitarian OpenStreetMap Team

Konferenzen

  • Am 26. September findet die erste MaptimeBER statt.

Karten

switch2osm


  • Mit diesem Bookmarklet switchst du von nicht freien Karte (GMaps oder Here) direkt zu OSM. Wie? Einfach das Bookmarklet “Switch to OSM” (evtl. etwas nach unten scrollen!) in die eigene Lesezeichenliste ziehen. Eine der o. g. kommerziellen Karten aufrufen, Bookmarklet klicken – Willkommen zurück in der freien Welt. (via @OSMBuildings)

Kennst du schon …

Programme

Programmierung

und sonst

  • Augmented reality auf Karten
  • Die NASA bittet um Mithilfe bei der Katalogisierung von Satellitenbildern der Erde, die nachts aufgenommen wurden.
  • Kartenfehler, die Leben kosten: Gasexplosion bei Bauarbeiten in Itzehoe wurde vermutlich durch eine im digitalen Kartenmaterial der Stadt nicht verzeichnete Gasleitung verursacht. Die Leitung war auf papierbasierten Karten vor der Digitalisierung noch eingezeichnet.
  • Beth Schechter sucht Karten für das 2014 Istanbul Design Biennial
  • Matthias Schwindt berichtet über seinen neuen Praxistest des Garmin Oregon 650t.
  • Die Raketenbetreibergesellschaft Arianespace berichtet, dass zwei neue Satelliten des europäischen Satellitennavigationssystems Galileo nicht in der richtigen Erdumlaufbahn ausgesetzt worden seien (via heise).

Wochenvorschau

Termine vom 27.08 bis 03.09.2014

Ort Name Datum
Düsseldorf Stammtisch 27.08.2014
Essen Stammtisch 30.08.2014

Hinweis:
Wer seinen Termin hier in der Liste lesen möchte, trage ihn in den Kalender ein. Nur Termine, die dort stehen, werden in die Wochennotiz übernommen.

In eigener Sache

flattr this!

by Wochennotizteam at August 27, 2014 06:10 PM

"OpenStreetMap.org User's Diaries"

Lisätty kaikki puuttuvat Bitcoineja hyväksyvät kauppapaikat karttaan

Lisäsin kaikki kartasta puuttuvat Bitcoineja hyväksyvät suomen kauppapaikat karttaan. Jos jotain puuttuu niin voi lisätä itse tai laittaa minulle viestiä osoitteeseen vraisa@prasos.fi

by vraisa at August 27, 2014 06:02 PM

Sorbas (Almería)

Terminado este municipio, añadido mas de 30 núcleos de población y sus accesos, tal como su info.

by Canellone at August 27, 2014 04:49 PM

What's up with the GPS traces?

I've noticed recently that a lot of the GPS traces in my area are doing weird things. Many of the traces are fine, but a good portion of them are what can only be described as 'dashed'. I thought it might not be just mine, so I checked other areas of the world, and it's happening other places too.

Here's a sample from my region. Most of those traces are my own, but some are dashed and others are smooth. I think that the dashed ones are more recent. Screenshot of weird traces

What's also perplexing is that I can download the trace from OSM, and it displays all the detail that there should be.

Does anyone know what's going on?

EDIT: I looked back over my traces, and everything back to about July 26 is fine, but prior to that for a couple months is all funky.

by pkoby at August 27, 2014 02:26 PM

Very impressed with Vespucci

I'm a bit behind the times with smartphones. I only got my first a couple of years ago, a small-screened, low-powered model on a cheap-ish contract. It did the job, but I recently upgraded to a far more powerful, larger-screened model with 4G capability and WOW suddenly I'm finding I can do all sorts of cool things. One of those things is OSM-ing on-the-go with Vespucci. I'd heard about it ages ago but dismissed it (who'd want to fiddle around with a phone when you can sit at home with JOSM?), but I installed it recently and it's brilliant. Smooth, intuitive, & great features like the "recall last tags" button and the fact it won't let you download new areas without finishing off the last area. I've become a father recently so don't have time for long hikes in the country right now, so instead I'm doing small things like POIs and addresses whilst walking to the station, and uploading them via Vespucci on the train to work. Perfect.

by Pgd81 at August 27, 2014 01:08 PM

laser puissants

j'ai un ami qui a acheter un laser sur laserachat.com ,c'est un laser 10000mw ,tres dangereux ,il a diriger vers les yeux de son achat ,et quelques secondes ,son achat est aveugle ,awful! je vous suggere de ne pas acheter un tel laser

by BOUHADIBA at August 27, 2014 05:29 AM

August 26, 2014

Peter Batty

Denver Union Station guide

This is slightly off topic, but as a side project I have just put together a small web site which is a guide to all the cool new developments at Denver Union Station. If you live in (or are visiting) the Denver area and haven't checked out Union Station recently, you definitely should! And to make it not totally off topic, there will be an interactive map appearing on the site shortly, I just

by noreply@blogger.com (Peter Batty) at August 26, 2014 04:16 AM

August 25, 2014

"OpenStreetMap.org User's Diaries"

Presenting about Wikimedia and OpenStreetMap at Wikimania 2014

Eugene Villar giving his presentation

Photo Ⓒ Harry Wood, CC-BY-SA 2.0

Over two weeks ago, I had the amazing opportunity to attend Wikimania 2014 in London. Wikimania is the annual conference for the Wikimedia movement, which includes the Wikipedia project. Coincidentally, the conference occurred on the same weekend as the 10th anniversary of OpenStreetMap. As my way of celebrating the anniversary, I gave a presentation about the collaborations between OpenStreetMap and the Wikimedia projects at the conference.

Read more at my blog

Presentation slides

by seav at August 25, 2014 08:05 PM

OSMBlog (German)

Nachlese Wochenaufgabe KW33/34 Adressen ohne Straßenname

#osmwa3334 Vorher#osmwa3334 Nachher

“Ein netter Spaß ist die Wochenaufgabe aber allemal.” (User:seichter)

Ich denke die Wochenaufgabe Straßennamen KW 33/34 11.08.–24.08.2014 hat eine kleine Nachlese verdient.

Vorweg:  Für Jan und mich und hoffentlich auch für euch waren das zwei interessante Wochen.

Mein Dank geht hier an Jan, ohne dessen Statistiken ich meine Tabellen nicht hätte füllen können, was ich als alter Excel Freak und Hobby Statistiker auch gerne mit Calc gemacht habe.

Aber die wichtigste Arbeit kommt natürlich von Euch, die Ihr über 90.000 Straßen korrigiert habt. Als wir losgelaufen sind, hatten Jan und ich 65.000 Korrekturen für ein sehr ambitioniertes Ziel gehalten.

Wir können mitnehmen, dass eine Wochenaufgabe allen Beteiligen deutlich mehr Spaß macht, wenn man Erfolge auch kurzfristig sehen kann. Für die tägliche Zusammenfassung gab es die aufbereitete Statistik von Jan und für den kleinen Check zwischendurch die verschiedenen Overpass API Abfragen.

Ein zweiter wichtiger Punkt ist sicherlich das Gefühl, dass im Team an der Wochenaufgabe gearbeitet wurde. Das kam im Forum mit fast 300 Posts gut rüber.

Bewertung der Kommunikationskanäle

  • Google Calc
    Google Calc hat gute Dienste geleistet. Es bietet eine einfache Möglichkeit die zeitliche Entwicklung über die Wochenaufgabe zu dokumentieren, kann zur Generierung von Grafiken genutzt werden und ist plattformübergreifend ohne Installation zu nutzen.
    Immer wenn ich die Tabelle gepflegt habe, habe ich bis zu fünf anonyme User online gesehen. Die Möglichkeit, dass mehrere Mapper Daten aus eventuell unterschiedlichen Quellen in einer Tabelle pflegen, wurde dabei noch gar nicht genutzt.
    (Ich gehe davon aus, dass die benötigten Features auch mit anderen Tools abgedeckt werden können, bei dem “Zahlenwust” war die Google Lösung für mich einfach naheliegend und simpel.)
  • Twitter
    In dieser WA eher unidirektional, was sich gegen Ende etwas aufgelöst hat. Der Bedarf an Instant-Kurznachrichten scheint bei OSM (noch) nicht so hoch zu sein. Zu diesem Punkt würde mich euer Feedback interessieren. Ich hatte ja unter #osmwa3334 etwas “rumexperimentiert”, da Twitter nicht mein Medium ist.
  • Forum
    Die inhaltliche Kommunikation fand ausschliesslich im Forum statt. Die Beteiligung im Forum war erfreulich.
  • Talk DE
    Auf Talk DE habe ich keine Diskussion zur Wochenaufgabe wahrgenommen.
  • Overpass Turbo
    Mit Overpass Turbo kam man zügig an die sehr konkret spezifizierten Fehler heran. Die Geofabrik Tools hätten zwar den einen oder anderen Fehler mehr gezeigt, aber die Aufgabe wollte sich ja bewusst fokussieren.

Bleibt die Frage, was kommt jetzt. Jan und ich können die Aufgabe gerne noch einmal auflegen – wie es im Forum ja schon gewünscht wurde, auch wird Jans Datenbank bestimmt genug andere Fehler zählen können. Aber eine kleine Verschnaufpause ist sicher gut.

Schön wäre, wenn das Konzept der (statistischen) Begleitung der Wochenaufgabe häufiger zum Einsatz kommt. (Anfragen, was da geht, hatte ich schon). Ich kann mir vorstellen, dass die Overpass API und ein bisschen Scripting drumherum auch Mappern, die keine eigene Datenbank mit OSM Daten zu Hause haben, die Möglichkeit gibt, Dinge zu zählen und in die tägliche Verfolgung zu nehmen.

Bis dahin,

Viel Spaß beim Mappen und nicht von anderen Karten abmalen. ;-)

Christoph

 

 

flattr this!

by TheFive at August 25, 2014 07:31 PM

Mappa Mercia (UK Midlands)

OpenStreetMap alternatives to Google’s Maps Engine and Fusion Tables

Often we just want to produce a simple map showing data as a collection of pins (map markers). In this post we look at two tools that both use OpenStreetMap base maps; namely geojson.io and uMap. These are both potential alternatives to Google Maps Engine and Fusion Tables. 

OpenStreetMap has come a long way in the last 10 years; we’ve reached 1.7 million registered users, over 300,000 of whom are the last editor an an object in OpenStreetMap, and we’ve contributed over 2.4 billion nodes (points) and 248 million ways (linear features). This has led to OpenStreetMap being deployed by large companies such as Apple, Mapquest and Foursquare, and relief organisations such as Médecins Sans Frontières.

However when it comes to individuals – including those who associate themselves with the Open Source or Open Data communities – many people stick with what they know. More often that not that’s Google. OpenStreetMap is quite different; as it’s not a traditional company, most of the interesting stuff is going on outside of the core openstreetmap.org website. Let’s look at two online tools that use OpenStreetMap for the base map display.

Geojson.io

Geojson.io describes itself as “a quick, simple tool for creating, viewing, and sharing map”. It has a strong focus on input/output (io) as it supports major formats such GeoJSON, KML, GPX, CSV, TopoJSON, amongst other formats listed on the OpenStreetMap wiki. The inputs can be from a file, from a GitHub (including Gist snippets) or added manually via the website’s user interface. The final map can be exported to file, to GitHub or shared as an embeddable iframe or as a standalone URL. Visual customisation is limited to a choice of three map backgrounds.

Example:

In this example I have a geojson file of pubs in and around Stratford-upon-Avon. All the data is as points (nodes) and I want to produce a simple map to display in this blog post. Using geojson.io the steps are:

  1. Go to geojson.io.
  2. Click Open and load the source file. The map data can then be edited in the right hand pane (I used the table edit to drop some columns)
  3. Pick a map background from the choice of Mapbox, Satellite and default OSM from the options in the bottom left.
  4. Click Share and copy and paste the embeddable iframe.

Here is the result:

Quick, simple and effective.

uMap

Similar to geojson.io above, uMap describes itself as a way to “create maps with OpenStreetMap layers in a minute and embed them in your site”. It supports a number of import formats including geojson, csv, kml and also allows you to draw content directly on the map. On the export side, you can download the map data in three different file formats, or share it via an embeddable iframe or stand alone URL. Unlike geojson.io, uMap has numerous customisation options, for example you can pick from 16 map backgrounds, map data can be styled with a colour, fill opacity and line type, and the popup content can be adjusted to display additional information such as images and web links. uMap also enables you to easily work collaboratively on a map. This flexibility does however make the user interface more complicated than geojson.io’s.

Example:

This example uses a file of school locations in and around Lichfield. Most of the data is stored as polygons, however it also include point data. As before the aim is to produce a simple map to display in this blog post. Using uMap the steps are:

  1. Go to umap.openstreetmap.fr or one of the alternate instances listed here.
  2. Click Create a map.
  3. Click Import data (the up arrow icon on the right or keyboard shortcut Ctrl+I).
  4. Customise using the options on the right. Here I’ve changed the map background to MapQuest Open.
  5. Click More on the left hand side, followed by Embed and share this map.
  6. Copy and paste the embeddable iframe (I also had to click Current view instead of deafult map view in the iframe options box.

Here’s the result from uMap:

Advanced, customisable, and featureful.

Tell us what you think

Have a go and let us know what you think. How do they compare with what is available from Google or others, and what would make them better. Drop us a line in the comment box below or via our twitter account, @mappamercia.

by RobJN at August 25, 2014 06:21 PM

"OpenStreetMap.org User's Diaries"

Sorbas (Almería)

Empezando y retocando el municipio de Sorbas y sus pueblos.

by Canellone at August 25, 2014 05:32 PM

What's new in uMap

Here is an overview of the changes made in uMap recently:

The biggest change, even if it's not the more spectacular, is that Leaflet.Draw has been replaced by Leaflet.Editable as drawing engine. The goal was to have more control over the API, have touch support, and have multipolygon/polyline support. For now two enhancements come from that move:

  • It's now possible to continue a line. There are two ways to achieve this: right click on the last (or first) vertex of the line, or ctrl-click on the first or last vertex:

continue a line

  • It's now possible to draw (and edit) polygon holes: right-click inside of a polygon to start creating a hole:

create a hole

Next step is to handle multipolygon and multipolyglines editing and touch support (Leaflet.Editable is ready for that, but uMap itself need a bit more work).

  • when clicking on an element (marker, polygon…), it's now possible to open a panel, instead of the classic Leaflet popup

popup panel

example of story mapping activate slideshow

  • When using a clustered layer, it's now possible to define the cluster text color:

change cluster text color

  • Added a basic GPX and KML download, thanks to togpx and tokml:

download data

  • Until a proper multipolylines support, they are now merged (instead of being skipped) at import

  • A table editor allows to edit all elements of a layer in one view:

table editor

  • sometimes, we want a polygon to act as background, without being clickable. This is now possible trough the clickable option.

  • it's now possible to take control of the popup template using variables. Those variables will be populated dynamically from the elements properties. For example, let's say you have imported a geojson having the following properties: price, name, image, description; by default, only the name and the description will be displayed. But you can now take control of this. Here is an example of template to use with such data:

    # {name}
    {description}
    {{{image}}}
    Price: **{price}**
    
  • Added shortCredit (displayed in the attribution bar) and longCredit (displayed in the caption panel) properties, for more custom captions

  • a basic HTTP concurrency control has been added: if two persons edit a given layer of a same map in the same time, the last to save will be prompted that its changes will erase the changes made by someone else and asked for confirm before really saving:

example of save conflict

  • the "filter" field in the data browser was only filtering on the name property of the elements; this can be now controlled in the map properties

  • added a datalayers parameter, to override which layers will be visible on map load (useful to have different URLs for the same map, or when using the iframe exporter)

  • it's now possible to set a marker lat/lng properties by hand:

set marker latlng

More about uMap on the wiki: http://wiki.openstreetmap.org/wiki/UMap

Good umaping!

by ybon at August 25, 2014 12:57 PM

ШТОСМ

В библиотеке, как культурные

В эту субботу, 30 августа, в Москве «Теплица социальных технологий» с нашей помощью организует мероприятие «Вечер оживших карт». Оно начнётся в 12:30 в Центральной научной библиотеке Союза театральных деятелей на Большой Дмитровке, 34 (ст. м. Чеховская). Возможно, вы помните «Ночь», которую мы проводили в феврале 2012 года: тогда она была просто поводом собраться, поговорить и помапить. В этот раз в программе заявлены выступления и мастер-классы, поэтому опытные осмеры могут задаваться вопросом, зачем им идти.

Для новичков этот вопрос даже не стоит. Обязательно идите. Участники проекта покажут и расскажут, как рисовать и использовать карту, и ответят на любые вопросы по тегам, инструментам и выгрузкам. Это уникальный шанс начать (или продолжить) создание карты, имея под рукой много участников OpenStreetMap с многолетним опытом. Как сидеть в чатике, но с улыбкой, без переключения окон и офтопика про «Доктора Кто». И это эффективнее обычных картовстреч, ориентированных на сбор данных, но не на их отрисовку. Зачем копаться в Map Features, когда любой опытный осмер быстро подскажет, каким тегом отмечать зелень во дворе? Подобная встреча едва ли повторится в ближайшие пару лет, так что лучше прийти, чем потом жалеть.

«Старички» же знают, что со временем на карту остаётся всё меньше времени за другими интересными занятиями: написанием программ, правкой вики-страниц, спорами на форуме. Картовстречи часто оставляют не улучшенную карту, а кипу обходных листов, к которой никак не подобраться. Обновились снимки, накопились свежие треки, и хочется сесть и, как когда-то, уйти в непрерывный маппинг, но всё что-то мешает: то семья, то работа, то сериалы. «Теплица» даёт уникальную возможность, место и время, для погружения в чистое картирование. И, разумеется, со знакомыми из OpenStreetMap и с закуской. Приходите в любое время, не обязательно к началу: библиотека открыта до 22:30.

Если вы планируете, или хотя бы обдумываете участие в «Вечере», зарегистрируйтесь, чтобы организаторы успели подготовить нужное количество стульев и еды. Обсуждение на форуме идёт сразу в двух темах: тут и тут. От вас потребуется ноутбук с мышкой — хотя, вероятно, в библиотеке будут свободные компьютеры.

August 25, 2014 08:08 AM

OSMBlog (German)

Wochenaufgabe Umweltzonen KW 35/36 25.08.–7.09.2014

Umweltzone-Zeichen

Umweltzone-Zeichen (Quelle: Wikipedia, gemeinfrei)

Von Tobias stammt die Wochenaufgabe, die sich zum Ziel gesetzt hat, mehr Umweltzonen in OpenStreetMap zu erfassen.

Seit 2008 existieren in Deutschland Umweltzonen. Diese wurden den europäischen Richtlinien folgend, eingerichtet, um die Grenzwerte für die Luftqualität einzuhalten. Zwischenzeitlich gibt es deutschlandweit in 48 Städten und Regionen Umweltzonen. Als Information für den Bürger veröffentlichen die Städte auf ihren Webseiten größtenteils Bilder, GIS-Anwendungen und in PDF verpackte Karten, auf denen der Verlauf der jeweiligen Umweltzone eingezeichnet ist. Oft sind diese digitalen Informationen schlecht nutzbar, weil der genaue Verlauf mangels hochauflösender Karte nicht erkennbar ist. Ein praktisches Problem ist zudem, dass sich diese Dateiformate auf einem Smartphone schlecht konsumieren lassen. Generell herrscht bei diesem Thema ein großer Mangel an offenen, maschinenlesbaren Daten.

Aus dieser Situation heraus entstand die Idee für die Android-Anwendung “Umweltzone”, die es seit Ende 2013 kostenfrei im Google PlayStore gibt. Die wenigsten Städte veröffentlichen die Geokoordinaten zum Verlauf der Umweltzone digital. Einige Städte verlangen Geld für die Bereitstellung dieser Informationen. Diese Einstellung verwundert, da der Verlauf der Umweltzone eigentlich jedem Bürger zugänglich sein müsste, was durch die offiziellen Veröffentlichungen oft nicht gegeben ist. Daher stützt sich die Anwendung maßgeblich auf in OpenStreetMap eingetragene Umweltzonen.

Im OpenStreetMap-Wiki findet sich eine Liste der deutschen Umweltzonen. Einen guten Gegenvergleich findet man auf der Seite des Bundesumweltamts und auf umwelt-plakette.de.

Der überwiegende Teil der in OpenStreetMap hinzugefügten Umweltzonen wurde als Umkreis eingezeichnet – einige wenige Städte haben die im Bereich befindlichen Straßen entsprechend gekennzeichnet. Ich hoffe dabei in der Diskussion auf einen Konsens für das erste Modell, da diese Variante auch in der mobilen Anwendung genutzt wird. Es gibt zwar eine Diskussion um die richtigen Tags, am verbreitetsten ist aber type=LEZ.

Über Overpass-Turbo kann man sich einen guten Eindruck über die bisher erfassten Umweltzonen machen. Im OpenStreetMap Forum kann über die Wochenaufgabe diskutiert werden. Ansonsten werde ich so oft wie möglich im IRC-Channel #osm-de auf irc://irc.oftc.net online sein.

Falls ihr auf Twitter über diese Wochenaufgabe berichtet, würde ich mich freuen, wenn ihr den Hashtag #osmwa3536 verwendet.

Wie Tobias, könnt auch ihr uns Vorschläge für die Wochenaufgabe zusenden. Wir haben derzeit noch ein paar Ideen auf Halde liegen. Euer Vorschlag wird für uns attraktiver, wenn ihr – wie Tobias – einen fertigen Text samt Overpass-Abfrage beilegt.

flattr this!

by Tobias at August 25, 2014 08:00 AM

"OpenStreetMap.org User's Diaries"

How do I indoor map my building?

Hi, I would like to upload floor plans to create some indoor maps. Is this possible with OpenStreetMap?

We'll map our office as a test and then our client wants to include indoor maps in their app. A point in the right direction would be helpful.

Thanks

Jon Director, ACAProjects.com

by acaprojects at August 25, 2014 02:54 AM

August 24, 2014

"OpenStreetMap.org User's Diaries"

Afrika

Hallo, ich würde gerne in den wenig kartierten Gebieten Marokkos oder Mauretaniens etwas tun. Ich bin mir jedoch unsicher, wie und wo am sinnvollsten zu beginnen.

Hat jemand Tipps?

Beste Grüße Bartgesicht

by bartgesicht at August 24, 2014 10:45 AM

OSM это... (Direct routing thru military objects)

Идиоты обрисовыющие дороги по Bing у военных объектов и позволяющие прямой роутинг по ним.

http://www.openstreetmap.org/#map=14/52.9451/158.4059 вместо тысячи слов

пиздец

пиздец прямо сейчас

by d1g at August 24, 2014 09:24 AM

August 23, 2014

"OpenStreetMap.org User's Diaries"

Benizalón y Benitagla (Almería)

Dos municipios terminados, Benizalón y Benitagla. El siguiente Municipio será Alcudia de Monteagud.

by Canellone at August 23, 2014 06:47 PM

State Parks and National Parks

There is a well-established definition for tagging National Parks in OpenStreetMap.

Hoge Veluwe Nationaal Park Hooge Veluwe National Park Image credit: Wikipedia

The definition,

A national park is a relatively large area of land declared by a government (just as boundary=administrative are declared/recognised by governments), to be set aside for human recreation and enjoyment, animal and environmental protection

would also apply to the many State Parks in the United States, and perhaps provincial and state parks in other countries as well. However, the boundary=national_park page does not give any specific guidance. I am about to map some state parks in my home state Utah, and I will tag them with boundary=national_park for now because they fit the definition.

How do you map state or provincial parks in your area?

by mvexel at August 23, 2014 05:06 PM

Well it finally happened...

Well it finally happened... last night I went to a pub, and I printed out an OSM map to find the way. However, 8 days earlier, someone had moved the pub to the wrong location! That's the kind of risk we run in an open crowdsourced system.

Luckily my beer hunting skills outweighed my trust in open data and I found the pub eventually. Pint drunk, map fixed, crisis averted.

by mcld at August 23, 2014 02:23 PM

ШТОСМ

Праздник со слезами на глазах

В техноблогах начали появляться заметки про десятилетие OpenStreetMap, как наш проект развился за это время и какое светлое будущее нас ждёт. Обычная шарманка про рождение из ничего, из желания Стива Коста сделать свободную альтернативу картам Ordnance Survey, смешную поначалу, но грозную теперь. И у нас, конечно, будет всего больше, сообщество станет мягче относиться к импортам, и настанет всеобщий API 0.7. Какая же бочка чуши.

Прежде всего, посмотрите на карту OpenStreetMap восемь лет назад, от 14 августа 2006 года (спасибо Фредерику Рамму за подготовку базы). Нет смысла искать там свой город: это чистое поле с парой линий в Англии и Дании. Чем занимались участники проекта предшествующие два года? Ну э-э-э, собирали треки. JOSM появился в январе 2006, Osmarender и API 0.3 — в марте. До этого OSM практически был на уровне идеи: почтовая рассылка и вики.

Технически развитие OSM остановилось в 2011 году: тогда уже вовсю работал рендер на мапнике, потлатч 2 заменил первый, продвинутые мапперы использовали JOSM, появились OSRM, Overpass API и Leaflet. С тех пор — только дописывание библиотек, смена дизайна или затухание заброшенных программ. Единственное исключение — редактор iD, часть большого и страшного проекта под названием «Mapbox». Страшный он потому, что может стать нашим будущим.

Что нас ждёт? Раньше я оптимистично бросался названиями типа «год редактора карты», призывал распространять весть об OpenStreetMap в школы. Думал, что вот-вот — и напишут удобные инструменты для отката ченджсетов, для классификации тегов, для сбора данных пешком, на велосипеде и на машине. Реальность такова, что банальное перемещение точек в лучшем редакторе JOSM сделано настолько криво, что пришлось включить в ядро два альтернативных способа (кнопки «W» и «X»). Нет ничего, и ничего не предвидится. Новый OWL заглох, роутинг и overpass на глагне так и пылятся в ветвях гитхаба, про API 0.7 и говорить смешно, даже если не вспоминать слово «полигоны». Кажется, проект окончательно стагнировал, только сотни тысяч участников обводят, рисуют, импортируют, воюют.

Нельзя прогнозировать на год или два вперёд, потому что за это время мы не смогли построить надёжной, предсказуемой организационной структуры (админы — единственное исключение). Можно лишь надеяться. Что кто-нибудь загрузит первые коммиты для нового API в ветку cgimap. Что появится настойчивый участник, который пробьёт стену безразличия и перфекционизма, добавив на osm.org полезную функциональность. Что некоторым программистам начнут платить за работу и требовать от них результатов в конечные сроки. Что у нас появится хотя бы один практикующий юрист, и мы узнаем, не зря ли провели три года в перепалках. Что Mapbox не захватит технологический стек OSM своим джаваскриптом. Что откат ченджсетов станет проще, а ошибки будут валидироваться на сервере.

OpenStreetMap — без сомнения, лучшая карта всего мира. Непонятно, почему его до сих пор воспринимают как несерьёзную поделку, и часто забывают упомянуть в сравнительных статьях. Хотя нет, понятно. Дайте нам ещё десять лет.

Также:

August 23, 2014 11:57 AM

Mapbox Studio

Наконец, вышел первый релиз TileMill 2, недавно переименованного в Mapbox Studio. Сборки для Windows, Mac OS X и Linux можно загрузить с официального сайта. Не удивляйтесь номеру версии: кажется, в компании специально собрали людей, обожающих цифру ноль.

Картинки на сайте очаровательны, и хочется скорее запустить и творить, но программа вас удивит: это уже не просто редактор файлов CartoCSS, как раньше. Теперь всё вертится вокруг векторных тайлов: в них держат не только данные OpenStreetMap, но вообще всё: CSV, шейпы, GeoJSON. Соответственно, вместо прямого подключения разных файлов, понадобится для каждого источника (набора однотипных слоёв) делать свой проект типа «Source», и затем ссылаться на эти проекты из стиля. Разумеется, проекты TileMill 1 и 2 несовместимы, хотя перевести проект на векторные тайлы должно быть несложно.

Также в комплект теперь входят около трёхсот шрифтов от FontShop и Monotype, но использовать их можно только для карт, публикуемых на серверах Mapbox (нужна подписка «Standard», 49$ в месяц). На сайте появились несколько коротких учебников, но документация в целом не улучшилась. Чтобы поместить стиль на векторных тайлов на свой сервер, понадобится копаться в модулях NodeJS.

August 23, 2014 09:09 AM

Рисование домов

Хотя обычно дома рисовать очень просто — прямоугольник с building=yes, — на практике постоянно всплывают какие-то сложности. Danidin9, автор картинок про дома в Петербурге, наглядно объясняет (полные версии — по клику):

Рисование домов переменной этажности обсуждается в большой теме на форуме, некоторые теги объяснены на этой вики-странице. Не забывайте отмечать подъезды точками entrance=*.

August 23, 2014 07:08 AM

"OpenStreetMap.org User's Diaries"

OpenStreetMap Sister Towns

Playing MapRoulette, you get dropped in all these random locations on earth. This really makes my mind wander sometimes.

Here is Cambridge, Ohio:

Cambridge, Ohio Image credit: Wikipedia

A nice enough town. Probably friendly people. BUT NOT VERY MANY MAPPERS AMONG THEM:

Cambridge on OSM

A fellow mapper suggested this simple & brilliant idea: OSM Sister Towns. Why don't we team up towns (and cities) in need of mapping help with towns elsewhere in the world that have mapping forces to spare? They could use the Tasking Manager, MapRoulette, or just iD / JOSM with aerial imagery to help out the town in need. I foresee face to face meet-ups at SOTM and new global bonds of mapping friendship!

by mvexel at August 23, 2014 12:57 AM

If you need something to do...

...check out this area: https://www.openstreetmap.org/edit#map=15/36.1631/-83.4612 There are tons and dozens of streets, that have completely wrong positions. It's just crazy! There is months of work to do.

Just switch on TIGER and Bing layer, zoom in a bit and you'll immediately see...

by Linutux at August 23, 2014 12:13 AM

August 22, 2014

"OpenStreetMap.org User's Diaries"

Oh TIGER...

Sometimes I wonder if those TIGER surveyors were just having a grand old time making up things as they drove past and casually looked out of the window of their cars...

TIGER...

This whole area needs lots of work - if you want to help out, that would be great!

(I have no personal or other connection to the area, but got dropped there by the MapRoulette 'Ways Needing Smoothing' challenge - freshly cleaned up, so most cases you get should now be real things to fix!

by mvexel at August 22, 2014 11:17 PM

Primo giorno di scuola...

Oggi ho iniziato ad aggiungere dettagli alla mappa del mio paese, Chiaravalle Centrale. Ho iniziato con le scuole e gli edifici pubblici. Questo tipo di "lavoro" posso farlo tranquillamente dal computer on-line ma, vista la carenza di strade in Openstreetmap, mi sto organizzando per tracciarne il più possibile. La cosa più semplice era quella di utilizzare uno smartphone con GPS incorporato ed una app che mi conserntisse di registrarne i tracciati. Ho constatato, purtroppo, che il GPS integrato nei vari telefoni non è proprio il massimo quindi ho rispolverato il mio "vecchio" Royaltek RBT-2010 con chipsert Sirf III, utilizzato per tanto tempo con un Nokia Symbian con installato TomTom, ed il problema sembra superato. Sempre oggi ho iniziato ad effettuare alcune prove con varie App e la migliore mi sembra Oruxmaps. Registra i tracciati e li esporta in vari formati ma, la cosa più importante, usa già di default le mappe di Openstreetmap quindi, quando mi trovo su una strada non registrata, me ne accorgo subito facilitandomi la ricerca delle nuove vie.

by Alfer67 at August 22, 2014 11:17 PM

Benizalón (Almería)

Empezando el término municipal de Benizalón.

by Canellone at August 22, 2014 06:41 PM

Uleila del Campo (Almería)

Terminado el término municipal de Uleila del Campo (Almería)

by Canellone at August 22, 2014 06:37 PM

SK53

WWII Bombs in Nottingham : discovering local history whilst mapping

Last Saturday I showed 2 visitors how I mapped addresses (more on this later).


Infill housing, St Cuthbert's Road, Nottingham NG3

One little vignette stood out.


We were walking down St Cuthbert's Road in the area called Sneinton Elements, and the house numbers were a little peculiar at the bottom of the road. Most of the road was terraced houses, but the last terrace was number 10 followed by a detached house at 8, a gap and then finally number 2 which was at the end of a terrace on the main road, Carlton Road. As I wanted to make sure I captured this particular group of numbers I took a few photos. A chap came out of one of the houses wanting to know, not unreasonably, why I was taking pictures of his house.

He was sure I was doing it for money. I explained a bit about OpenStreetMap, and tried also to satisfy him that I mapped as much for my own interest as anything else. As an example I said I was interested what had happened to the original houses 4-8. The answer was unexpected: they'd been destroyed by a World War II bomb.

I knew about some of the bombing in Nottingham, but not very much. At this point we were away. He was deeply interested in local history and could explain a lot about the neighbourhood, even though, as he explained, he was not local. In fact his family came from Hyson Green (about 2 km away) and had perhaps lived there for several hundred years.

I have this sort of heartening exchange now and then when I'm out mapping. Every time I'm still surprised just how interested people are in these things, and how much knowledge they have. In many ways my own interests in these things are a significant motivator for my own involvement in OSM. Of course I dont expect someone in their '70s to necessarily want to get down to the nitty gritty of editing, but I think it does show that map and/or local history geekiness are much more widespread than one might think.

The other thing of note is probably something which is disappearing: my interlocutor had a really precise sense of locality.

 photo scan0059.jpg
Locations of WWII bombs in Nottingham
Image from Nottstalgia forum
Original source and image rights unknown.
I promised my visitors to do a little research arising from this conversation. So here are a few links to material on the bombing of Nottingham during WWII:
  • Wikipedia has a decent overview.
  • Oral history transcribed by the BBC
  • A discussion on a local forum (source of the map above).
The London Blitz has recently had nice digital resources developed to show where bombs fell during the war. The Bombsight project of the Imperial War Museum development team included a significant proportion of OSMers. I've regretted that it was never extended outside London: my mother's family were bombed out just before Christmas 1940, and my grandfather was fire watcher in Liverpool for most of the war. There's lots more old maps left in the National Archives: yet another reason why we are working towards OpenHistoricalMap as a platform to make digitising such data easier.

Once again the link between mapping, old maps and local history had reasserted itself.

by noreply@blogger.com (SK53-osm) at August 22, 2014 03:56 PM

"OpenStreetMap.org User's Diaries"

Как нарисовать ровный домик в JOSM

Кривые дома - известная проблема OSM. Многие используют кнопку q, чтобы исправить ситуацию, но и это не панацея... Давно хотел сделать это, и вот, наконец: некоторые простые приёмы, позволяющие качественно рисовать здания в JOSM: http://img-fotki.yandex.ru/get/6835/51351719.14/0_a6440_b0644cfb_orig

by Danidin9 at August 22, 2014 12:02 PM

Oh, Amaroussi....

This survey goes to eleven!

"We can't pay you to hold a mapping party at a location like that, even if it makes Knightsbridge less rubbish!"

Rest assured that it is not my intention to bankrupt OSM with surveys like this.

by Amaroussi at August 22, 2014 06:36 AM

Me and OSM Update

So I now have a Sat Nav bought in May 2012, which now rely on in the car. Although choose a Garmin model that could use OSM maps by time I downloaded some OSM maps for it onto MicroSD card and tried it with them I then found it did not work as well with them present so ended up removing them.

Since I got a smart phone in Dec 2012 I have used various OSM compatable apps but ended up on just using OSMAND and OSMtracker but tracks not as vital now with availability of Bing aerial background. But still use full for odd path and be road.

I hardly have time to contribute now, but just observe the project from sidelines. Only odd edit now and again. But I was just yesterday showing online ID editor to a collegue who knows GIS well. It will be interesting to see if he starts to contribute.

by %20Bunny at August 22, 2014 01:51 AM

August 21, 2014

"OpenStreetMap.org User's Diaries"

Moje edycje

  1. Białystok/Pietrasze, Podlaskie
  2. Białystok/Osiedle Bagnówka, Podlaskie
  3. Białystok/Osiedle Wygoda, Podlaskie
  4. Białystok/Osiedle Leśna Dolina, Podlaskie
  5. Białystok/Skorupy, Podlaskie
  6. Białystok/Dojlidy, Podlaskie
  7. Czarna Białostocka, Podlaskie
  8. Choroszcz, Podlaskie
  9. Baciuty, Podlaskie
  10. Roszki-Wodźki, Podlaskie
  11. Krzyżewo, Podlaskie
  12. Poświętne, Podlaskie
  13. Łapy-Osse, Podlaskie
  14. Pruszanka Stara, Podlaskie
  15. Brzozowo Stare, Podlaskie
  16. Brzozowo-Chrzczony, Podlaskie
  17. Brzozowo Panki, Podlaskie
  18. Płonka Strumianka, Podlaskie
  19. Płonka Kościelna, Podlaskie
  20. Płonka-Kozły, Podlaskie
  21. Gąsówka-Skwarki, Podlaskie
  22. Płonka-Matyski, Podlaskie
  23. Sokoły, Podlaskie
  24. Wysokie-Mazowieckie, Podlaskie
  25. Mazury, Podlaskie
  26. Brzóski-Falki, Podlaskie
  27. Łapy-Kołpaki, Podlaskie
  28. Łapy-Plusniaki, Podlaskie
  29. Łapy-Kołpaki, Podlaskie
  30. Łapy-Osse, Podlaskie
  31. Łapy Dębowina, Podlaskie
  32. Łapy, Podlaskie

by DanielM86 at August 21, 2014 04:54 PM

Importing 1 million New York City buildings and addresses

As of June, New York City buildings and addresses have been fully imported to OpenStreetMap. While we are tackling remaining cleanup tasks I wanted to share a full recap of the effort. I am very happy with the overall result. There are lessons to be learned here from what went well but also where we could have done better - read on for the details.

More than 20 people - volunteers and members of the Mapbox team - spent more than 1,500 hours writing proposals, discussing, programming, uploading, processing and reviewing. Between September 2013 and June 2014 we imported 1 million buildings and over 900,000 addresses. We fixed over 5,000 unrelated map issues along the way.

Here are screenshots of the resulting work:

Building coverage on Manhattan island, the southern tip of the Bronx to the northwest and Wards island to the right.

JFK airport buildings in Queens, bordering on the Hamilton Beach neighborhood to the left and South Ozone Park to the north.

Coverage around Battery Park and Wall Street in Manhattan. This is an area that already had many buildings. We filled in the gaps and replaced buildings where the New York City data set was clearly better.

We imported over 900,000 addresses. Here is an example of the Park Slope neighborhood in Brooklyn.

Buildings contain height information and render nicely as seen here on this example of downtown Brooklyn on Fmap.

The import covers all of New York City's five boroughs

Overview

This is a full writeup sharing my experience with the New York City import in the hope that there is one or the other valuable lesson, good idea, or line of code for you to walk away with. Note that this post is very specific to the work in New York City. If you're planning to do an import, make sure to check out the Import Guidelines for a more universal checklist of how to go about imports.

If you're looking for the 30 seconds version, I'd summarize my take aways like this:

  • Importing is a lot of work, make sure you have the time to commit.
  • Be prepared to continuously improve your conversion scripts and already uploaded data throughout the import.
  • Importing is a skill. It looks easy at first, but everyone involved uploading will need proper support, advanced knowledge of mapping practices and data validation by peers.
  • Involve community where possible, clear and frequent communication is clutch.
  • Invest in your tools

Read on for the deep dive.

OpenStreetMap as a collaboration space for citizens and government

Using New York City's data for OpenStreetMap became possible thanks to the then-mayor Michael Bloomberg's open data policy. Local Law 11 of 2012, releases all New York City government data "without any registration requirement, license requirement or restrictions on their use" (23-502 d). This effectively puts the data in the public domain, making it compatible with OpenStreetMap's contributor terms.

Both, address point data and building data fall under this law and are available for download on New York City's open data web site:

The way we used this data in OpenStreetMap is an illustration of how Bloomberg's plan to stimulate the economy with open data is starting to pay off. This data in OpenStreetMap is now benefiting everyone using OpenStreetMap and this includes the New York City based startup Foursquare which is using OpenStreetMap data on its Mapbox powered maps.

But the relationship between OpenStreetMap and New York City should be ideally a two way street. How can the creator and maintainer of the building and address datasets - New York City's GIS department - benefit directly from their work being imported in OpenStreetMap? The vision of edits in OpenStreetMap directly helping improve a crucial government dataset is very promising. OpenStreetMap is a unique data collaboration platform while datasets like building or address catalogs are incredibly hard to maintain - even for a large municipal government like New York's. How can government become a part of OpenStreetMap?

OpenStreetMap's share alike license means that OpenStreetMap data can't be taken over directly into New York City public domain datasets but we can use OpenStreetMap to find out where changes happened. We set up a daily change feed flagging modifications to buildings and addresses to subscribers. Here's a copy of a change notification email how New York City GIS receives it every day:

Daily change notifications from OpenStreetMap, flagging building and address changes to New York City government.

The notification contains a list of relevant changesets from the previous day with a link to each modified building and address. We are right now assessing the utility of these emails. Another way of leveraging OpenStreetMap as a change signal would be to periodically extract all building and address data and identify all changes in a certain time frame at once.

All code powering the change feed is available as open source on Github. If you'd like to receive the New York City change feed notifications, please let me know. Happy to subscribe you.

Import procedure

To import New York City data we had to convert it to OpenStreetMap format first and cut it into byte size chunks so we could review and import it manually, piece by piece. Once it was imported, a different person than the original importer would validate the data. This means reviewing it for errors and cleaning it up where needed.

Selecting a task on the tasking manager, opening existing OpenStreetMap data and opening importing data in JOSM.

Each participant would set up their workspace according to documentation we provided on Github. In the same document we laid out the actual import procedure. Some of the key items of the import procedure were:

  • Use a separate import account
  • Run full JOSM validation, fix all conflicts with existing data
  • But also fix all existing unrelated issues in area
  • Spot check data - for instance, do street names line up?
  • Merge POIs where appropriate
  • In case of duplicate data, keep the best data if there is a clear difference. In case of any doubt, keep the local data.
  • Add a note where a local mapper could solve a problem

As we imported, we ran into a series of recurring issues that we shared in a common issues guide - a useful resource for training new mappers and agreeing on fixes for unclear situations.

Community import or not?

From the beginning, the import was planned as a community import. There is no standing definition of this practice, but the rough idea is that uploads to the map would be done predominantly by members of local community familiar with the areas uploaded. Once started into the import, we quickly ran into a series of issues.

For Mapbox data team members participating in the import full time it was very easy to outpace local volunteers by a huge factor. In addition, I underestimated the complexity of the actual review and upload work. While not hard, there was a certain learning curve which meant that every new individual joining required significant training and support to get started - which meant plain and simple time that someone had to spend. Add to this that the individual time commitment is huge. I estimate we spent about 1,500 hours among everyone involved - and this is on the conservative side. Assuming 20 people work on the import, each one of them would look at 75 hours on this project. Very few people spend this much time on OpenStreetMap in a year.

The pace of uploads turned out to be key friction point. At the same time a series of data quality issues arose. This is why a couple of months into the import the loosely formed group around the project including community members and myself decided to pause the import and when we restarted a month later, slow it down and stop billing it as a community import. This would allow everyone to participate better and it would set expectations straight as to who was doing the uploading work. I think this adjustment was a good one. Overall it took us 10 months to get the job done - longer than I thought but still a pace that I was comfortable with to commit help finish the job. In the end a vast majority of uploads, validations and programmatic updates were done by the Mapbox team and I'm glad we had the opportunity to contribute.

Still, community involvement was clutch. The incredible input everyone gave, the many reviews, advice and personal time people invested was crucial to make this import a success. Everyone weighing in has helped make the resulting map better.

Sh** happens

We dealt with data corruption and conversion script bugs all using Github issues. Over the course of the import, we opened and closed 120 issues flagging suspicious data found in data reviews and sometimes working through protracted problems with New York City's head of GIS directly chiming in and helping interpret data correctly.

Some of the issues we discovered required updates to data we already imported. Once we were into the import even a couple of days, updating existing data manually quickly wasn't an option anymore. This is where automated edits came in, updating OpenStreetMap data programmatically. We captured all scripts for automated edits in the same code repository as the data conversion scripts. Some examples of programmatic updates are:

  • We fixed wrong tagging on school buildings where we tagged amenity=school instead of building=school.
  • We added ordinal suffixes like "th" in "4th".
  • We expanded abbreviations we had overlooked like "Ft" to "Fort".

We prepared this import well and we had good peer reviews on the imports list running up to the first uploads. We could head off many issues before we started importing. But in the end, the amount of issues we encountered after we started was still an unpleasant surprise. Having gained a lot more experience with this import I am sure the next time we can avoid a series of pitfalls - but the need for being able to programmatically update data after it's been uploaded is crucial for a successful import. You simply cannot plan for all eventualities and you need to be prepared to apply fixes as you go.

From this perspective, the next time I would want us to write data integrity tests from the get go. These tests would assert data quality on data before it is uploaded. This would allow us to be much more agile in updating and refactoring conversion scripts as we go.

Another set of tests would assert data quality of already uploaded data. This would help to identify existing systematic problems and catch data issues due to negligent uploads fast.

So far, we have a rudimentary directory with validation scripts we started to build up during the import. There is a real need across the OpenStreetMap community to further develop and share easy to use tools to test and validate data. What if we could reuse the validators available in JOSM from the command line on arbitrary portions of OpenStreetMap data?

Data processing

To get source data ready for upload, a conversion script would download the data, split it, convert it and store the resulting files in OSM XML format on Amazon S3. We set up a tasking manager job that would expose each file as a task for people to import. To upload a dataset, a mapper would select a task, download OpenStreetMap data and load OSM data. We used the excellent JOSM editor to merge and review data before uploading to OpenStreetMap.

The entire data processing script is captured in a Makefile and can be run from download to upload to Amazon S3 with a single command. In sequence, the processing script would perform the following actions:

  • Download and unpack buildings (polygon data in shapefile format)
  • Download and unpack addresses (point data in shapefile format)
  • Reproject and simplify building geometries
  • Reproject addresses
  • Split buildings and addresses into byte size chunks
  • Merge: Where only a single address is available for a building, merge the address attributes onto the building polygon.
  • Convert: Map attributes to OpenStreetMap tags, convert street name formatting and house number formatting and export in osm format
  • Put to S3

All code is open source under a permissive BSD license - feel free to lift where convenient.

Repeatable conversion

The conversion script is repeatable with a single command and it is organized in stages: Each significant processing step creates files on disk and can be run separately. All that's needed are the output files of the previous processing stage. Running the entire script would take on the order of several hours on an extra large Amazon EC2 instance. Being able to run steps like the merge stage or the convert stage separately was saving important debugging time. Throughout the import, we wound up reprocessing the data countless times as we fixed issues.

# Download, convert and push to s3
make && ./puts3.sh

# Download and expand all files, reproject
make download

# Chunk address and building files by district
make chunks

# Generate importable .osm files.
# This will populate the osm/ directory with one .osm file per
# NYC election district.
make osm

# Clean up all intermediary files:
make clean

# Put to s3
./puts3.sh

# For testing it's useful to convert just a single district.
# For instance, convert election district 65001:
make merged # Will take a while
python convert.py merged/buildings-addresses-65001.geojson # Very fast

Reprojecting and simplifying

New York City data comes in its own special projection and it is way too detailed for OpenStreetMap, so we reprojected and simplified it using ogr2ogr:

ogr2ogr -simplify 0.2 -t_srs EPSG:4326 -overwrite buildings/buildings.shp buildings/building_0913.shp

Splitting into byte size chunks

We couldn't upload all data in one go, it had to be cut into byte size chunks for manual review and upload. For splitting up the data we used New York City voting districts. This was an arbitrary choice, it just so happens that New York City voting districts are of a manageable size for manual uploads. There are 5,285 voting districts, the processing script generated an OSM file for manual upload for each one of them. The script chunk.py uses the great Shapely and Fiona libraries for doing this. It is nicely reusable for any task where you need to split up one geospatial dataset by the polygons of another geospatial dataset.

Merging

In OpenStreetMap, addresses tend to be merged onto building polygons where only one address is available for the building. We wanted to follow this convention and thus merged addresses where only one was available onto the corresponding building. The python script merge.py uses Shapely, Fiona and Rtree to do this. The script also converts data into geojson format - which was extremely useful for debugging as we could inspect them in any text editor. Here is an example output file of the merge stage.

Most of our fixes during the import happened on later stages so we could always work off of the merged files, saving about 50% of the total processing time.

Conversion

This is where most of the actual conversion is happening - this is also the part of the script that was the most significant time investment. It captures the full complexity of the conversion and handles hairy problems like house number conversion, street name conversion, cleanly merging geometries, generating multipolygons and more. The script convert.py uses Shapely and lxml for attribute mapping and exporting data in OSM XML format. OSM XML is directly readable by JOSM, so the resulting files of this stage could be opened and directly uploaded to OpenStreetMap with JOSM.

One tricky problem we're solving on this stage is merging T-intersections. OpenStreetMap's data model is unique in that it allows for sharing vertices between polygons. In the picture below, you see a typical T intersection. The node with the arrow is supposed to be part of the two ways describing the corner of one building but also part of the ways describing the straight walls of the other building.

It took us a while into the import to notice unmerged T-intersections. What makes this issue vexing is that OpenStreetMap's native decimal precision is lower than our source data. The result was that data we uploaded to OpenStreetMap looked fine, but once we downloaded it again it came back with truncated precision, moving nodes just far enough to place some within neighboring buildings.

Nodes on T-intersections between buildings need to be part of both buildings.

Our conversion script merges all incidents of T-intersections. This requires truncating decimal point precision to OpenStreetMap's native 7 positions and buffering - the technique to test not only whether a point sits on a line, but whether a point is in the close vicinity of a line. Read up on appendBuilding() in convert.py for details.

Pushing to S3 and exposing the data in the tasking manager

For exposing tasks to mappers we used the OSM Tasking Manager - a great tool for coordinating mapping tasks among large groups of individuals. We used a patched version that allows for tasks shaped as arbitrary polygons - instead of the usual squares. Each task polygon pointed to the file we've made available on s3, and the tasking manager exposed two buttons: one for loading OpenStreetMap data into JOSM, the other one for loading the import data into JOSM. We labeled those buttons "JOSM" and ".osm" which doesn't make all too much sense, but hey!

Loading data into JOSM from the tasking manager.

Reusing and the elusive import toolchain

Writing these scripts we avoided overthinking the problem. Creating generalized solutions for these functionalities is hard and we simply didn't have enough data points to do so. Now having gone through this import, I see a couple of opportunities to solidify a toolchain for import:

  • Generalize a command line script for splitting data (like a properly abstracted chunk.py)
  • Generalize a library for converting Simple Features to the OpenStreetMap data model, including XML export
  • Consider using PostGIS - I avoided it intentionally here, but built in spatial operations and indexing is appealing
  • Identify a pattern for reusable validation scripts that can be used to assert data quality before and after uploads

Continuously improving the map

Here is the full time line of the import:

We are not done yet. While all data has been imported to OpenStreetMap, there are final cleanup tasks we are tackling as we speak. Help us further improve the map: if you find a building or address related issue on the New York City map, please let us know by filling an issue on Github. As soon as new data is available from New York City, we will also take a look at updating OpenStreetMap where it makes sense.

Thank you

Huge thanks to all who have helped make this import happen. Through your work reviewing, coding, organizing mapping parties and doing data uploads you have helped make this import better than it would have been without you: Serge Wroclawski, Liz Barry, Eric Brelsford, Toby Murray, Ian Dees, Paul Norman, Frederick Ramm, Chris MacNally, and many others. A special thanks to Colin Reilly from New York City GIS who has helped on many occasions fully understand the source data and find the best decision translating it to OpenStreetMap. A big shout out to my colleagues who've put a ton of work into this endeavour: Ruben Lopez, Edith Quispe, Aaaron Lidman, Matt Greene, and Tom Macwright among others. Say hello if you bump into them on the internet, or maybe at one of the next conferences.

Cheers to making the best map in the world.

by lxbarth at August 21, 2014 02:50 PM

My first day on OMS

This is my first day on OpenStreepMap. I'm staying in Philippines at the moment, and because OSM was helpful in the past I'm trying to contribute as much as possible.

by NRgizR at August 21, 2014 02:18 PM

How to prepare for your first mapping party!

So, a friend invites you to a mapping party...

Or, an NGO you really respect asks for help mapping the area they are working in....

Or, you see an article calling for mapping volunteers and sign up...

But, you've never mapped before!

What do you do?

It's easy. Just remember this catchy acronym: HPSSU

on flickr

Firstly, get your HARDWARE ready. This is a posh way of saying make sure your laptop is working and, if you have one, your mouse is in your bag (a wheelie one is best)

Secondly, create your Open Street Map (OSM) PROFILE (do that here - it takes almost no time at all)

Thirdly, select your preferred SOFTWARE. As a complete beginner, the iD editor is the easiest to use. You don't need to download anything, it's all available online. No fuss. However, the java based editor, JOSM, is also an option. You'll need to download and install it (do that here), but once you have it, it is well worth it. It is very easy to use, makes editing much faster and has much more scope than iD if you continue to develop as a mapper. Having said that, iD is perfectly fine for your fist time!

Lastly, SWATTING UP will help. If you have an hour or two before you come to do a bit of online training, you will find it that much easier at the mapping party. There are loads of resources available for OSM. A few from the LearnOSM project are listed below, but a search on your preferred engine will bring a myriad of interesting results. Go explore!

by pedrito1414 at August 21, 2014 10:01 AM

Делим страны

Иногда возникает необходимость разделить страну/регион/область на примерно равные части. Для этих целей сделал себе небольшой скриптик. На вход подаем osm-файлик с отношениями регионов

divide-country.py -f input.osm

А на выходе получаем две половинки в виде списков отношений

0: 350340, 350424, 350459, 350585, 351063, 353779, 354089
1: 350144, 350303, 351246, 351379, 352449, 352454, 353776, 353812

Алгоритм достаточно прост, и идеальное разбиение врядли получится. Но если нет повышенных требований к точности, то это вполне себе альтернатива ручному труду

Вот так, например, будет выглядеть Краснодарский край в двух частях: Краснодарский край Или четыре кусочка Московской области: Московская область

by gryphon at August 21, 2014 06:41 AM

Routing error in San Fernando, Cadiz, Spain.

Hi,

On my holiday I found a very illogical route using Osmand. Now that I am home I see that OSRM is doing the same: not taking the on-ramp to the highway.

http://osrm.at/94f

When I inspect the map with iD, nothing seems wrong with connections, oneway or nodes. ITO maps also doesn't show recent edits in that area so the problem should still be there:

http://www.itoworld.com/map/127?lon=-6.18710&lat=36.46896&zoom=17

How to progress now to fix this?

Thomas

by jutezak at August 21, 2014 06:06 AM

Midwest/Plains Trip - Secondary Routes and Street Names

Spent the last few days after vacation going over my OSM notes from the roads of the Midwest and Plains (trip to the Black Hills of South Dakota).

I noted some of the differences in naming and signage conventions between counties, especially secondary level numbered roads (ref=) and street names (name=) in counties with mile after mile of PLSS grids. Many of these differ from TIGER, and will have been renamed, especially for E911 purposes. On the way back I even saw two separate name signages, one the old street names in town, and the other a new grid naming system.

Pennington County, SD

Blade signage:

White on green, standard pentagons on end of sign. No county numbers given, only blank pentagons with county outline on blade signage, and also logos for USFS or city maintained roads.

Route markers:

County roads are marked with (unnumbered) pentagons reading “Begin Pennington County Road”

Street grid:

Number grid system in the flat parts of the eastern county. Was driving, so no further details.

Deadwood, SD

Had brown street name signs for walkways within Mt. Moriah Cemetery. Standard green in the rest of town.

Custer County, SD

Blade signage:

White on green, standard pentagons on blade signage for county, also logos for USFS or city maintained roads.

Route markers:

Small vertical markers, white on green, “CUSTER CO ### CS”, also some National Forest markers.

Street grid:

No system to speak of, too mountainous to have an obvious grid system, except in the towns.

Crook County, WY

Blade signage:

standard with road names only

Route markers:

Standard pentagons.

Street grid:

None observed, no numbered names. US/State Highways had no other names.

Rest of South Dakota in grid areas

Street grid:

Streets EW, Avenues NS, numbers increasing eastward and southward.

Kansas overview

Blade signage:

Most areas small, white on green. Small variations between counties. Route markers: No county shields noted along US 36 corridor where I traveled.

Street grid:

Varied by county. NOTE: roads near/on county lines are named by both counties in their respective grid systems.

Brown County, KS (Hiawatha)

Street grid:

  • EW Numbered Streets from 100th in the south to 340th in the north, increasing 10 per mile. Intermediate streets ending in 5 as well.
  • NS Roads, alphabetized proper names with one or two grid streets beginning with each letter, increasing eastward. Intermediate roads share the first letter as the full grid roads. (example Goldfinch, Hazelnut, Horned Owl, Jackrabbit, Kestrel.)

Jewell County, KS (Mankato)

Street grid:

  • NS Roads Numbered, 10 Road to 310 Road increasing 10 per mile eastward.
  • EW Alphabetized Roads southward
    US 36 was P Road.

Marshall County, KS (Marysville)

Street grid:

  • NS Roads Numbered, increasing 1 per mile eastward.
  • EW Alphabetized Roads southward
    US 36 was Pony Express Highway along this route, but would in the grid would be a name beginning with K.

Nemaha County, KS (Seneca)

Street grid:

NS Roads lettered, EW Streets lettered. Increasing south and east. Letter and number for intermediate road. Observed an M4 Road halfway between M and N. along US 36.

Republic County, KS (Belleville)

Street grid:

  • NS Numbered Roads, 10 Road to 310 Road increasing eastward. TIGER may have these as “County Road 1“ through “County Road 31“
  • EW Lettered Roads A Road to Z Road, increasing southward. TIGER may have these as “County Road A“ through “County Road Y“

Smith County, KS (Smith Center)

Street grid:

  • NS Lettered Roads, increasing eastward, A-Z,AA-EE.
  • EW Numbered Roads southward, 10 per mile..
    The geographic center of the contiguous USA is at the intersection of 130 Road (K-191) and AA Road.

Washington County, KS (Washington)

Street grid:

  • NS Alphabetzed Roads, increasing eastward, including AA-CC after Z.
  • EW Numbered Roads increasing northward
    US 36 was named 19th Road along much of the trip before turning SW diagonal, then taking up 17th Road.

Nebraska

Street grid: System varies by county, mostly saw numbered.

Minnesota along I-90

Route markers:

White squares along I-90 until reaching the eastern half of the state, then pentagons.

Street grid:

Avenues NS, Streets EW, increasing 10 per mile northward and eastward. Statewide?

Iowa

Blade signage:

In the town of Castalia, there were still old blades with the Tiger name (Merrill St), and a new set of number-based grid (128th Street)

Route markers:

Standard pentagons. CR Numbering system is Axx-Jxx for EW routes, and Kxx-Zxx for NS. In the northeast corner, EW were A-B prefix and W-X for NS. The central part of the state reaches further east, and this is where the Zxxs are located.

Street grid:

Streets EW, Avenues NS, incrementing upward 10 per mile westward and northward.

Ohio Turnpike overpasses

Route markers:

None seen until Portage and Warren Counties. White square marker Street grid: Signs on overpasses had both standard name and grid-based number, e.g., “Brown St CR 500 N”

by mdroads at August 21, 2014 04:31 AM