Making the creation of before/after comparisons for OpenStreetMap more accessible
If you pick dates close to each other, they also make for a great “find the differences”-puzzle.
tl;dr: One can now create before/after maps of OpenStreetMap with osm-mapping-party-before-after right in the browser, thanks to MyBinder. Just click here to launch it.
For a good 6 months now, I’ve been contributing to OpenStreetMap virtually every day. One of the things that is very addictive about it is the rapid feedback, as one can immediately see the effect of an edit show up on the map. And of course, in regular intervals those changes find their ways into great apps such as OrganicMaps.
What can be a bit harder to see is the cumulative effect made of edits over a longer distance of time. Of course, not just based on one’s own edits, but the joint edits by all contributors that are active in a place. For a very long-term view, OSM: Then & Now provides a comparison between a point in 2008 to now for the whole world, while baato’s before/after allows to make comparisons for small slices of the world (think city-level) for a user-chosen year to now.
If one wants to make more targeted comparison, down to the second-level for the before and after date, Amanda McCann wrote a very comprehensive, bash-based pipeline. It takes an OSM history file, the before/after timestamps, the bounding box one wants to have displayed in a (static) map, as well as the zoom levels of interest. With this it creates both png-based, side-by-side comparisons as above, as well as animated gifs.
Unfortunately, being able to create such comparisons locally means needing to have all the prerequisites for rendering your own map tiles installed. And as for any complex software project, this means installing a ton of dependencies, which in turn come with their own dependencies, which in turn… …you get the idea. Trying to get all of those set up, and avoiding to run into incompatibilities, isn’t for the faint of heart (or at least that’s how it feels to me).
So when I started to try setting it up for myself, I decided to do so in a Docker container. On the one hand, to avoid me accidentally breaking my operating system. And on the other hand, to make it a bit more predictable and reusable for others. Thanks to lots of feedback by Máté Gyöngyösi, that approach now works quite well. I also made a ready-to-use image under gedankenstuecke/osm-mapping-party-before-after:v1.
While this approach side-steps the trouble of having to locally install the whole tile generation pipeline, it replaces it with the need to install Docker, having to connect into the container, get the files out, … In other words, it’s maybe easier but still far from easy.
Which is why I tried to package this approach up into a Docker container that can run on MyBinder, which allows running containers in ‘the cloud’, without having to install anything locally. The way MyBinder works is that it launches an ephemeral container for the user, based on a GitHub repository that is build into the container. The container then launches a web-interface that the user is being forwarded to, to interact with the custom-build environment right through the web browser.
I decided to use the standard JupyterLab interface as the web-entrypoint for making this work. To do so I expanded the image I had already used as a starting point (the postgis image), added JupyterLab to it, and modified the entrypoint to launch both the needed Postgres database & the JupyterLab server. Once that worked, I wrote a little Python-based notebook that makes use of the ipywidgets to allow people to pass the necessary parameters before creating the actual images.
Combined, all of this means that one can now click this link and it will automatically launch the container through MyBinder, and even open up the notebook. All that’s left is first uploading the right OSM history file from Geofabrik, and then one can enter the parameters and create the comparison images.
When to use which approach?
Compared to my prior, local Docker solution there’s some benefits and some drawbacks: The most obvious benefit is that the MyBinder solution doesn’t require installing anything locally, as it all happens in a browser. So there’s not even a need to install Docker.
The main drawback of using the Binder approach comes from the fact that the virtual machines that MyBinder provides are ephemeral. I.e. None of the files that are uploaded into it or created within it are stored long-term, as the virtual machines are being shut down if not used for some time. As a result, creating a new comparison image after starting a new Binder will take a while. This is, because it first downloads a lot of external data which openstreetmap-carto needs. Similarly, the OSM history file that needs to be uploaded as the basis for creating the images will have to be re-uploaded.
In contrast, the local container is made to store the external data, how it has been processed, as well as the history file for the long-term, making it easier to return to creating more images later on.
So, when to use which approach? I’d say if you don’t want to install anything and/or don’t plan on returning to creating maps regularly, using MyBinder is a great way to get started. But if you’ll regularly create updated images, then figuring out the local Docker setup might be worth it, to avoid waiting every time.
Lastly, I’m sure there’s still potential to improve both approaches, as I’m extremely far from being an expert in any of these technologies. 🙈 So, if you have ideas for that, please have a look and maybe even leave a pull request.
ps: Thanks to an invite by Daniel Bellomo, I gave a talk about citizen science and ways of creating community knowledge (including OpenStreetMap & Panoramax at Universidad Nacional de Río Cuarto last week. You can find my slides online.
