About Shoaib

Shoaib Burq is a geospatial engineer based in Berlin and Melbourne depending on the time of the year.

Humanitarian OpenStreetMap Mapping Party, Brittany France

Beg Meil Mapping Party
Photo by Anna Hodgkinson

An email to the HOT list from Nicholas Chavent earlier this month (March 2011) announced the opportunity to participate in a Mapping Party for the Humanitarian OpenStreetMap Team – yep a “HOT” Mapping Party. Many OpenStreetMap enthusiasts would be familiar with the notion of a Mapping Party. Mapping parties are local social events with a focus on having fun while mapping for OpenStreetMap an open licensed map of the world. But what’s a HOT mapping party? HOT mapping parties are for OSM enthusiasts who are also interested in humanitarian mapping. We all know maps form a critical part of disaster response. But how can HOT and OSM be better prepared for the eventuality of assisting during a disaster? Disaster mapping in contrast to general OSM mapping has some specific requirements.

Disaster response must incorporate good geographic data collection practices:

During disasters mapping serves to allow quick and accurate decision making. But there can be no maps without any data. So good disaster response workflows incorporate good geo data collection practices. The HOT Kit is designed to do just this. Over the last year the HOT team has been working in Haiti and have continued to refine and add to a HOT Kit. HOT Kit merges traditional practices of disaster response coordination with the best that the open source geospatial tool-set has to offer.

In the event of a disaster mapping sits close to the planning, support and coordination units. Map-makers working in these efforts must be ready to answer any number of questions thrown at them by the coordinators of the disaster response. Thus being organized and knowing where everything is, is critical, an important goal of the HOT Kit. The kit is organised as five top-level folders under which sit almost anything a HOT team member supporting an operation would need.

Steps during a response will include data collection, cleaning and merging and product creation. The product creation may involve analysis and summarizing. Supporting the operation means having all the data and analysis at your finger tips. In addition it requires a systematic procedure for collection of geographic data and its inclusion into the map making process.

Hot Kit systematizes the steps of map making during disasters:

So the five root directories of HOT Kit are

00_DATA/         <--- geo data ready for use
01_SURVEYS/      <--- data collection
02_PROJECTS/     <--- desktop GIS projects (QGIS being default)
03_PRODUCTS/     <--- products like pdf, shpfiles, garmin files, kml
04_SUPPORT_DOCS/ <--- a kitchen sink of documentation

OSM community must get to know and help refine the HOT Kit: As the HOT Kit gets used it will continue to change and improve. HOT Kit must be able to handle changing requirements based on the kind of project or disaster its being used in. It also needs to elegantly support users of varying skills. Lastly it needs eyeballs. To help folks quickly getting acquainted with the HOT kit I have just published a ruby gem: https://github.com/sabman/HOT-Kit-Generator

HOT Kit Generator is only a day old gem written during the drive from Brittany to Paris. Its aimed at quickly creating a fresh HOT Kit for a new deployment. It currently only stubs out the folders needed during a HOT deployment. This is done based on a configuration file that defines the directory structure of a HOT Kit. My vision for this generator is to allow pulling distributed web and data resources using a single command to create an up-to-date HOT kit in seconds. Allowing different configurations depending on the disaster type, geographic region or project goals or length.

OpenRouteService.org:Home

Give it a try and feel free to fork to get started on some of the TODO items. I would also appreciate any feedback on the idea itself.

On-ground surveys and OSM tags:

Once a HOT team is in the field and collecting data, they should be making the most of the opportunity. This means collecting details that would only be available to someone in close proximity to the resource being mapped. This is where the HOT tagging and forms come in. Over the last few months a number of HOT team members have been developing forms for the OSM Humanitarian Data Model. This along with the Humanitarian Data Model presets in JOSM can make the task of data entry after a survey go a lot smoother. Nicholas Chavent’s work on HDM is particularly impressive.

Role of remote mappers and tagging:

In an emergency the focus is on saving lives. Depending on the kind of disaster the tagging scheme will vary but the regular tags in OSM aren’t always compatible with the needs of first responders. As the HOT team matures so will their ability to engage with the regular OSM mappers who are willing to help support disaster mappers often using satellite imagery. As an arm-chair mapper, instead of mapping for fun you are mapping to create raw data which will form input into products that can give actionable information to first responders.

So where does Haiti-like remote mapping fit into a disaster response. Mapping a disaster from home can be a very useful activity. Useful to the first responders, provided some common sense is applied to the process. For one thing mappers who want to assist in the work of HOT should look at the current mapping priorities here: Current Humanitarian OSM Team Projects Then its worth noting that the mapping you do from home can give first responders some very useful base data. In the case of Haiti it was location of IDP camps, destroyed buildings, blocked roads and landslides. These are all things that first responders can take and plan their missions. It should however be noted that its possible only if the imagery is of high enough resolution. Haiti was and still is, an exception, where sub-meter post disaster imagery was made public and as a result the first-responders were able to save many lives. But we have yet to see such high resolution and clear images for other disasters. So when mapping from home its better to err at the side of caution and not tag an item if you are unsure. Having said that there is no doubt that some data is better than no data and so getting as much of the base infrastructure such as transports, hydrology, admin and populated areas mapped is a good thing.

Long live the HOT Mapping party:

Finally one man without whom the HOT mapping party in Beg-Meil would never happened was Roderic Bera. He was behind the smooth running of the weekend, having organized everything ranging from food to accommodation to transport – it was all very well thought out. Thank you Rod. Then of course Nicholas and Fredric worked late into the night on Friday to get everything to do with the mapping ready. Fred’s mock reports of a humanitarian crisis in Brittany and Beg-Meil read like a true forced migration.

Thanks to all who came and made the HOT mapping party possible, in order of appearance:

  • Nicholas Chavent
  • Roderic Bera
  • Frederic Bonifas
  • Manuel Robert
  • Maxime Leguillier
  • Anna Hodgkinson
  • Joseph Reeves
  • Rodolphe Quiedeville

A special thanks to Manuel for letting me hang at his place in Paris on Sunday night. Good luck to him and Maxime in Haiti. Until next time.

Building a real-time mapping app at Random Hacks of Kindness Berlin

This is a light hearted look at a great weekend of hacking at the Berlin Global Random Hacks of Kindess held at Betahaus, Berlin on the 4th of December 2010

No doubt about it – Berlin has a fantastic tech scene and this weekend it hosted the 2nd Global Random Hacks of Kindness. I had a slow start to the morning so missed out on the introduction to the problem statements. But when I arrived I began looking around for projects to work on.

Having looked over the problem statements I had thought I’d be working on Risk in a Box – a project that we had done some consulting work for in the past. However, upon arrival I had a look at the problems that were proposed by local orgnizers. One that stood out due to its technical challenges was Caritas Germany’s field mapping system .

A team was contemplating taking on this problem and they gladly allowed me to join them, despite the fact I spoke no German. More on that later.

The arguments

Brainstorming the idea further led to the expected friction between team members. But everyone was very civilised – except maybe when I proposed the database – I heard some German words fly around. The words either expressed extreme complements or the opposite :-) NoSql will do that.

1. What are we building?

Our first argument was over what we were building. Was it a web application, was it a database, was it a native mobile app, was it a mobile web app with an HTML5 (local storage)? The argument was resolved with an agreement to build a core API with a database backend and have loosely coupled clients on top that could work in offline mode if needed. For the purpose of the hackathon we would build a mobile client.

RHoK Berlin

2. What mobile platform?

As alluded to above, eyebrows wrinkled and tempers flared (just kidding) with iphone vs. android vs. html5 (local storage). Given that we had an iPhone developer in the team (Engin Kurutepe) we decided the html5 and android client could easily be built later but for the proof of concept we would build an iphone client. This client would update the database via the API in real-time using the REST web service and have a local storage for off-line operations. Another client would be build for the office or operations room of an emergency coordination center. This would show the field updates in real-time using websockets.

3. What database?

Then came the question about what database to use. It had to support some degree of geospatial data and indexing. We could have gone with PostgreSQL+PostGIS or Sqlite+Spatialite or even MySQL but I intervened. As if we hadn’t already exhausted all available buzz words I suggested mongodb. Yes, thats right nosql. After some mumbling we agreed there were advantages to an document storage – especially when the data model was somewhat nebulous.

4. The data model

What were we storing in our database? … hummm … after some fluffing around on what we interpreted from the problem statement, we decided to called Gernot Ritter, the Disaster Relief Coordinator at Caritas. He was really excited to hear about our idea to build the real-time application. He recommended we make the data model as flexible as possible. So for the sake of simplicity we went with storing Events which would have the following attributes:

Event:
  tags
  description
  location
  photo attachment

Some more discussion on the finer details of the data model began but we felt this was enough to start writing code. And so we wrote code. Initial commit was a Rails app and by the 3rd commit we had decided Rails was over kill and we would build a light-weight API in Sinatra which anyone could build on. Thus I began work on the Sinatra app with Max, JoJo and Florian, while Klas started to investigate the web client and Engin sat in a corner writing Objective C for an iphone client.

The coding stints were very productive thanks to the great setup by the organisers and sponsors of the event. The food was great. One of the rooms had a XBox 360 + Kenetic set up so we could take a break and get some physical exercise.

Final application

Here is a screen cast of the final application. You can get the code at https://github.com/mschneider/disaster_maps

Photos

Some photos from the weekend: http://www.flickr.com/photos/rhokberlin

Presentation

I gave a quick talk:

Judges

Sadly I didn’t get a chance to meet all the judges but hope to catch up with them while I am in Berlin. Here is a shot of the judges. We were very pleased to have our project judged as the winner – though it took me a second to register we were the winners since the announcement was in German.

Awesomeness

We won some awesome goodies thanks to the generous sponsors RHoK Berlin

The winning team :-)

RHoK Berlin

Some more coverage of RHoK Berlin: MSDN Channel 9 Movements.org Frank Krippendorf’s blog gov20.de CIO.de Rest.to

Architecture of online spatial modelling

With our most recent project we came another step closer to the realisation that government geospatial projects are moving out of the 90′s. We were asked by a client to build an online modelling tool for evaluating risk resulting from a natural hazard such as an earthquake. Understanding risk requires identifying the hazard (e.g. Tsunami) and exposure (e.g. population in a coastal city).

Data on hazards and exposure forms inputs to the risk and impact models. Such data is normally under the custodianship of national governments, with the responsibility of custodianship often distributed across departments and agencies. Bringing the data together into a single modelling tool is not just a technically challenging task but also requires cooperation and sharing of data across agencies. And anyone who’s worked in government knows this is often harder to do than solving the engineering challenges. We were very fortunate to have the involved custodians onboard with an eagerness to learn and deploy new technologies.

Back in the 90′s such hazard models would be run on a desktop with access to the data via a local disc copied from all the DVD’s posted by the data custodians using snail-mail.

This was a fine solution back in the 90′s but in the 21st century, quite frankly, it starts to look embarrassing. Our proposed solution was a distributed architecture which pulled hazard and exposure data using OGC web services and ran the model on a modelling server that could be scaled. The modelling server pushed the results of the run to another OGC compliant server. This is a simplified drawing of the architecture:

Once the model run was complete the client was notified via websockets. Here is the final application in action:

If you can’t access the video below – you may download the 16MB m4v file here or 10MB for the iphone.

Cool eh! With all these moving parts we needed to ensure that the various services were being monitored. Introducing Monit. If you haven’t heard of Monit check it out here: System Monitoring With Monit. It keeps tabs on your server daemons. If you have any questions feel free to post in the comments or get in touch via email.

… and I am in Berlin, if you are in town come say hi.

Pakistan Wetlands and Ramsar listed sites

Ramsar Sites in Pakistan Page
Wetlands are regarded by nature conservationist as unique ecosystems. They often exhibit delicately balanced environmental conditions and support a wide range of exotic plant and animal species. Wetlands are also often protected by various environmental laws and conventions. Last year when I was visiting Pakistan I learned about the Pakistan Wetlands Project that is under way. The aim of the project is to build capacity for improved wetlands management at the Federal, Provincial and Local government level, as well as the private sector in Pakistan. I thought it would be nice to be able to see all 19 wetlands in Pakistan that are listed under the Ramsar convention on GoogleMaps. This site was built using RubyonRails + GeoRuby. http://pakistan-wetlands.dyndns.org/.  Many thanks to Guilhem for his help during trouble shooting.