This is a short summary of progress and learning from the first couple of months since launching AfTerFibre, a project to map terrestrial fibre projects in Africa. From the beginning AfTerFibre has been designed as an open project both from the point of view of transparency and from the point of view of participation. So the first goal was to make it easy to share information. This involved setting up a AfTerFibre Google Group so that anyone could contribute information or ask questions. Next we needed a place to store resources as we found them so a wikipedia page was set up to capture information and links to maps of terrestrial fibre projects. That part has gone reasonably well. We now have 83 people in the Google Group and the Wikipedia page now has 67 African operators known to have fibre projects listed, for which about half have links to maps. Where possible I have linked directly to the map on the web if it exists. In other cases, I have uploaded map images that I have found to a Flickr set. Finally there is also a Diigo list of news links related to AfTerFibre.
Having gotten the repositories for the raw information in place, the next challenge was to find out how to create an information chain that would make it easy not only for people to contribute map information but also to submit updates. I’ve been working with some of Google Africa’s GIS team in Nairobi to solve this.
Step 1 – Convert Image Maps into GIS Maps
This stage of the mapping turns out to be remarkably simple. GoogleEarth (GE) is an amazingly simple yet powerful tool. You can import any jpg or png image of a map as an overlay into GE. GE makes it a piece of cake to stretch the image to neatly match the image map borders with the real GIS borders in GE. This makes it relatively simple to then trace fibre routes with the Path tool. I really can’t overemphasise what an amazing job the Google Earth designers have done to make it easy to trace maps. As an example, when you are actively tracing a path in GE, the mouse is no longer available to move the geography as you trace along a line. Happily the keyboard arrow keys are enabled for this purpose allowing you to trace with the right hand and move the geography with your left hand. More software should be this well designed. Because I am a GIS greenhorn, I have just assumed that KML is the appropriate format for this data.
Step 2 – Create a online updateable repository of map data that can feed the AfTerFibre Map
Our initial assumption was that Google Fusion Tables would make a good data repository. It is capable of directly importing the KML files generated from the tracing of maps in Google Earth. The map you see above is a direct representation of the AfTerFibre maps after a simple import into Fusion Tables.
Here is where things get more complicated. Fusion Tables turns out to have some idiosyncrasies (it’s still in Beta) which make it somewhat problematic for AfTerFibre. For a start, it doesn’t allow for easy multi-row editing so if you wanted to make a change across an entire country or operator, it is a very slow process. Second, while Fusion Tables allows you add fields, which is essential for AfTerFibre to capture all the additional information on fibre projects, Fusion tables stops importing KML files properly after you change the database structure. The geo-coordinates are mapping into text fields etc. Chaos and confusion.
An alternative then is to import the KML files into Fusion Tables and directly export as comma separated files (CSV) files which can then be imported into a spreadsheet program. A spreadsheet solves some of the problem by allowing multi-record editing and also making the data easier to massage in general. Each time a new fibre map is made, it could be imported into Fusion Tables, saved as CSV, imported into a separate spreadsheet and edited to match the field layout of the master spreadsheet and the imported into the master spreadsheet. The master spreadsheet can then be exported to CSV which can then be re-imported into Fusion Tables for display in Google Maps.
If this were a one-time process, the above might be an acceptable solution but not only will new maps regularly be added but existed maps are likely to receive numerous corrections.
So, what to do? It seems to me that what is needed is some kind of versioning repository (like Github) for the KML files for AfTerFibre. I’ve done some googling around this but haven’t come across anyone who is storing their KML in a versioning repository. It seems like a clever idea but it worries me that I can’t find anyone else doing it.
If KML files could simply be updated in a Github or similar repository, then it wouldn’t be that hard to write a script that retrieved the files from the repository and massged into a GIS display with the appropriate mouseover information displays etc.
So, I’m a little bit stuck here and open to suggestions.