Irish Vice Counties : the creation of a specific dataset on OpenStreetMap

I've been meaning to write about OpenStreetMap Ireland's townland mapping project for some time.

It's a wonderful example of how historical maps are of significant value in creating really useful data on OpenStreetMap which is just as relevant as it today as was in the past.

The immediate reason for writing about them is that I have been creating vice county boundaries for Ireland. In doing so I have not just been using the data, but the fantastic range of resources made available through the activities of the townlands project.

The Vice Counties of Ireland
My first complete draft of the boundaries on OpenStreetMap

Strava & OpenStreetMap GPS traces: a quick comparison

Strava introduced their heatmaps and their Strava Slide tool at the Washington DC conference of the OSM US community SotM-US in the spring. I had a quick look at the time and it seemed interesting, but there was little data in areas which I map.

A question came up recently regarding how accurate the Strava heatmaps are for mapping routes on OpenStreetMap, particularly in wooded areas. This prompted me to have another look at the data.

It happens that I have made a lot of traces across two paths on an open playing field (an area sufficiently unobstructed that it is used periodically for calibrating and testing professional grade GPS equipment). A short distance from these two paths is NCN 6, a heavily used national cycle route. It was therefore very easy to grab some screen grabs of Strava and OSM data:

Strava Cycling Heat Map
(NCN 6 on left, university cycle paths centre & right)

The same area showing traces which have been uploaded to OpenStreetMap
(probably mainly by me)

Street Lights: Local Government Open Data 3

Another data set released by my local council contains the locations of all the street lights owned (or operated) by the council. Now I have no intention (or any desire) to import this data into OpenStreetMap. Is it any use in the open data context?

There are two obvious questions that can be asked using street light data:

• How well mapped are all public highways (i.e., including footways and cycleways) in the city?
• Are the highways on OSM accurately mapped?

Street lights are spaced at specific intervals according to the type of highway, it's width and the nature of the light source. This means that their accurate placement is determined by the council beforehand, all location data is to the nearest metre. Since most lights are situated immediately adjacent to the road carriageway they will be a short distance from the road centre line (say under 10 metres).

It is trivial to buffer the network of highways from OSM (via a Geofabrik download) and then use geometry operators to find all lights within the highway buffer envelope, and all outside the envelope. Using a very simple constant buffer of 10 metres I obtained the following result:

Street lights which are not close to OSM roads (best viewed large).

Why do lamp-posts have asset numbers?

A few weeks ago there was an involved, and sometimes, heated, discussion on the main OSM mailing list about imports. Many of the comments were interesting and useful, but one particular strand has attracted my attention. A slightly grotesque paraphrase of these various messages might be "OSM is a poorly managed Computer Science project, with inadequate tools, particularly for version control."

Leaving aside that OSM is neither a project, nor managed: I'd like to focus on what seems to be a surprising mis-perception about OSM as a database.

Firstly, databases don't have to be digital or stored electronically. Phone books, card indexes, and many reference books are easily recognisable as databases. Therefore a primitive database operation is only valid if it can be applied also to non-digital media.

Secondly, and this directly follows, information stored in a database does not have to have a unique identifier (a primary key). There's nothing stopping a telephone number appearing several times in a phone book, or complete pages being duplicated.

Once data is moved onto digital media, it really helps to assign unique identifiers: data can be restructured to be stored more efficiently or be easier to change; it's easier to spot duplication or bad data. This is exactly what OSM does, nodes, ways and relations are identified internally by system generated keys.

And it's what my local council does. They maintain an asset register of street furniture (bollards, traffic signals, parking signs, parking meters) and within this application assign identifiers. These days all this information is geocoded and available in the council's GIS. This information is obviously useful for financial planning, maintenance and other activities. BUT, they've gone a step further and each asset now has it's system assigned number marked on it. WHY? Because, "replace the bulb in lamp 35621" is a lot more specific than "replace the bulb in the lamp outside 25 Main Street". There may be lamp standards opposite each other at that location, or there might be two Main Street's or there may be no number on 25 Main Street, or the house might have been demolished. However, this number DOES NOT uniquely identify the lamp-post: it might do when combined with its location data, location data of the organisation which has assigned the number and information about the status of the system used to generate the number.

What does this have to do with OSM (apart from the wonderful possibility of collecting lamp-post numbers). Well OSM is like my local council, except that our local patch is a bit bigger, and we cannot go stencilling numbers on anything we map. So we have no means to tie an OSM object to its corresponding thing in the real world.

A corollary to this is that we cannot confidently tie OSM objects to geolocated objects in other databases. There are far too many variables to even inspire confidence in fuzzy matches: when was the OSM data mapped? what sources were used? how accurately was it mapped? when was the external data mapped? how current is it? how complete is it? does it have unique identifiers? are the identifiers persistent?

So, we have a host of problems in matching data from an OSM dataset and an equivalent external dataset. These problems relate to location accuracy, temporal accuracy, matching identifiers, and accuracy of associated data.

A good example of these problems is shown by OS Locator Musical Chairs and ITO's OSM Analysis which compare OSM street name data for Great Britain with the OpenData Locator dataset from the Ordnance Survey. This is a nice clear domain with the OS Locator data being from a known source and date and from a highly reputable national mapping agency. In some areas we have enough separately sourced data in OSM to have a handle on how accurately we can match these datasets. In most areas in England about 0.5-1.0% of Locator records cannot be matched to OSM. (I am not aware of reverse statistics, but in a recent survey aimed at hunting down some 20-odd of the mis-matches I found 5 street names used for addresses which are not present in the OS dataset). Even different datasets from OSGB have enough inconsistency to prevent complete matching. And these cases are relatively easy ones.

There are also problems relating to the purpose of external datasets: cadastral data might not reflect the building outlines we would draw naturally (e.g., French & Spanish cadasters); hydrography data might be segmented for water-flow measurement (e.g., NHD); vector data might be optimised for rendering (OS OpenData VectorMapDistrict); road data might not need to be very accurate (TIGER). The imported data should be restructured to reflect what is important for OSM, not maintained in aspic for some putative update.

So those advocating data imports or having 'development forks' of OSM need to answer : how on earth can you easily relate objects between two different data sets, or even the same data set at different times. Alternatively, we could all add some stencils to our mapping toolkits, but even then we'd have to leave our armchairs.

Postscript: the council are busy replacing all the local lamp posts, wonder what number they'll put on the new ones.

Frustrated in Oakham

I visit Oakham about twice a year, and on my last couple of visits have done a bit of ad hoc mapping. The town, like the rest of the county of Rutland, was largely mapped many many years ago over one weekend by the Rutland Mapping party. It has received only a little attention since.

When I visit the Rutland Bird Fair I usually travel by bus, so the first thing I ever noticed about Oakham on OSM was that it was missing the road off the main street to the bus terminus. I was eventually able to fix this during the 2009 Bird Fair. I made a few other corrections, but also added an 'e' to Catmos Street, which provoked comment on talk-gb. (In my defence, OSM had Uppingham Road as Catmos Street for 3 years). Last year I added the Tescos car park and a couple of shopping arcades.

On Wednesday, I thought I'd sneak off and clean-up some mis-matches between OSM data and OS Locator. More or less as soon as we arrived I noted an unrecorded footpath, and then a small residential road opposite the library. This is the problem with Oakham, superficially it looks to be mapped in detail. In practice, there are still plenty of significant features missing. For instance, there are loads of 20 mph speed limits around schools (e.g., on Kilburn Road, Ashwell Road and Braunston Road).

Furthermore, most mapping is now four and a half years old, and Oakham is changing. The most obvious change is a huge construction site on the Barleythorpe Road: the Catmose Campus which will house a sports centre, and new buildings for the main secondary school in the area. I walked past it in the rain (photo below): apparently the school may move in next Monday (a bit optimistic I'd have said). However, if the last couple of years are anything to go by, this dramatic addition won't get mapped in detail for a while.

Other changes are impending: Sainsbury's just had a planning application for a supermarket turned down, and Waitrose have one pending. In the summer there was a for sale sign over the Agricultural Showground suggesting that it has been zoned for housing. Even the shops on Mill Street which I've mapped show many changes from the same street a year or two ago as can be seen by looking at Google StreetView.

There are other issues with the mapping: both tagging and mapping practice have changed since 2006. Most GPS data is probably more accurate, and of course we have aerial photos, and OS data as well.

BUT, most of all, what we lack,is someone based locally. Someone aware of what is happening in the community, such as this interestingly acrimonious planning meeting. Someone able to pop down to the library or the study centre in the Rutland County Museum to check old maps or other sources for names; Someone who knows whether the sixth-form college is called "The Rutland College" or "Rutland County College", and , indeed, what's going to happen to it if Waitrose build a supermarket on its current site; Someone who can act as an advocate for OSM with groups like the formidably active local history society. Surely if someone is willing to compile a list of bells, clocks, scratch dials for Rutland, there might be one person interested in something as mainstream as contributing to a map. This is not just true for Oakham and Rutland, but for many places in Great Britain.

Not for the first time I wondered if a mapping party, consisting mainly of visitors, might have the same effect as an import. The town looks nicely mapped on the slippy map, so no-one notices that there's lots to check and correct: indeed if I regularly drove to Oakham I might not have noticed these deficiencies in the first place. I collected data in the rain with these doubts in mind. I'll map what I can, there is far more which needs to be checked, corrected, and enhanced than anyone can collect in a fleeting visit.