Nottingham OSM Pub Meet March 8th

After Mark Iliffe & I met at State of the Map 2010 in Girona, we talked about arranging an event for OSM contributors in the East Midlands to meet-up. As Mark was then Leicester-based we'd been thinking of a mapping party. However, with Mark doing a PhD in Nottingham it seemed that we could start with something a little less formal. A pub meeting seemed a good chance to meet some fellow mappers face-to-face and talk about the activities which interested people.

At the outset I hoped we might at get at least five people, that I'd get to meet another local mapper, and that we'd make contact with other interested parties.

I was immediately reassured when I saw that Shaun was coming. Neatly, he was my first ever OSM contact point when I first attended a London pub meeting almost exactly two years ago. Joining us were: local mappers Andy (SomeoneElse), Kev (kevjs1982) and Laura; David came along representing Pedals, the local cycle advocacy group; and Marcus came from Nottinghack, the Nottingham Hackspace. So 8 of us altogether was a nice turnout. It was nice to make contact with Pedals, and a wonderful surprise to discover about the hackspace.

Subjects which came up in the course of the evening:

• Inability of the automated voice in Garmin nuvi to cope with "Huthwaite".
• DINTY, monads, tetrads and octads (dont ask).
• The lonely bus stop on Erewash Field (see image at right, discovered by Kev)
• Differences between Lancashire towns with lots of rows of terraced housing and Nottingham's huge legacy of inter-war ditto
• Co-ordinating mapping activity, specifically for crisis mapping. I alluded to the difficulties involved, and the general point that usually mapping activity tends to self-correct. Mappers are both ants (self-organising) and cats (not easy to herd).
• Pubs which have gone (The Gate at Jacksdale).
• Marcus asked why the map round the Borlase Warren had lots of buildings but few Points of Interest. We agreed that adding individual shops was surprising labour intensive and error prone. Kev, who has mapped all the shops on Central Avenue, West Bridgford, said it was pretty easy to accidentally miss one. (Also, I had surveyed these but have not added them to the map).
• David asked the active mappers if we ever used OSM. He got an emphatic yes, Andy showing that his Garmin showed pubs with a real_ale=yes tag.
  

My OSM logo muffins were rejected! Largely (I hope) because most people had either eaten pancakes before hand or had ordered one of the absurdly generous portions from Borlase Warren's menu. At least there was some semblance of cake at the meeting.

A straw poll suggested that we should do it again, and so I'll be scheduling a date for April real soon now.

Any OpenStreetMap is, of course, an opportunity to add to the map, so I'm glad to report, that after an extensive survey, Andy (SomeoneElse) summarised our conclusions.

PS. I went to Nottinghack's weekly opening evening the following day and was able to add them to OSM.

Garmin overlays of GNIS names

I've just been reminded of something I did last summer during the Pakistan Flooding: creating a garmin overlay of names from GNIS.

I'd been trying to enter names from old US DMA (military) maps and was getting frustrated by the difficulty of being sure of names. Plus with landsat imagery it was possible to identify villages, but I wanted to put names to them. I downloaded the GNIS names for Pakistan, originally with the aim of creating a separate layer in JOSM for assigning names. This required converting the names into OSM format (I have no skills suitable for building a tile or other service). In the end it was still difficult to reconcile names to residential areas (mainly GNIS quality issues, I suspect), but I did create a transparent 'names' overlay for garmins from the data using mkgmap.

As it might be useful again, I thought I'd quickly describe the process. Essentially there were seven steps, each simple on its own, but involving a bit of trial and error before I got there:

1. Create an image table of GNIS format in a PostGIS database. By image table I mean one that copies the columns of the source data one-for-one, often using character columns even if the data is numeric to avoid losing data. type conversion can be carried out as a post-processing step.
2. Load the downloaded GNIS data with a simple COPY statement.
3. Add a POINT geometry column to the table. Populate this from the lat/lon in the data
   
4. Create a 'simple' OSM schema in the same database.
5. Populate the nodes table of the schema, either directly from the image table or from a table created to post-process the data.
6. Extract the data to an OSM XML file using Osmosis.
7. Build a garmin file using mkgmap.

I'll go through each of these steps in more detail below.

The starting point is to have a PostgreSQL db server running, with PostGIS installed. I have a template database which has not only PostGIS, but also hstore, and the OSM simple schema already present. I can then immediately create a new database specifically for a given OSM task. What I describe works for Postgres DB 8.4, PostGIS 1.5 on Windows with Osmosis 0.38 and mkgmap r1443. Small modifications may be needed for other software versions.

Image Table:

-- Table: pakistan_gnis_names

DROP TABLE pakistan_gnis_names;

CREATE TABLE pakistan_gnis_names
(
region_font_code numeric(1),
unique_feature_id numeric(38),
unique_name_id numeric(38),
latitude numeric(10,8),
longitude numeric(11,8),
latitude_ddmmss character(6),
longitude_ddmmss character(7),
military_grid_ref character(15),
jog_ref character(7),
feature_classification character(1),
feature_fesignation_code character(5),
populated_Place_Class numeric(1),
primary_country_code character(2),
first_order_admin_code character(2),
population numeric(38),
elevation numeric(38),
secondary_country_code character varying(128),
name_type character(2),
language_code character(3),
short_form character varying(128),
sort_name_ro character varying(255),
full_name_ro character varying(255),
full_name_nd_ro character varying(255),
sort_name_rg character varying(255),
full_name_rg character varying(255),
full_name_nd_rg character varying(255),
note text,
modified_date date
)
WITH (
OIDS=FALSE
);
ALTER TABLE pakistan_gnis_names OWNER TO jrc;

Add & Populate Geometry Columns
(in this case in a table just containing a subset of names):

drop table pk_gnis_p_name;
create table pk_gnis_p_name as
select * from pakistan_gnis_names
where feature_classification ='P';
alter table pk_gnis_p_name drop wgs_geom;
select AddGeometryColumn('pk_gnis_p_name','wgs_geom',4326,'POINT',2);
select populate_geometry_columns()
;
update pk_gnis_p_name
set wgs_geom = ST_geometryFROMTEXT(
'POINT('|| longitude || ' ' || latitude || ')'
,4326);

Create OSM 'Simple Schema':
This needs to be compatible with the version of Osmosis being used. Normally the DDL are in one of the osmosis distribution sub-directories. If doing in a dedicated database just use the default names, otherwise stick a prefix on the standard table & index names. I used 'pk' for Pakistan.

Populate the NODE table

SQL to be added. This is the only table needing populating so a single simple statement is needed, but you do need to use hstore syntax these days. This is a single insert into the nodes table, but as several columns are NOT NULL we need to ensure that we create suitable values. Fortunately, referential integrity is not enforced across tables so we don't have to worry about ensuring the reference values are populated (although it's not a bad idea). Furthermore, the data is not being merged with any OSM data. For gnis data there is already a unique identifier column so we can use this to identify the node. Version, changeset and user can all be given a standard default value (0, -1 are most commonly used).

INSERT INTO nodes (id, "version", user_id, changeset_id, tstamp, tags, geom)
SELECT unique_feature_id as id
, 0 as "version"
, 0 as user_id
, 0 as changeset_id
, clock_timestamp as tstamp
, 'name' => full_name_ro as tags
, wgs_geom as geom
FROM pk_gnis_names

Extract with osmosis:
This is a standard extract using --rp and --wx flags. See Detailed usage.

Create a specific style for mkgmap:
I created a special style for mkmap called gnis_names. A zip of the subdirectory is here.

I'll add a bit more about the logic later.

Build the Garmin IMG file using mkgmap:
This is the command I used to build the overlay. Additional work is required to get this into MapSource or integrate it into a gmapsupp.img file. I used MapSourceToolkit to change the registry settings on windows to see it in mapsource and thereby transfer it to my garmin. Usual stuff applies about family/product & file names.

java -ea -Xmx1536M -jar mkgmap.jar --mapname=88031021
--preserve-element-order
--style-file=resources\styles\gnis_points --transparent --description="GNIS Pop Places PK"
--series-name="GNIS Pop Places PK" --input-file=pk_gnis_names.osm --product-id=88 --family-id=1 --product-name="PK GNIS" --overview-mapname=88031020 --tdbfile

I'll add a bit more to this later & tidy it up too.

Former Pubs

An inevitable problem arises when mapping in Britain these days. How does one map dead pubs?

Pubs have been dying at an increasing tempo ever since the Beer Orders of 1989. Some of this is due to the rapacity of the 'pubcos', but much is due to cheap supermarket booze, social change in inner-city areas, and a broader range of alternative means of entertainment. Although dead pubs exist everywhere, it's the ones in inner-city areas, or suburban estates, which I notice most frequently.

Some pubs are 'zombies', having fallen into a 'close-reopen-fail-close' cycle (phrase nicked from Richard on IRC). These can often be identified by the 'run your own pub' banners outside. There are also pubs which are only slightly more alive: often remarkably difficult to work out their actual status when walking past during the day. Just occasionally one of this class of pub gets resurrected. I tend to tag all such pubs as ordinary pubs, but if currently closed will add "(closed)" after the name. I do this for two reasons: an amazing number do show a triumph of hope over experience, and re-open; and, secondly, pubs are important navigational landmarks. The Target roundabout at Northolt on the A40 is a good example of the latter, even though the pub closed in 1986 when it was converted to a burger bar.

Once the pub sign has gone and all the windows are boarded up then I'm willing to accordingly change the tagging to amenity=dead_pub or similar. Usually such pubs either change use, or get demolished. Local to Nottingham they've turned into supermarkets (two Tesco Express stores : the Jolly Higglers, 17/21st Lancers), Indian restaurants (The Poachers Tavern now the Gurkha Kitchen) and mosques (Le Grand, Hyson Green , and possibly The Boulevard). The A610, Nuthall Road, has a large number of recently demolished pubs, typically they are replaced by housing. Other recent housing developments include : Cremorne Drive is a private gated road in The Meadows on the site of the Cremorne Hotel; Beaumont Square has been built on the site of The Wollaton Arms.

Elsewhere, the Stonor Arms in Stonor has been closed since the early 2000s, and despite recentish press coverage didn't look likely to open again last summer, but The Robin Hood in Pinkneys Green sprang into life again in 2007 after being boarded-up for 6 years. So its remarkably difficult to be certain about dead pubs.

A very few remain intact as buildings but with a changed use, like The Dover Castle (photo above), now let as student accommodation. I don't know when this closed, but it was already there by the time of the 1891 census. Familiarity with an area, or close inspection of the building will reveal the former use: then it's worth recording them as building=pub.

P.S. Chris Richards (riffdesign) has some nice atmospheric photos of Radford between '70s and early 90s, including several dead pubs.

A not very mysterious Mystery Walk

Given that mapping takes me to many local places I signed up with the Ramblers Association to do a 'mystery walk'. Of course, these are so-named by analogy with 'mystery shoppers', but I find the description engaging in an oxymoronic way.

I registered ages ago, but only received my starting point co-ordinates last week: SK 530 380 (drop 4 of the digits and you'll see the origin on my OSM username - a geocode). Unfortunately, this is right on the edge of Nottingham, and part of the idea is to stay within the boundary of a single highway authority (the local government body responsible for footpaths), so my route was already constrained. A second constraint was that towards the city is a large contiguous area of parkland, consisting of the main campus of Nottingham University (University Park) and two public parks (Highfields and Wollaton Park). Together these provide a large area of pleasant and varied walking: but they are not public highways, so there wasn't much point reporting path quality and accessibility there.

I really had only one viable destination: a small remnant of 'countryside' between Nottingham city and the suburb of Bramcote Hills. This is a ridge of Sherwood Formation sandstone (Permo-Triassic age), with remnants of the characteristic heathland flora of Sherwood Forest. I know the area, but not particularly well: it is too far for casual naturalising and not so hugely different in terms of plants and insects from places closer to home. For those in the immediate locality Bramcote Ridge is highly valued with an active friends group. The part in Broxtowe district is largely managed as two separate Nature Reserves: Alexandrina Plantation and Sandy Lane Open Space. I hadn't realised until yesterday, but the part of the ridge in Nottingham does not seem to have any particular designation.



The whole area is accessible from many entry points, but many of these are obscure little paths at the end of dead-end streets. It is more-or-less impossible to work out access from the OS Explorer map: fortunately the area is pretty well mapped elsewhere! Using OSM-based maps I plotted a route which would take me onto Bramcote Ridge and (largely) stay within the City boundary. Of course I added a couple of mapping desiderata to my route: a couple of postboxes and addresses. I also wanted to see if I could find any waymarks for the notoriously elusive Robin Hood Way, one of two long distance paths promoted by the county council.

My route took me along the Robin Hood Way from close to Wollaton Park, but I saw no signs of waymarking until I reached the top of Bramcote Ridge, just as the path left Nottingham. I did pass a surprising large number of side roads which enabled me to add a large number of address interpolation ways. Very high quality Bing imagery means that it's usually possible to extrapolate and assign addresses for the whole street. Although the paths on to the ridge are not easy to find they are heavily used : and anyway most are in OSM.

I did find a 1933 Nottingham City Boundary Post. The city boundaries were extended in 1933 to incorporate some surrounding villages which had already been suburbanised. Boundaries with other places were also adjusted. Sseveral are shown on the provisional edition 1:25000 map of SK53, so I was on the lookout. These seem to have once been quite common: I've put a couple of others on OSM: they're an antidote to the often made claim that boundaries cannot be surveyed.

The only negative to report back to the RA is a complete dearth of waymarking, but this might be expected in a city. Nottingham, like many cities, did not have to maintain a definitive map of Public Rights of Way, and is only doing so now, because of the cut-off date of 2026 in the CRoW legislation. And, I only managed about a quarter of the 2 mile distance off residential roads.

Overall, I learnt nothing I didn't already know: The Robin Hood way is badly waymarked (or rather it's original waymarks have not been maintained); OS mid-scale mapping is not much use for finding off-road walking routes in cities; and OSM provides more useful cartography and GPS maps for this kind of walk. Urban local authorities might be relucant to spend money on waymarking for several reasons (vandalism, fly-tipping and lack of money come to mind). An alternative would be to providing focussed mapping for walkers. They already do this for cyclists, and I know a good cheap source of information.

Frustrated in Oakham

I visit Oakham about twice a year, and on my last couple of visits have done a bit of ad hoc mapping. The town, like the rest of the county of Rutland, was largely mapped many many years ago over one weekend by the Rutland Mapping party. It has received only a little attention since.

When I visit the Rutland Bird Fair I usually travel by bus, so the first thing I ever noticed about Oakham on OSM was that it was missing the road off the main street to the bus terminus. I was eventually able to fix this during the 2009 Bird Fair. I made a few other corrections, but also added an 'e' to Catmos Street, which provoked comment on talk-gb. (In my defence, OSM had Uppingham Road as Catmos Street for 3 years). Last year I added the Tescos car park and a couple of shopping arcades.

On Wednesday, I thought I'd sneak off and clean-up some mis-matches between OSM data and OS Locator. More or less as soon as we arrived I noted an unrecorded footpath, and then a small residential road opposite the library. This is the problem with Oakham, superficially it looks to be mapped in detail. In practice, there are still plenty of significant features missing. For instance, there are loads of 20 mph speed limits around schools (e.g., on Kilburn Road, Ashwell Road and Braunston Road).

Furthermore, most mapping is now four and a half years old, and Oakham is changing. The most obvious change is a huge construction site on the Barleythorpe Road: the Catmose Campus which will house a sports centre, and new buildings for the main secondary school in the area. I walked past it in the rain (photo below): apparently the school may move in next Monday (a bit optimistic I'd have said). However, if the last couple of years are anything to go by, this dramatic addition won't get mapped in detail for a while.

Other changes are impending: Sainsbury's just had a planning application for a supermarket turned down, and Waitrose have one pending. In the summer there was a for sale sign over the Agricultural Showground suggesting that it has been zoned for housing. Even the shops on Mill Street which I've mapped show many changes from the same street a year or two ago as can be seen by looking at Google StreetView.

There are other issues with the mapping: both tagging and mapping practice have changed since 2006. Most GPS data is probably more accurate, and of course we have aerial photos, and OS data as well.

BUT, most of all, what we lack,is someone based locally. Someone aware of what is happening in the community, such as this interestingly acrimonious planning meeting. Someone able to pop down to the library or the study centre in the Rutland County Museum to check old maps or other sources for names; Someone who knows whether the sixth-form college is called "The Rutland College" or "Rutland County College", and , indeed, what's going to happen to it if Waitrose build a supermarket on its current site; Someone who can act as an advocate for OSM with groups like the formidably active local history society. Surely if someone is willing to compile a list of bells, clocks, scratch dials for Rutland, there might be one person interested in something as mainstream as contributing to a map. This is not just true for Oakham and Rutland, but for many places in Great Britain.

Not for the first time I wondered if a mapping party, consisting mainly of visitors, might have the same effect as an import. The town looks nicely mapped on the slippy map, so no-one notices that there's lots to check and correct: indeed if I regularly drove to Oakham I might not have noticed these deficiencies in the first place. I collected data in the rain with these doubts in mind. I'll map what I can, there is far more which needs to be checked, corrected, and enhanced than anyone can collect in a fleeting visit.

Updating Pub Density

I was hugely gratified by the interest shown in the pub density map. My original intent was rather more serious than just visualising a class of POI popular with OSMers.

I now have access to the government generated areas used to present census, and other population-based data, in a uniform way. These are called "Super Output Areas", and, I think, roughly contain 7500 people. Unfortunately, I only have these areas for England and Wales (I currently have a request with the General Register Office for Scotland for those North of the border).

So, here is a new pub density map:

A larger version is available on Flickr.

There are lots of things to notice, so I'm just going to highlight a few:

• Outer London has loads of gaps.
• Grimsby seems to be teetotal.
• A whole swathe across the Pennines from Liverpool to Doncaster and beyond is rather pub light. Many parts here have been traced from various sources.
• Wales is still light on pubs, which is to be expected because it is seriously under-mapped.
• Nottingham has three areas without pubs, one probably does not contain any pubs, another is poorly mapped, and the third has been traced.
  

Broadly speaking the idea of using equal population areas doe reveal more about the data.

Government Open Data License FAIL

I'd like to illustrate this post, but it's taken so long to work out whether I'm allowed to I haven't had time left.

I wanted to extend my analysis of various OSM datasets by correcting for population density. A simple and obvious way of doing this is using the Super Output Areas defined by the Office of National Statistics.

Firstly, the data are not available on-line. Last week I completed the rather horrible Word document form available at the ONS site. I was pleased to get the CD promptly in the post yesterday. So far so good, just 5 days delay compared with an online download.

Next, I thought I'd check through the T&Cs: I expected a non-commercial license or restrictions similar to those of Natural England and other bodies which allow you to use their data, but not to show it to anyone else. So here are the main terms:

There is no restriction on the use of the material except that:
• a click-use licence must be obtained for re-use and publication;
• an end-user licence issued under a click-use licence must contain these terms and
conditions;
• copyright and source must be acknowledged[4] on publication;

If I want to make an image using the boundaries and show it on this blog, I require a click-through license. In fact even if I printed out an image and tried to photocopy it in a public library I need a click-through license!

OK, how do I get a click-through license?

A link in the terms took me to the National Archives site, which mentions two kinds of license: an open government license, which has nice clear terms:

• copy, publish, distribute and transmit the Information;
• adapt the Information;
• exploit the Information commercially for example, by combining it with other Information, or by including it in your own product or application.

You must, where you do any of the above:

• acknowledge the source of the Information by including any attribution statement specified by the Information Provider(s) and, where possible, provide a link to this licence;

There is also something called the Parliamentary License which does have a link to a click-through license page, this time at OPSI (Office of Public Sector Information). This site does state:

1. Did you know?
We have completely changed the way you can use Crown copyright information. In fact you probably don't even need a Click-Use Licence anymore. This is because the information previously made available under the PSI Click-Use Licence is now offered under the Open Government Licence

Although I'm probably OK under an OGL, it's a bit difficult to be sure, so I thought I'd better register anyway. Finally there is a clear statement:

IMPORTANT CHANGES TO LICENSING
Crown copyright Information previously made available under the PSI Click-Use Licence is now offered under the Open Government Licence

Yeah!

But I've already waited 5 days for the data, and now I've spent a frustrating hour going from pillar to post between sundry websites belonging to different government departments all with slightly inconsistent messages.

If the ONS had published the data online they could have up-to-date license information, rather than old stuff pertaining to when the printed the CD, and I would have known exactly what I could and could not do with the data.

Enough moaning, time to use the data. Oh, there's another file of T&Cs, this time from the Ordnance Survey ...