Lamp-posts and Botanical Records

Readers of this blog will know I have an fondness for using Local Government Open Data on the positions of street lights (see here, here, here and here). Over the last couple of weeks I've found another one.

Newly planted Quercus robur next to Derby Road Nottingham
note lamp post!

My original reason for buying a GPS was to make it easier to record wildlife records: I'm only 5 years into the minor distraction called OpenStreetMap! However, over the New Year I managed to drag myself away from addresses and postcodes and participated in the BSBI (the national botanical society) New Year's Day Plant Hunt. This is a relatively recent innovation, but has captured the attention of more and more botanists. The idea is to find as many different species of plants in flower during a 3 hour survey window.

One that got away. I missed this fine Common Knapweed Centaurea nigra on Jan 1.

Many people have been intrigued by common flowers appearing out of season. (Of course, some, such as Gorse are rarely seen out of flower ("When gorse is out of bloom, kissing is out of season"). Milder winters almost certainly due to global climate change are an important factor in allowing some plants to continue flowering. In other cases management of the land can cause plants to make new growth in the Autumn. Another possibility is that because the length of the day is the same in the Autumn as in the Spring that plants respond to some photoperiod effect. I've been noticing these things for years but never recorded them in a systematic way.

Green Alkanet Pentaglottis sempervirens
An attractive enough flower, both for bees and people, but a pernicious weed which seems to infest everywhere.

Tim Rich's very successful expedition in Cardiff on 1st January 2012 really set the ball rolling. Tim is a professional botanists, co-author of the Plant Crib, an indispensable guide to difficult plants, and author of a number of specialised to very specialised identification guides. Amazingly for someone who has been a bedrock for supporting amateur botanists across the country over the past 20 years, he is also out of work.  He's not the only one: it seems each year I get more mail bounced back or a new email address for a museum professional who has 'been let go'. (For non-naturalist readers, taxonomists are roughly equivalent to sysadmins in the botanical world: lots of things wouldn't happen without them, but they're not noticed when things are going OK).

Yarrow Achillea millefolium
Yarrow plants seem to thrive at this time of year, rather than just 'hanging on in'.

Another aside, people like Tim also play another non-obvious role: that of encouraging future generations of scientists. Many kids start out with an interest in natural history which can, if encouraged, lead to a careers in a whole slate of economically important areas such as medicine, and molecular biology, and not just ecology. Reducing the number of able scientists in museums who also perform substantial outreach and education activities, is eating the seed-corn for far more than just their direct specialisms. At least one British Nobel Laureate was inspired in this way.

Anyway, back two years ago I was really pleased to hear Tim on the Radio 4 Today programme describing his first plant hunt: my casual interest was validated by a similar interest by a serious professional botanist.

Red Valerian Centranthus ruber, growing in Churchyard car park.
Normally a very vigorous population of these pop-up all over the churchyard in Summer.

I managed to do about an hour's surveying on January 1, 2013, but still managed to find 20 plants in flower. This year I was determined to do better. I wasn't able to join the Leicester botanists who set the bar high with 59 or so species. The weather and my health prevented me finding much before the end of the year.

However, the rain stayed off on the afternoon of New Year's Day, which meant I could visit all the places I'd already seen plants in flower, and spend more time looking for others. My main locations for searching for plants where: two areas of parkland ,both campuses of Nottingham University; grass verges along some local roads; a local churchyard; and the car parking around local shops. Many plants survive in the road gutters, or at the edge of the pavement and a wall or on little patches which only get cut from time to time.

Short daylight hours meant that it was getting dark towards the end of my survey
Ribwort Plantain Plantago lanceolata

My advantage is that I have used most of these places to learn how to identify plants: I thus know exactly where to find around 400 species of flowering plant within 1 km of home. Naturalists call this 'patch watching' (The Urban Birder talks a lot about the concept): it works because one builds up an intimate knowledge of one's local flora. Things which are new or unfamiliar stand out remarkably quickly in this context (in computing terms, its a very efficient vdiff). Unfortunately familiarity with a patch can also mean that one struggles to identify ordinary plants in an unfamiliar context.

unprepossessing

A bit of unprepossessing highway verge, a typical place worth hunting in.
At least 6 different plants in flower here, fortunately it hadn't been cut in late Autumn.
Plants: Yarrow, Groundsel, Dandelion, Chickweed, Wall Barley and Shepherd's Purse.

For my recording I used the Obsmapp Android app, which meant that I didn't have to carry much with me. The survey time was about right before the battery ran down. I also collected samples of many of the plants to double check my identifications down a microscope when I got home. (A cold dank January day with poor light is not the place to check if a plant has star-shaped hairs on the undersides of it's leaves).

One minor problem with Obsmapp is that you need to make an identification at the time (or at least use a known species as a place holder). Now like many naturalists I'm not great at identifying grasses or dandelion-like flowers. For the latter I used a place holder species and checked the identification when I got home. However there were several grass specimens which I was not at all sure about. I've never chunked grasses in such a way that I can make a good approximate identification: I have to take them home and use a key.

Red Dead-nettle Lamium purpureum

This meant that I could not record the geolocation of these specimens through Obsmapp. Fortunately, street lights were to hand: instead I took a photo (geolocated, but that's not too important) of the nearest lamp-post's asset identifier. I then used the Nottingham Open Data Street Lamp dataset to get the lat/lon for the lamp-post and added these records into Observado once I got home.

A tuft of False Oat Grass was under this lamp post

All in all I managed to get 40 different species (my unknown grasses all turned out to be False Oat Grass). This was a very gratifying total. Annoyingly, I walked straight past a fine example of Knapweed which I found a couple of days later, and I noticed a Gorse bush which I was unaware off only on Monday!

One other aspect of looking for flowering plants at this time of year is that often they are stunted and don't always have the same habit as in the Spring and Summer. It is therefore an excellent way to challenge and improve one's identification skills. Not least know that in Britain we have Poland and Clements' Vegetative Key for identifying the plants when not in flower. I also used Eggenberg's lovely Flora Vegetativa which has many fine illustrations which complement the Vegetative Flora. (even though this is book on Swiss plants it works very well in Britain).

The other day I noticed a newly planted oak tree (one of 10 trees planted at the behest of our local Councillor Sally Longford to replace ones which had to be felled because of disease). Although I took geo-located photos I took great care to include 2 lamp-posts to enable me to locate this new tree accurately before adding it to OpenStreetMap.

Bee Orchid at Nottingham University Jubilee Campus
(location originally noted using lighting asset numbers)

I've previously used the asset numbers on the lighting in the University to check on locations of some Orchids in the Jubilee Campus, and I've also used them as markers when trying to map the trees in a belt of woodland more accurately.

So for urban wildlife, lamp-posts can be an invaluable resource for accurately locating specimens!

Assigning addresses from Land Registry Prices Paid data

After the disappointment of the Land Registry INSPIRE land parcels, it is nice to report a large and useful open dataset from the same source: the Prices Paid data (LRPP). These are the actual prices paid for houses and flats in England and Wales from late-1995 or thereabouts to the present.

New Residential Roads in England and Wales
Roads were identified from Land Registry Prices Paid data
and matched by name to OSM highways within 2km of postcode centroid

Mapping off-piste ski routes

The tragic death of a 16 year-old boy in Lech on Wednesday (18th December 2013), prompted me to look at aerial photos and various maps of the Arlberg area. As is usual with news reports of such accidents the information on the precise locality is fairly sketchy. This is always annoying to anyone who knows an area, and particularly if one goes off the marked pistes, because detailed knowledge of the route and conditions are essential to understanding the news. (Compare this with the volume of knowledgeable and detailed commentary on cycling deaths in London).

The boy was skiing with his father and his twin brother and a local ski guide. They started an avalanche which carried one of the boys, his father and the guide down the mountain around 150 metres. Fortunately the local news (understandably) provided a bit more detail: "Bereich Madloch-Fauler Stock Richtung Stierlochbach" and a photo of the avalanche.

The area NW of Madlochjoch on OpenSnowMap (View Larger Map)
(close the overlay to see the map)

The whole Arlberg area is well-known for a plethora of named off-piste ski routes, to quote "... every variant, however rarely skiable has a name...", so there was a good chance that this accident happened on a well-known run: particularly as it was so early in the season. As the avalanche debris fan was next to some Pinus mugo scrub the number of locations could be reduced. I own two books on Arlberg off-piste routes, but neither was to hand. I did find this fine photo with the main itineraries marked:

Overview of off piste routes accessible from Madlochjoch
((c) Author: Roland Wasmayer, Source: freeskiers.net via OutdoorActive)

Details of the accident will be released by the Vorarlberg Avalanche service. These short accounts usually have an aerial photo giving an overview of the accident site, such as this one which was very wdely reported because it left a Dutch Prince in a coma from which he never recovered..

I mapped a little bit of the scrub on OpenStreetMap and also found this image of the entry to the Stierfall route. I suspect the the accident happened in either Heuwet or Heuwetrinne: both so frequently skied in mid-season to be heavily mogulled. None of these itineraries are mapped on OpenStreetMap, as is true for other itineraries throughout the Arlberg ski area.

Entering the Stierfall, photo Stegaku

In April 1992 I was lucky enough to have great weather in St Anton after an exceptional snowfall in March. Conditions meant that there was wonderful spring snow for a couple of hours in the morning, but everywhere routes which are rarely skiable were fully covered. It was astonishing to see ski tracks in the most unexpected places: particularly on the W side of the Arlberg pass between St Christoph and Stuben. Many of these routes were skiied long before mechanical uplift existed: nowadays the position of ski lifts determines which are favoured.

One of the odd things about this is to discover long vanished ski lifts. A post on Edward Tufte's forum alerted me to a website featuring Heinrich Berann's ski maps and panoramas. (Berann may be familiar to others as the illustrator / cartographer of Ocean Floor maps). Berann's style is very recognisable. 

Ski map of St Anton-am-Arlberg by Prof H.C. Brann

What is particularly interesting is that he made ski 'maps' for the Arlberg area from 1937 through to the late 1970s, and therefore it is possible to trace some of the evolution of the area's infrastructure. Most noticeable is that until around 1980 there was a lift on the W side of the Arlberg road at St Christoph. Also noticeable is the curious way the grading of ski runs has changed: particularly true of runs now classed as freeride. Runs which were once just graded as black turned into ski itineraries, and some have just disappeared.

Of course someone has thought of mapping freeride territory: Freeridemaps of Altstätten, Switzerland. The sample image on their website is the Arlberg area around St Anton-am-Arlberg. In the main the map shows areas appropriateness for different ranges of skill and experience: but as far as I can tell largely based on objective factors, such as safety of run-outs, and likely consequences of a fall. Above all it is based on the steepness of the land.

The Swiss and Austrian avalanche service also produce map overlays based on steepness to show avalanche risk: in the Swiss case this data is available by subscription in the White Risk app for Android and the iPhone.

Mapping avalanche risk on it's own is a challenge. There are numerous different factors to represent: altitude, age and depth of snow pack, recent changes to snow pack, aspect, underlying terrain (steep grass slopes often avalanche). In writing this post I came across some interesting papers on the subject of the cartography of avalanche maps.

Profile of Snowpack derived using DAVOS ELF SNOWPACK program

One of the more intriguing ideas is how the history of the snowpack at a point can be displayed (see above). Quite how such data might be transformed to convey information on a map is beyond me.

Despite the great advances made in the past year with OpenSnowMap there is considerably more which we could do to provide information relevant to freeride and back-country skiing, and ski mountaineering.

• Grading. In my view the current use of a single category of freeride downhill routes is misleading, Many off-piste guides such as those for the major French resorts published by Editions Vamos, or the similar Polvere Rosa covering the south side of the Monte Rosa massif, make use of grading schemes (4 classes of technical difficult in the former, 7 in the latter which includes some extreme itineraries; and 4 classes of objective dangers). I am hesitant to mark freeride areas which I know because of the wide range of difficulties and objective dangers likely to be encountered.
• Routes. To a certain extent showing routes spoils some of the key aspects of being away from the piste. Picking ones own line, both in the light of one's ability and safety of surroundings is part and parcel of developing the skills and awareness necessary. In many places (Albona at Stuben, most of Les Grands Montets below 2800m, Pavi in A-basin) all that is needed is to show that the area is free-ride territory. In others entry lines and exit routes may not be immediately obvious and are worth showing (it is not uncommon for these to be the least pleasant part of a route), particularly if there are cliffs or other hazards lower down (e.g., above Le Fornet, Val d'Isere).
  
  

  

  Seracs on Glacier des Rognons, Les Grand Montets by LensScaper, on Flickr
  IGN Geoportail shows what's underneath the snow.
  
  

  
  

• Objective Hazards. If more freeride terrain is to be added to OpenStreetMap there needs to be much more mapping of the obvious objective hazards: cliffs as mentioned above are the prime example. Others might include: streams (which may not be frozen enough to bear loads), known crevasse and serac fields (mapping individual crevasses can wait a while), 'bullet-proof' ice on glaciers, and notorious avalanche gullies.
• Safety. Obviously marking ski routes creates some concerns over people placing too much reliance on such information. In reality people undertaking such routes require a minimum level of skill, equipment, knowledge and experience.

Freeride and backcountry skiers need a sizeable amount of cartographic information: some is persistent, much of it is transient. The ability to convey this data in suitable maps is a significant and interesting challenge. The flexibility of OSM data and technologies suggest to me that we can play a role. However, we need much more base data: in particular GPS traces, and good perspective photos of routes with suitable open licenses.

Note: This post contains a number of images which are copyright by their respective authors, and are not licensed under creative commons. Normally I try and only use CC images, but in this case suitable images were not available. Please respect the rights of the image owners.

building=collapsed : history of an OpenStreetMap tag

The collapsed Maxima XX Supermarket in Riga
Source: Valenciano via Wikimedia Commons. CC-BY_SA

Today I noticed a local news item about a pub, The Bridge, which caught fire on Saturday. Apparently it is so badly damaged that it will need to be demolished. Naturally I checked OpenStreetMap, but the changes had already been mapped. This is just another example.

There are a number of other recent instances where the badly damaged buidlings' changed circumstances have been mapped very promptly on OpenStreetMap. Notable example are: The Clutha Vaults, hit by a helicopter in Glasgow and the collapse of the roof of the Maxima XX supermarket in Riga.

The normal changes are to either remove any amenity tags (usually by pre-pending them with old_, which facilitates finding the location), mark the surrounding area with landuse=demolition, or landuse=construction, and to change any building=* to building=collapsed. The process now occurs as swiftly as the creation of the associated article on Wikipedia: naturally there is substantial synergy between the two processes.

However, the building=collapsed tag did not evolve in order to better illustrate news events.

Presidential Palace, Port-au-Prince after 2010 earthquake.
Source: Wikimedia Commons CC-BY-SA.

It's possible that it was in use earlier (an exercise in trawling OSM's history files), but my memory is that it really entered OSM within an hour or so of availability of the first post-'quake imagery of Port-au-Prince in January 2010. We had a discussion on IRC (no doubt someone will have squirrelled away archives of IRC logs from the time) about the best way to tag this. I was for something slightly more elaborate: largely to separate the type of building from its intactness, but the simplicity of building=collapsed won over any longer term semantic issues: and this is the tag which is used. Currently it has 53,622 instances on OSM.

Central Port-au-Prince showing collapsed and damaged buildings, 17 January 2010
The Presidential Palace is top right.
Source: Haiti Earthquake Damage Map

When we tagged collapsed building and debris on roads forming barriers I don't think that we had any real expectation that this data would be used: the first responders had no real idea about the likely accuracy of such data. In practice standard mapping of Haiti was OSMs most useful contribution at the time.
However, over time, OpenStreetMap's capabilities and credibility have increased. One factor is that after a major disaster OSM can deploy a significant number of mappers who already have experience of interpreting aerial imagery after catastrophes. Another, as in the Philippines recently, first responders, such as the Red Cross, are seeking to use OSM directly.

Most of these entities have been mapped very recently as part of the concerted effort to provide high quality maps for humanitarian relief efforts for Typhoon (Yolanda) in the Philippines.

Collapsing house, Babbacombe
  © Copyright Derek Harper and licensed for reuse under this Creative Commons Licence.

Can we learn anything from how usage of this tag has evolved?

For me these are the significant points:

1. Simplicity prevailed over exactitude in the tagging. We still don't cleanly separate building type and status. This is not just because the simple tag is much easier to remember and apply: reusing an existing tag often means that downstream data consumers require little change to meet the particular humanitarian situation. An additional tag for building status would have required a whole host of changes to allow simple maps to be rendered. Using building=collapsed meant that one rule needed to be added and maps (see image above) were being produced within hours of the tag going into use.
2. Tags for major disasters often reflect things which happen anyway, but which in the normal course of things are unusual. Buildings collapse from time to time: from storm damage, after fires, from structural defects and so on. It's just that for those of us living in North America and Europe these are relatively rare events.
   
   The advantage of a worldwide database is that, however rare certain events are locally, they will be happening somewhere and if there is a mapper there they will want to know how to tag it. Although in this case the tag was created by armchair mappers. I'm sure that there are plenty of tags already in use for particular situations somewhere in the world which would fix tagging problems in my local patch.
   
   
3. Tags persist. Once a tag enters widespread usage it is very hard to kill. The OSM wiki is full of deprecated tags, XXX 2.0 schemas etc. However well-intentioned these ideas just don't match how tagging works in practice. Despite 3 years of advocacy of detailed tagging guidelines for public transport we still have plenty of highway=bus_stop tags. (More on this type of problem anon).

In conclusion, a tag developed to meet the particular circumstances of the 2010 Haiti Earthquake has been found to be useful outside its original scope to the extent it's now entered the tagging vocabulary of the local patch mapper. Mapping disasters isn't just a good way to try and help other people who are suffering, but it hones mapping skills which can be applied not only in other disasters but in everyday mapping.

Food Hygiene Open Data : an easy way into mapping addresses and postcodes

I've written a bit about the Food Safety Agency's Food Hygiene Open Data (henceforward FHRS) before, but as it was the focus of my hacking at last weeks London Hack Weekend, it's about time it had a dedicated post.

FHRS premises spread out along a line normal to shortest line to associated street (See article for details)
Data (c) from OSM contributors 2013, and Food Hygiene Rating Scheme (OGL)

To recap the salient points about FHRS data are:

• It's pretty comprehensive, covering most local authorities in the UK.
• It represents more than 15% of all postcodes.
• Most records come with full address data and a location (the postcode centroid).
• Premises (shops, pubs, cafes etc.,) are grouped into categories which map fairly easily to OSM tags. 
• Daily updates. High data currency (last reviewed date on all records).
• It's Open Data under the plain vanilla Open Government Licence.

For me the obvious uses of this data with OpenStreetMap are:

British Postcodes on OpenStreetMap

British contributors to OpenStreetMap are known for an apparently unhealthy obsession with postcodes.

London Postcode Sectors with more than 5% of regular postcodes mapped on OpenStreetMap.
See image for copyright notices. All completeness percentages are shown below.

We are not alone in Britain: there are many groups who need access to postcode data. This is because postcodes in the United Kingdom provide an excellent proxy for geolocation, and as such are widely used: in SatNavs; for geodemographics (such as Acorn, Mosaic, and the (open) ONS Output Area Classfication ); in a host of analytic applications, as well as their more prosaic role in delivering the post. In 2010 postcode centroids were made available through the Ordnance Survey Open Data scheme, under the brand CodePoint Open. Subsequently it was found that the license associated with this data prevented it being used directly in OSM. More recently the Office of National Statistics have released an (identical) data set which is not encumbered by the license of CodePoint Open.

Harry we missed you! London OSM Hack Weekend

Thura (half out of picture, sorry), Dan, Matt and Shaun do a "Harry"

I attended my first ever OSM Hack event over the weekend. I knew I needed to be down in South East England around the end of November so I managed to schedule my plans to include the hack event at AOL/MapQuest London offices. My own contribution was slight because I was still sorting out moving data from my old laptop to my new one, and didn't really achieve this until Sunday morning.

Andrew, Thura, Dan, Matt, Shaun, Grant, Serge, Mick, me (standing), Jon
(photo Simon, Cyclestreets)

The rest of this post is merely a collection of quick bullet points about things which happened, were talked about and so over the two days.

• The major thing that was done on the Saturday was updating the main OSM website: most OSM users should have noticed these changes by now. Not everyone is happy (although this is a given for any change on the OSM site), and following SomeoneElse's comments about the browse pages I have to agree with him that giving the map more space reduces the usefulness of these pages. However, I am equally sure that most glitches will get ironed out quite quickly.


• Dan worked on getting offline maps running on Firefox phones. (Several of us were interested in Dan's digitizing tablet too).


• Dan, and perhaps others, did some work on the HOT OSM Tasking Manager.


• Frederik brought a case of Club Mate along. To my surprise I found it quite palatable.


• Half of OSM London are getting married next year. Congratulations to Shaun and Andrea, Harry and Francine, & Derick and Morag.


• Simon, Shaun and I had a long chat about whether OSM can be used for sensible multi-modal transport planning.


• Robert was working on That Shouldn't be Possible, his nifty tool for finding bad map data by comparing OSM data with real GPS traces of routes.


• Serge, Robert, Grant and I had a passionate, but I hope, good-natured, debate about the FDA requirement that 23andMe's DNA sequence kits need to be licensed. I plan to blog about genetics, open data, big data and related topics real soon now: it was an opportunity to rehearse some of my arguments.


• Kat Arney - embryologist, science communicator, journalist, harpist, and knitter - was around on Saturday morning. She interviewed most of us: so one or two of the more coherent statements may appear on a BBC Radio programme in the New Year. Her parting shot when the pizza arrived was that we lived up to all the stereotypes of hackers! (Male, Club Mate drinking, pizza eaters.)

Food Hygiene postcodes buffered to 100m matched to OSM named highways.
(Note NG8 2AE which does not find any highways).
(Image contains copyright Ordnance Survey, Royal Mail and OpenStreetMap open data)

• MickO and I made a start on working with Food Hygiene Open Data. Mick wrote a parser for the XML and then generated a shape file from a subset of the data. I (eventually) managed to match FHRS postcodes to CodePoint Open postcodes and then find all OSM streets within 100 metres. Mick tried inputing the FHRS data into Nominatim, but for some reason Nominatim won't match the full addresses from FHRS. (I regard this dataset as an excellent exemplar for trying out a host of issues with fuzzy matching of OSM data to external data and conflation).


• A wikipedian, whose name I didn't catch, visited late on Sunday afternoon. Like Kat, who has written a chapter about my former boss in a forthcoming book, he has been involved in improving the coverage of the work of female scientists in public media. (Wikipedia have an initiative and a lot of work was done recently on Ada Lovelace day). He wanted advice on what information he could collect and in particular about geotagging photos.

MapQuest (Matt) looked after us well, and we had a couple of enjoyable (and not particularly boozy) sessions in pubs in the evening. All-in-all a very interesting and diverse couple of days. Look forward to doing it again late January, early February.