Lamp-posts and Botanical Records

Readers of this blog will know I have an fondness for using Local Government Open Data on the positions of street lights (see here, here, here and here). Over the last couple of weeks I've found another one.

Newly planted Quercus robur next to Derby Road Nottingham
note lamp post!

My original reason for buying a GPS was to make it easier to record wildlife records: I'm only 5 years into the minor distraction called OpenStreetMap! However, over the New Year I managed to drag myself away from addresses and postcodes and participated in the BSBI (the national botanical society) New Year's Day Plant Hunt. This is a relatively recent innovation, but has captured the attention of more and more botanists. The idea is to find as many different species of plants in flower during a 3 hour survey window.

One that got away. I missed this fine Common Knapweed Centaurea nigra on Jan 1.

Many people have been intrigued by common flowers appearing out of season. (Of course, some, such as Gorse are rarely seen out of flower ("When gorse is out of bloom, kissing is out of season"). Milder winters almost certainly due to global climate change are an important factor in allowing some plants to continue flowering. In other cases management of the land can cause plants to make new growth in the Autumn. Another possibility is that because the length of the day is the same in the Autumn as in the Spring that plants respond to some photoperiod effect. I've been noticing these things for years but never recorded them in a systematic way.

Green Alkanet Pentaglottis sempervirens
An attractive enough flower, both for bees and people, but a pernicious weed which seems to infest everywhere.

Tim Rich's very successful expedition in Cardiff on 1st January 2012 really set the ball rolling. Tim is a professional botanists, co-author of the Plant Crib, an indispensable guide to difficult plants, and author of a number of specialised to very specialised identification guides. Amazingly for someone who has been a bedrock for supporting amateur botanists across the country over the past 20 years, he is also out of work.  He's not the only one: it seems each year I get more mail bounced back or a new email address for a museum professional who has 'been let go'. (For non-naturalist readers, taxonomists are roughly equivalent to sysadmins in the botanical world: lots of things wouldn't happen without them, but they're not noticed when things are going OK).

Yarrow Achillea millefolium
Yarrow plants seem to thrive at this time of year, rather than just 'hanging on in'.

Another aside, people like Tim also play another non-obvious role: that of encouraging future generations of scientists. Many kids start out with an interest in natural history which can, if encouraged, lead to a careers in a whole slate of economically important areas such as medicine, and molecular biology, and not just ecology. Reducing the number of able scientists in museums who also perform substantial outreach and education activities, is eating the seed-corn for far more than just their direct specialisms. At least one British Nobel Laureate was inspired in this way.

Anyway, back two years ago I was really pleased to hear Tim on the Radio 4 Today programme describing his first plant hunt: my casual interest was validated by a similar interest by a serious professional botanist.

Red Valerian Centranthus ruber, growing in Churchyard car park.
Normally a very vigorous population of these pop-up all over the churchyard in Summer.

I managed to do about an hour's surveying on January 1, 2013, but still managed to find 20 plants in flower. This year I was determined to do better. I wasn't able to join the Leicester botanists who set the bar high with 59 or so species. The weather and my health prevented me finding much before the end of the year.

However, the rain stayed off on the afternoon of New Year's Day, which meant I could visit all the places I'd already seen plants in flower, and spend more time looking for others. My main locations for searching for plants where: two areas of parkland ,both campuses of Nottingham University; grass verges along some local roads; a local churchyard; and the car parking around local shops. Many plants survive in the road gutters, or at the edge of the pavement and a wall or on little patches which only get cut from time to time.

Short daylight hours meant that it was getting dark towards the end of my survey
Ribwort Plantain Plantago lanceolata

My advantage is that I have used most of these places to learn how to identify plants: I thus know exactly where to find around 400 species of flowering plant within 1 km of home. Naturalists call this 'patch watching' (The Urban Birder talks a lot about the concept): it works because one builds up an intimate knowledge of one's local flora. Things which are new or unfamiliar stand out remarkably quickly in this context (in computing terms, its a very efficient vdiff). Unfortunately familiarity with a patch can also mean that one struggles to identify ordinary plants in an unfamiliar context.

unprepossessing

A bit of unprepossessing highway verge, a typical place worth hunting in.
At least 6 different plants in flower here, fortunately it hadn't been cut in late Autumn.
Plants: Yarrow, Groundsel, Dandelion, Chickweed, Wall Barley and Shepherd's Purse.

For my recording I used the Obsmapp Android app, which meant that I didn't have to carry much with me. The survey time was about right before the battery ran down. I also collected samples of many of the plants to double check my identifications down a microscope when I got home. (A cold dank January day with poor light is not the place to check if a plant has star-shaped hairs on the undersides of it's leaves).

One minor problem with Obsmapp is that you need to make an identification at the time (or at least use a known species as a place holder). Now like many naturalists I'm not great at identifying grasses or dandelion-like flowers. For the latter I used a place holder species and checked the identification when I got home. However there were several grass specimens which I was not at all sure about. I've never chunked grasses in such a way that I can make a good approximate identification: I have to take them home and use a key.

Red Dead-nettle Lamium purpureum

This meant that I could not record the geolocation of these specimens through Obsmapp. Fortunately, street lights were to hand: instead I took a photo (geolocated, but that's not too important) of the nearest lamp-post's asset identifier. I then used the Nottingham Open Data Street Lamp dataset to get the lat/lon for the lamp-post and added these records into Observado once I got home.

A tuft of False Oat Grass was under this lamp post

All in all I managed to get 40 different species (my unknown grasses all turned out to be False Oat Grass). This was a very gratifying total. Annoyingly, I walked straight past a fine example of Knapweed which I found a couple of days later, and I noticed a Gorse bush which I was unaware off only on Monday!

One other aspect of looking for flowering plants at this time of year is that often they are stunted and don't always have the same habit as in the Spring and Summer. It is therefore an excellent way to challenge and improve one's identification skills. Not least know that in Britain we have Poland and Clements' Vegetative Key for identifying the plants when not in flower. I also used Eggenberg's lovely Flora Vegetativa which has many fine illustrations which complement the Vegetative Flora. (even though this is book on Swiss plants it works very well in Britain).

The other day I noticed a newly planted oak tree (one of 10 trees planted at the behest of our local Councillor Sally Longford to replace ones which had to be felled because of disease). Although I took geo-located photos I took great care to include 2 lamp-posts to enable me to locate this new tree accurately before adding it to OpenStreetMap.

Bee Orchid at Nottingham University Jubilee Campus
(location originally noted using lighting asset numbers)

I've previously used the asset numbers on the lighting in the University to check on locations of some Orchids in the Jubilee Campus, and I've also used them as markers when trying to map the trees in a belt of woodland more accurately.

So for urban wildlife, lamp-posts can be an invaluable resource for accurately locating specimens!

Maps for Dogs, or Lamp Posts in chains

Wet Lamp Post, photo by Don3rdSE (all rights reserved)

Not more stuff on street lights, 'fraid so.

In my earlier post I mentioned that it would be nice to see if it was possible to find the location of streets simply by linking together all the lamp posts on the same street. This is a (technical) account of how I achieved this using PostGIS.

Lamp Posts to Addresses: gold dust in the attributes

Local Government Open Data 4

I'm so used to a lot of open data being poorly attributed that I did not look at the Nottingham street light data until I'd imported it into PostGIS. Apart from lacking the asset identifier, and having redundant fields for the x and y positions of the geometry, it has two fields of fantastic value for enhancing OpenStreetMap.

Lenton Sands area of Nottingham showing House Numbers from Lamp Post data overlaid on OSM

Street Lights: Local Government Open Data 3

Another data set released by my local council contains the locations of all the street lights owned (or operated) by the council. Now I have no intention (or any desire) to import this data into OpenStreetMap. Is it any use in the open data context?

There are two obvious questions that can be asked using street light data:

• How well mapped are all public highways (i.e., including footways and cycleways) in the city?
• Are the highways on OSM accurately mapped?

Street lights are spaced at specific intervals according to the type of highway, it's width and the nature of the light source. This means that their accurate placement is determined by the council beforehand, all location data is to the nearest metre. Since most lights are situated immediately adjacent to the road carriageway they will be a short distance from the road centre line (say under 10 metres).

It is trivial to buffer the network of highways from OSM (via a Geofabrik download) and then use geometry operators to find all lights within the highway buffer envelope, and all outside the envelope. Using a very simple constant buffer of 10 metres I obtained the following result:

Street lights which are not close to OSM roads (best viewed large).

Why do lamp-posts have asset numbers?

A few weeks ago there was an involved, and sometimes, heated, discussion on the main OSM mailing list about imports. Many of the comments were interesting and useful, but one particular strand has attracted my attention. A slightly grotesque paraphrase of these various messages might be "OSM is a poorly managed Computer Science project, with inadequate tools, particularly for version control."

Leaving aside that OSM is neither a project, nor managed: I'd like to focus on what seems to be a surprising mis-perception about OSM as a database.

Firstly, databases don't have to be digital or stored electronically. Phone books, card indexes, and many reference books are easily recognisable as databases. Therefore a primitive database operation is only valid if it can be applied also to non-digital media.

Secondly, and this directly follows, information stored in a database does not have to have a unique identifier (a primary key). There's nothing stopping a telephone number appearing several times in a phone book, or complete pages being duplicated.

Once data is moved onto digital media, it really helps to assign unique identifiers: data can be restructured to be stored more efficiently or be easier to change; it's easier to spot duplication or bad data. This is exactly what OSM does, nodes, ways and relations are identified internally by system generated keys.

And it's what my local council does. They maintain an asset register of street furniture (bollards, traffic signals, parking signs, parking meters) and within this application assign identifiers. These days all this information is geocoded and available in the council's GIS. This information is obviously useful for financial planning, maintenance and other activities. BUT, they've gone a step further and each asset now has it's system assigned number marked on it. WHY? Because, "replace the bulb in lamp 35621" is a lot more specific than "replace the bulb in the lamp outside 25 Main Street". There may be lamp standards opposite each other at that location, or there might be two Main Street's or there may be no number on 25 Main Street, or the house might have been demolished. However, this number DOES NOT uniquely identify the lamp-post: it might do when combined with its location data, location data of the organisation which has assigned the number and information about the status of the system used to generate the number.

What does this have to do with OSM (apart from the wonderful possibility of collecting lamp-post numbers). Well OSM is like my local council, except that our local patch is a bit bigger, and we cannot go stencilling numbers on anything we map. So we have no means to tie an OSM object to its corresponding thing in the real world.

A corollary to this is that we cannot confidently tie OSM objects to geolocated objects in other databases. There are far too many variables to even inspire confidence in fuzzy matches: when was the OSM data mapped? what sources were used? how accurately was it mapped? when was the external data mapped? how current is it? how complete is it? does it have unique identifiers? are the identifiers persistent?

So, we have a host of problems in matching data from an OSM dataset and an equivalent external dataset. These problems relate to location accuracy, temporal accuracy, matching identifiers, and accuracy of associated data.

A good example of these problems is shown by OS Locator Musical Chairs and ITO's OSM Analysis which compare OSM street name data for Great Britain with the OpenData Locator dataset from the Ordnance Survey. This is a nice clear domain with the OS Locator data being from a known source and date and from a highly reputable national mapping agency. In some areas we have enough separately sourced data in OSM to have a handle on how accurately we can match these datasets. In most areas in England about 0.5-1.0% of Locator records cannot be matched to OSM. (I am not aware of reverse statistics, but in a recent survey aimed at hunting down some 20-odd of the mis-matches I found 5 street names used for addresses which are not present in the OS dataset). Even different datasets from OSGB have enough inconsistency to prevent complete matching. And these cases are relatively easy ones.

There are also problems relating to the purpose of external datasets: cadastral data might not reflect the building outlines we would draw naturally (e.g., French & Spanish cadasters); hydrography data might be segmented for water-flow measurement (e.g., NHD); vector data might be optimised for rendering (OS OpenData VectorMapDistrict); road data might not need to be very accurate (TIGER). The imported data should be restructured to reflect what is important for OSM, not maintained in aspic for some putative update.

So those advocating data imports or having 'development forks' of OSM need to answer : how on earth can you easily relate objects between two different data sets, or even the same data set at different times. Alternatively, we could all add some stencils to our mapping toolkits, but even then we'd have to leave our armchairs.

Postscript: the council are busy replacing all the local lamp posts, wonder what number they'll put on the new ones.