Here are a few takeaway thoughts from my afternoon.

1. I barely knew anyone

It’s years since I was a fish in a geeky pool, active in the free culture movement, the KDE community, software patent activism and other odds and sods.

For the past five years or so I’ve moved onto land, or perhaps a coral reef, to be more involved with issues around the environment, housing and pay inequality. The past two or so have been working as a local government employee at the GLA, supporting Green Party Members of the London Assembly. They have pushed for open data, but it’s not exactly a hot topic in our weekly meetings. My only remaining connection has been OpenStreetMap, my one geeky obsession.

Still, it didn’t matter, go along even if you know no-one at all.

2. It was nice to reconnect with optimistic techies

The event reminded me of one of the things I most like about these crowds: they’re all optimistic about the future and enthusiastic about the common interest.

I’m glad I managed to quickly chat to a few people I did know, sort of… Gail via Twitter, and Giles Gibson from the Herne Hill Forum, but sadly I only said as much as “hello” to people like Emer Coleman and Chris Osborne. That’s what you get for arriving late and leaving early.

3. It’s more meaty than you’d think

That’s “meatspace” as in “the real physical world”, compared to “cyberspace” online. Compared to events a few years ago on open data and technology, most of the discussion I heard was about councils and companies working on staff structures and consultation processes, and then thinking about how technology and data could help.

I used to get frustrated with discussions that started with the assumption that open data and technology was going to revolutionise the world. That seemed upside down to me. So I was pleasantly surprised at this.

4. There’s a lot of “we”

Somebody pointed this out in one session – it’s very easy to apply “we” to the wider population when you really mean “we sort of people in this room”.

Often “we” are innovators or early adopters of ideas that become more mainstream, like using a smart phone to access services. Sometimes “we” are set to be a significant minority, like journalists, bloggers and politicians who use data to enhance their investigatory work. Just as often “we” are a world unto our own.

It’s fine, innocent mostly, typical of any event with like-minded people. It just grated on me when people talked about reconfiguring public services or management around their preferences, as though the rest of the world will thank them.

I might make a badge for myself if I go again, with the slogan “we’re not normal” or similar!

5. Theres a lot going on out there

Cocooned in City Hall, working on affordable housing or the pay gap, it’s hard to keep even a toe dipped in this pool. It was great hearing from so many people in so many walks of work and life doing so many useful things.

Sometimes when I map an area for OpenStreetMap, walking down a street noting house numbers, I feel a bit bewildered by all these people living here! London feels impossibly enormous. I left UKGovCamp feeling similarly bewildered by the enormity of work going on in this field, relative that is to my own small bits and pieces in my job and my free time.

Almost nobody seems to know which ward they are in, often because the names are a bit abstract (e.g. “The Lane” in Peckham, which I presume is because “Rye Lane” runs through the middle) or because almost nobody would say they live in the area described (e.g. “Peckham Rye”, which has Peckham Rye Common and Park in the middle and includes areas normally thought to be part of East Dulwich and Nunhead).

Since the Ordnance Survey published open data, including political boundaries, it’s been possible to put this information into OpenStreetMap. I’ve finally bothered to start doing this for Southwark – you can see the results on this nice ITO map.

Unfortunately the default map on the OpenStreetMap homepage draws the names of the wards along the rather nice dotted boundaries, displacing actual road names and leaving junctions that could easily confuse the user. Here are three examples:

You’ll notice I’ve added “ward” to the end of the names to try and help, but it’s not much of a solution. Three different proposals have been put forward on the OpenStreetMap bug tracker (a dedicated map, hide them, make them less bold).

A simple solution to the problem above would be to remove the names; a more sophisticated solution would be to give road names priority, change the text colour to the purple of the boundary lines, and hope Mapnik allows us to offset the labels so you can have the two ward names either side of the line).

It’s a bit of a shame that they create a mess because they’re useful data to include in OpenStreetMap (much like trees, which I’ve asked for a solution to).

For example, the holy grail of software to help us canvass voters would be to connect the electoral register to the OSM database of houses, allowing us to visualise and manage information on voting intentions and canvasser visits by ward on a nice map.

Another useful application could be Nominatim, which could tell you the political boundaries that any chosen OSM-mapped home, business, park or set of co-ordinates lies within.

For now I just need to finish getting all those Southwark boundaries into the database…

Data quality

One other quick point. The London Borough of Southwark boundary was already in the database, but it’s not very well mapped.

It’s really important to know if a boundary runs down the middle of the road, so that homes on one side belong to one borough and homes on the other side to another borough; or whether the boundary is offset away from the road, usually down back gardens, so that all homes on both sides are in the same borough.

Fellow OpenStreetMappers should be careful to put the boundary in exactly the right position, ideally sharing nodes/ways with the actual roads where the boundary goes down the middle so it’s precise and won’t go wrong if somebody adjusts the road position.

Tagged: Elections, Free data, OpenStreetMap, Southwark ]]> http://tom.acrewoods.net/2011/12/08/problems-and-possibilities-with-ward-boundaries/feed/ 3 tomchance Problems and possibilities with ward boundaries http://tom.acrewoods.net/2011/12/01/why-map-data-sometimes-matters/ http://tom.acrewoods.net/2011/12/01/why-map-data-sometimes-matters/#comments Thu, 01 Dec 2011 20:53:52 +0000 Tom Chance http://tom.acrewoods.net/?p=514 ]]> I was contacted recently by a parent campaigning for a local school to ensure its admissions policy is properly applied. Over-subscribed schools like this one are a common source of frustration and worry up and down the country.

Here’s the rub. Which of these two homes would you say is closer to the school, and therefore more likely to secure a place?  By the way, I’m not sure that the location on the left actually is within the catchment area, it’s just a place I randomly chose to illustrate the coming point…

Routes to the school from two locations using CloudMade maps, the home on the right wins by 500m.

Parents at the location on the right were told they were too far from the school. The method they use to calculate safe distances to the school actually suggests that the location on the right is farther away than the location on the left!

Why?

Because they are calculating distances using a model that measures the distance as if you are driving a car. If you try that, you get a totally different result:

Routes plotted for cars to get to the school, the home on the left wins by 400m.

The school’s model uses the Ordnance Survey ITN maps, and apparently doesn’t account for this short footpath at the end of one road. It was pedestrianised 25 years ago.

Happily OpenStreetMap has all the relevant data (and a few minor corrections the parent, Jasia, pointed out to me) so anybody can plot the route to prove the point.

Incidentally, if you fancy showing your support for this campaign download this letter to the governors, sign it and send it to the address at the top of the document.

Maperitive tries to make it much easier to create nice looking maps. It has been in gestation since late 2007, and is now close to being easy to use.

It took me about half an hour of playing around to produce my first nice hiking map of Snowdon, although a problem with NASA’s elevation data led me on a frustrating journey to get Ordnance Survey open data in there to fill the gaps. I also had to work out Maperitive’s settings file for the way features are drawn to make the maps look a little neater and, well, British.

(Click on the images to see them on Flickr, where you can look at full sized versions).

Another hour messing around with the settings file and I had a nice map of an area my new father in law likes to go walking, the Long Mynd in the south of Shropshire. This time I aimed for something familiar to users of the Ordnance Survey walking maps.

The latest beta of Maperitive also allows you to export a 3 dimensional model using elevation data, and a flat image of the map. You can import these into a modelling tool, laying the map image over the 3d model, to produce nice graphics like this one of walking routes up Snowdon:

If the NASA elevation data works for you and you don’t want to change the style of the maps, it’s already a fantastic and fairly usable free-to-download tool. It’s a shame it isn’t free software with the code open sourced.

Emer draws on the work of Jürgen Habermas, a sociologist and philosopher whose theory of the system and the lifeworld helps describe the difference between New Public Management and Open Governance. In short, the New Public Management model involves government (“the system”) choosing the tune and forcing the public (“the lifeworld”) to dance to that tune (“colonisation”). You can only hold government to account according to the data that public managers decide to collect and disseminate. Open Governance on the other hand allows for meaningful dialogue between government and civil society, leading to what Habermas called “communicative action” where people talk about complex issues and then act (and hold to account) based on a common understanding.

While the London Assembly has started to embrace open data and social media, I think it has been fairly superficial. Or rather, in Emer’s terms, it has remained within New Public Management model. Scrutiny of the Mayor is carried out by committees of Assembly Members with goals and performance established by public managers and politicians; social media is used for the most part to communicate that to the public.

The London Assembly, for those out of the loop, is an elected body of 25 politicians who hold the Mayor of London to account.

I want to pick up on a comment by Janet Hughes, Head of Scrutiny in the London Assembly, that “media, bloggers, members of the public… should all form part of an overall picture – its not about competing with each other but it’s about all these different parts coming together to ensure maximum accountability and transparency”.

I want to very quickly sketch how committees currently carry out their scrutiny work, then suggest a way in which they might embrace an open scrutiny model that allows for communicative action rather than colonisation. I don’t pretend to have arrived at a complete description of Scrutiny 2.0, but hopefully it will provoke some further comment.

Public scrutiny in the old model

When the Environment Committee carried out two investigations into street trees, first in 2007 and then again in 2011, the subject and the terms of reference for the scrutiny were drawn up by Assembly Members and support staff. In each case the public were invited to submit written evidence according to those terms and further framing questions; the public were already colonised by the system. Invited experts gave evidence at public meetings, with scant opportunity for further public input. The reports were then drafted by support staff entirely within the Assembly, behind closed doors, and when the Assembly Members signed off on the reports the public were offered a finished product.

Some very good scrutiny products, I should add. But to the extent that they could be involved further, the public were left to comment on the committee’s work and press Assembly Members if they wished the committee to follow any aspect up.

At times, there is even an understandable but worrying tendency to think that the public’s input isn’t particularly helpful. Committees often scrutinise complex technical subjects such as the housing market, and so highly trained support staff look to highly trained experts for input. No effort is made to engage in a wider discussion with the public; it is more time consuming and, compared to expert discussion, does little to advance scrutiny according to the terms and frames set within the system by support staff and politicians.

Now, the second report on street trees was particularly interesting because it identified lack of data as a major barrier to the committee’s being able to hold the relevant public bodies to account. In the New Public Management model, the system wasn’t enabling proper performance management.

It also carried a case study on the local community in Elephant & Castle crowdsourcing their own data to bolster their case for saving the “Heygate forest”, something I’ve blogged about here.

Darren Johnson, Green member of the Assembly visiting the Heygate Estate to meet local activists

The activists didn’t map the trees and estimate their financial value as part of an antagonistic anti-council and anti-developer campaign; they are trying to engage in a meaningful dialogue with both before the council and developer draw up plans behind closed doors that decide the fate of the trees. Planning consultations at that stage are a perfect example of the system colonising the lifeworld.

If only the council and developers would really embrace the activists’ open approach it would be a wonderful example of Habermas’ communicative action.

Open scrutiny

Now then, what would the Assembly committee’s next scrutiny on street trees look like if this process really was “adaptive and iterative”, if the committee sought to “come together [with the public] to ensure maximum accountability and transparency”?

The second report published this year was iterative in that it updated the findings of the first, and adaptive in that it looked at new issues such as open data. But that iteration and adaptation was done within the system, by Assembly Members and support staff.

Perhaps in the next four years we will see more examples of activists using data to record and communicate their stories just like the Heygate activists – about trees being cut down, ruined by overzealous council pollarding or damaged by weapon dog training. If the support staff were to pick up on this not just as an indication of issues it should examine but as data and perspectives to be analysed then the iteration and adaptation between reports would start to be more open without needing a costly and difficult public consultation exercise.

Maybe social media like Twitter will also be so widely used by Assembly Members, support staff and activists that the work in between public meetings will be communicated and therefore in part conducted online, in public. Assembly Members might discuss terms of reference with activists; support staff and activists might discuss initial findings and potential recommendations. Bloggers from avid Mayor watchers to tree huggers would contribute their own more developed thoughts as the scrutiny process unfolds, offering useful new data, questions and perspectives for the committee to consider. Iteration and adaptation would be rapid, throughout the process of a single report than rather than in the years between reports.

In this model, the committee doesn’t assert itself outside the lifeworld, part of the system that scrutinises another part of the system. Instead, the committee conducts its scrutiny in partnership with the lifeworld, with the public.

The London Assembly remains relevant at least because of its well trained and managed staff, and necessary in view of its democratic function.

What do you think?

I’m no futurologist so my sketch of open scrutiny is probably deeply unadventurous. It is conceived from within the New Public Management model of a committee of politicians scrutinising a topic and producing a report. But I think it takes us beyond that model in an important sense, seeing scrutiny as a kind of communicative action where the committee, bloggers and activists come together to, in Janet’s words, “ensure maximum accountability and transparency”.

What do you think?

My main example is my work on OpenEcoMaps, which mixes imported (and checked) open data with crowdsourced OpenStreetMap data to provide a great environmental geodata resource. My concluding hope? That “this will spur growing interest amongst data hoarders in data collaboration rather than plain old data dumps.” Have a read.

A photo of the Heygate Estate in Elephant & Castle, London

One of the best features of the Heygate Estate is the urban forest that has grown there in the past thirty or forty years. But the few remaining residents and local campaigners fear the “regeneration” will see many or even most of them cut down.

Through a friend who is involved with the campaign, I came along to help them map the trees that are there now. Knowing what you have seems like a good first step to saving it.

So I helped them enter the trees into OpenStreetMap using the OpenEcoMaps install of the Potlatch 2 editor, set-up a simple map that shows them as clickable objects on a map, and provided them with a spreadsheet of all the data at the end of the process.

Some other clever bods in the campaign then used a system called CAVAT (Capital Asset Value for Amenity Trees) that puts a financial value on the trees. They estimate the value to be well in excess of £7.6m! Here is the CAVAT valuation laid on top of OpenStreetMap:

I’m glad to say this has become a case study in a recent London Assembly report into the state of street trees in London, which makes recommendations about the need for open street tree data and uses this Heygate mapping to show both the demand for this data and how useful it can be.

Since mapping the trees ourselves, I’ve received a file with all the trees in Southwark from the council with permission to use and share it, which is brilliant. I did a test import in East Dulwich/Peckham Rye, but stopped short because of rendering issues. It would be great to be able to import the lot and see if citizens can keep the data more up to date than the council, or perhaps even collaborate with the council and Trees for Cities?

You can use these KML files on your own map, or in Google Earth; you can embed the OpenEcoMaps map in your own web site; or you can just browse around the site.

At the moment there are packs of overlays for London and Exeter, but I can quite easily add other local areas with any combination of the features. I can also add in new features, they’re just little chunks of Python code to specify how OpenStreetMap tags are interpreted. Leave a comment if you want something in your area – I need a bounding box, the title for the area, and a set of layers specifying the title for the layer and the features you want in it.

I’ve been waiting for a reliable XAPI service before launching, and thanks to Mapquest I’m now able to say – use and abuse it!

Most of the data was added by various OpenStreetMap mappers over the years, it’s really fun to pull out something you think is very niche and find how much we already have in the database. But I’ve also been sourcing open data from organisations like the GLA and Southwark Council, particularly for renewable energy installations and food growing spaces. What’s interesting is finding out just how little these public bodies actually know.

Over the summer I’ve got a few events with food organisations where I’m going to present the project following my work with the council, the GLA and Sustain. That will be a nice opportunity to see how usable the average citizen finds the web site, and how useful it really is to them.

There are still a few features I think it needs before announcing it to networks of local green groups like Transition Towns. So if you’re a geek interested in this project please have a look at the TODO list on github and get stuck in.

For those of us lacking the time, money and skills enjoyed by some of the cooler data users (i.e. unable to run a dedicated server with a PostgreSQL database and all the programs and storage space needed to maintain an up-to-date clone of the OpenStreetMap database) there are two main ways to grab OpenStreetMap data.

One is to simply download a defined area using the API. The “export” tab on the web site and editors like JOSM make this easy. The disadvantage is that you get everything in that area. If you wanted to get, say, all the power generators in the UK you would need to download and process gigabytes of data for the whole of the UK to pull them out.

Thankfully we have the wonderful eXtended API (XAPI, or “zappy”). This lets you define the features you want to download, and the area you want them for. So I can easily download a small file just containing power generators in the UK.

I have used XAPI for my OpenEcoMaps project. It’s great. It lets me build a web site on cheap shared hosting without needing ninja data processing skills. I love it.

Unfortunately, for the past few months XAPI has gradually become so overloaded that it is useless. I don’t think I have been able to make a batch of five calls for data from XAPI for weeks. OpenEcoMaps tries to download fresh data every hour, to no avail. Downloads either time out, return empty files, or return partial files that stop half-way through.

As a result I’ve not been able to launch OpenEcoMaps, and I’ve had to slow down conversations with local community groups, charities, councils and regional government about various exciting collaboration ideas.

I don’t begrudge hard-pressed volunteers who set XAPI up, especially when so much good work has gone into making the main web site, the default map, the wiki and the underlying database run so reliably.

But I wish and cross my fingers for the day when there is a resource – either from the community or a company – to run a reliable XAPI service. I’d even be willing to pay a small annual fee for access. In the meantime, sadly for me, the work I’ve been building up over the past few years networking with those other groups has ground to a halt because of my inability to get data out of OpenStreetMap. Suggestions are welcome.

Maps are one medium where I’ve been trying to bring these worlds together.

Stepping into the ring

In the left corner we have people like Rob Hopkins, who has just written a great summary of Transition Town groups mapping wild food, local groups and visions of the future. This wonderful work makes use of relatively open tools like Google Maps, but (so far as I can see) they make absolutely no use of open data, and keep all of their data in their own separate mapping systems.

In the right corner we have open data crowds like OpenStreetMap, and after some prodding from me the Greater London Authority and the Department for Energy and Climate Change. Together we have stacks of open data on renewable energy generators, allotments, recycling bins and more. But so far we haven’t made it easy for activists who aren’t super-geeks to do interesting things with this data, nor to use platforms like OpenStreetMap to store data they gather.

This is a great shame because both camps believe in the value and power of co-operation and collaboration.

Here in Southwark (south east London) I have found several local groups, the council and the Greater London Authority all trying to map local food growing, or at least interested in getting the results. Why not all work together on one open dataset that everyone can then use?

With OpenStreetMap it is possible for everybody and their dog to gather data of interest to them, and put it all in one place. That way you don’t duplicate effort, and you benefit from other people’s work.

It should also be possible to share the tools so local groups don’t need a resident geek to reinvent the wheel. Google Maps enabled people to make maps of local fruit and nut trees with ease; sadly OpenStreetMap has required too many geeky power skills to do this.

Touching knuckles

Which is why I have been working on the grandly-titled Sustainable London Map (ta-dah!) with much-appreciated help from another Sam Smith, Shaun McDonald and Andy Allan. This offers two tools for local groups:

First, easy access to the data we hold. My tool generates KML files with nice pointy clicky icons for all sorts of data related to low carbon power, waste, transport, food and culture. It pulls fresh data out of OpenStreetMap every hour. You can use these KML files on your own map or desktop programme, and you can embed the map itself if you don’t already have one.

Second, a customised editor (using Potlatch 2) that focuses only on the features that the map shows and that makes the presentation of all the OpenStreetMap data a little less overwhelming.

If every community group, charity and government body in London used OpenStreetMap then we would all be contributing to one definitive map instead of all doing our own thing ignorant of each other.

I have extended a hand to friends and contacts in my local Southwark who want to map food growing and renewable energy generators. Through various emails and pub meetups I hope they will begin to use the maps on their web sites (as Peckham Power have done) and to use the customised editor to enter new data.

I have also started discussions with staff at the GLA (who lead on Londonwide food strategy and projects like Capital Growth) and Southwark Council. To my slight surprise, they have been very enthusiastic about the potential of this work. If our tentative first steps in Southwark bear fruit, there is interest in rolling this approach out across London.

Pulling my punches

Given that this is a hobby, competing with a life and my Green Party responsibilities, I’m taking it all quite slowly. I know there are good reasons whymany groups will want to stick with the tools they already have, perhaps because they don’t have the time to make the switch, or because we don’t yet offer something they need.

But if you’re involved with any mapping exercises for local community groups and would like to find out how you could make better use of open data, or if you’re an open map data geek interested in helping bridge the divide with local groups, get in touch by leaving a comment below.