Tagged with Free data

Sitting around the data campfire

Similar to Gail Ramster, I went along to the Friday afternoon part of UK GovCamp 2012 without really knowing why. I suspect most people would say the same thing. You go because… well, you never know which useful people you might bump into, and what interesting things you might hear about. Plus a colleague Janet Hughes was going, and I’d cleared my desk of essential work for the week.

Here are a few takeaway thoughts from my afternoon.

1. I barely knew anyone

It’s years since I was a fish in a geeky pool, active in the free culture movement, the KDE community, software patent activism and other odds and sods.

For the past five years or so I’ve moved onto land, or perhaps a coral reef, to be more involved with issues around the environment, housing and pay inequality. The past two or so have been working as a local government employee at the GLA, supporting Green Party Members of the London Assembly. They have pushed for open data, but it’s not exactly a hot topic in our weekly meetings. My only remaining connection has been OpenStreetMap, my one geeky obsession.

Still, it didn’t matter, go along even if you know no-one at all.

2. It was nice to reconnect with optimistic techies

The event reminded me of one of the things I most like about these crowds: they’re all optimistic about the future and enthusiastic about the common interest.

I’m glad I managed to quickly chat to a few people I did know, sort of… Gail via Twitter, and Giles Gibson from the Herne Hill Forum, but sadly I only said as much as “hello” to people like Emer Coleman and Chris Osborne. That’s what you get for arriving late and leaving early.

3. It’s more meaty than you’d think

That’s “meatspace” as in “the real physical world”, compared to “cyberspace” online. Compared to events a few years ago on open data and technology, most of the discussion I heard was about councils and companies working on staff structures and consultation processes, and then thinking about how technology and data could help.

I used to get frustrated with discussions that started with the assumption that open data and technology was going to revolutionise the world. That seemed upside down to me. So I was pleasantly surprised at this.

4. There’s a lot of “we”

Somebody pointed this out in one session – it’s very easy to apply “we” to the wider population when you really mean “we sort of people in this room”.

Often “we” are innovators or early adopters of ideas that become more mainstream, like using a smart phone to access services. Sometimes “we” are set to be a significant minority, like journalists, bloggers and politicians who use data to enhance their investigatory work. Just as often “we” are a world unto our own.

It’s fine, innocent mostly, typical of any event with like-minded people. It just grated on me when people talked about reconfiguring public services or management around their preferences, as though the rest of the world will thank them.

I might make a badge for myself if I go again, with the slogan “we’re not normal” or similar!

5. Theres a lot going on out there

Cocooned in City Hall, working on affordable housing or the pay gap, it’s hard to keep even a toe dipped in this pool. It was great hearing from so many people in so many walks of work and life doing so many useful things.

Sometimes when I map an area for OpenStreetMap, walking down a street noting house numbers, I feel a bit bewildered by all these people living here! London feels impossibly enormous. I left UKGovCamp feeling similarly bewildered by the enormity of work going on in this field, relative that is to my own small bits and pieces in my job and my free time.

Tagged , , , , , , , ,

Problems and possibilities with ward boundaries

Being actively involved in my local branch of the Green Party means I’ve spent a lot of time wandering around carrying a map of a local ward.

Almost nobody seems to know which ward they are in, often because the names are a bit abstract (e.g. “The Lane” in Peckham, which I presume is because “Rye Lane” runs through the middle) or because almost nobody would say they live in the area described (e.g. “Peckham Rye”, which has Peckham Rye Common and Park in the middle and includes areas normally thought to be part of East Dulwich and Nunhead).

Since the Ordnance Survey published open data, including political boundaries, it’s been possible to put this information into OpenStreetMap. I’ve finally bothered to start doing this for Southwark – you can see the results on this nice ITO map.

Unfortunately the default map on the OpenStreetMap homepage draws the names of the wards along the rather nice dotted boundaries, displacing actual road names and leaving junctions that could easily confuse the user. Here are three examples:

You’ll notice I’ve added “ward” to the end of the names to try and help, but it’s not much of a solution. Three different proposals have been put forward on the OpenStreetMap bug tracker (a dedicated map, hide them, make them less bold).

A simple solution to the problem above would be to remove the names; a more sophisticated solution would be to give road names priority, change the text colour to the purple of the boundary lines, and hope Mapnik allows us to offset the labels so you can have the two ward names either side of the line).

It’s a bit of a shame that they create a mess because they’re useful data to include in OpenStreetMap (much like trees, which I’ve asked for a solution to).

For example, the holy grail of software to help us canvass voters would be to connect the electoral register to the OSM database of houses, allowing us to visualise and manage information on voting intentions and canvasser visits by ward on a nice map.

Another useful application could be Nominatim, which could tell you the political boundaries that any chosen OSM-mapped home, business, park or set of co-ordinates lies within.

For now I just need to finish getting all those Southwark boundaries into the database…

Data quality

One other quick point. The London Borough of Southwark boundary was already in the database, but it’s not very well mapped.

It’s really important to know if a boundary runs down the middle of the road, so that homes on one side belong to one borough and homes on the other side to another borough; or whether the boundary is offset away from the road, usually down back gardens, so that all homes on both sides are in the same borough.

Fellow OpenStreetMappers should be careful to put the boundary in exactly the right position, ideally sharing nodes/ways with the actual roads where the boundary goes down the middle so it’s precise and won’t go wrong if somebody adjusts the road position.

Tagged , , ,

Why map data sometimes matters

I was contacted recently by a parent campaigning for a local school to ensure its admissions policy is properly applied. Over-subscribed schools like this one are a common source of frustration and worry up and down the country.

Here’s the rub. Which of these two homes would you say is closer to the school, and therefore more likely to secure a place?  By the way, I’m not sure that the location on the left actually is within the catchment area, it’s just a place I randomly chose to illustrate the coming point…

Routes to the school from two locations using CloudMade maps, the home on the right wins by 500m.

Parents at the location on the right were told they were too far from the school. The method they use to calculate safe distances to the school actually suggests that the location on the right is farther away than the location on the left!

Why?

Because they are calculating distances using a model that measures the distance as if you are driving a car. If you try that, you get a totally different result:

Routes plotted for cars to get to the school, the home on the left wins by 400m.

The school’s model uses the Ordnance Survey ITN maps, and apparently doesn’t account for this short footpath at the end of one road. It was pedestrianised 25 years ago.

Happily OpenStreetMap has all the relevant data (and a few minor corrections the parent, Jasia, pointed out to me) so anybody can plot the route to prove the point.

Incidentally, if you fancy showing your support for this campaign download this letter to the governors, sign it and send it to the address at the top of the document.

Tagged , , , , , , ,

Making open data maps the almost-easy way

One of the annoying things about open data is that you often need ninja skills to do anything with it. OpenStreetMap contains a wealth of geodata, but most tools make you jump through several steps involving the command line and all manner of data wrangling just to produce a custom map.

Maperitive tries to make it much easier to create nice looking maps. It has been in gestation since late 2007, and is now close to being easy to use.

It took me about half an hour of playing around to produce my first nice hiking map of Snowdon, although a problem with NASA’s elevation data led me on a frustrating journey to get Ordnance Survey open data in there to fill the gaps. I also had to work out Maperitive’s settings file for the way features are drawn to make the maps look a little neater and, well, British.

Making open data maps the almost-easy way

(Click on the images to see them on Flickr, where you can look at full sized versions).

Another hour messing around with the settings file and I had a nice map of an area my new father in law likes to go walking, the Long Mynd in the south of Shropshire. This time I aimed for something familiar to users of the Ordnance Survey walking maps.

Making open data maps the almost-easy way

The latest beta of Maperitive also allows you to export a 3 dimensional model using elevation data, and a flat image of the map. You can import these into a modelling tool, laying the map image over the 3d model, to produce nice graphics like this one of walking routes up Snowdon:

Making open data maps the almost-easy way

If the NASA elevation data works for you and you don’t want to change the style of the maps, it’s already a fantastic and fairly usable free-to-download tool. It’s a shame it isn’t free software with the code open sourced.

Tagged , , , ,

Open scrutiny in the age of open data

This is the first of perhaps two or three short essays inspired by Emer Coleman‘s masters dissertation on open data, written in a personal capacity and not as part of my job. In this post I want to look at what her proposed model of “iterative and adaptive open government” would mean for scrutiny of the Mayor of London. Her dissertation considers the difference between the New Public Management approach, characterised by public managers setting the goals and other public managers auditing their performance, and an emerging “Open Governance” approach using open data.

Continue reading

Tagged , , , , , , ,

Can crowdsourcing improve open data?

I’ve written a guest blog over at the London Datastore about efforts to provide really good open data through OpenStreetMap.

My main example is my work on OpenEcoMaps, which mixes imported (and checked) open data with crowdsourced OpenStreetMap data to provide a great environmental geodata resource. My concluding hope? That “this will spur growing interest amongst data hoarders in data collaboration rather than plain old data dumps.” Have a read.

Tagged , ,

Maps, open data and activism on the Heygate estate

Andy Allan’s excellent post on cycle campaigning reminded me to blog about some mapping help I’ve given a campaign group called the Elephant Amenity Network. One of their long-running issues has been the clearance and demolition of the unfairly maligned Heygate Estate, over 1000 council homes that should have been refurbished for council tenants instead of being knocked down for aspiring home owners to move into the area.

A photo of the Heygate Estate in Elephant & Castle, London

One of the best features of the Heygate Estate is the urban forest that has grown there in the past thirty or forty years. But the few remaining residents and local campaigners fear the “regeneration” will see many or even most of them cut down.

Through a friend who is involved with the campaign, I came along to help them map the trees that are there now. Knowing what you have seems like a good first step to saving it.

So I helped them enter the trees into OpenStreetMap using the OpenEcoMaps install of the Potlatch 2 editor, set-up a simple map that shows them as clickable objects on a map, and provided them with a spreadsheet of all the data at the end of the process.

Some other clever bods in the campaign then used a system called CAVAT (Capital Asset Value for Amenity Trees) that puts a financial value on the trees. They estimate the value to be well in excess of £7.6m! Here is the CAVAT valuation laid on top of OpenStreetMap:

I’m glad to say this has become a case study in a recent London Assembly report into the state of street trees in London, which makes recommendations about the need for open street tree data and uses this Heygate mapping to show both the demand for this data and how useful it can be.

Since mapping the trees ourselves, I’ve received a file with all the trees in Southwark from the council with permission to use and share it, which is brilliant. I did a test import in East Dulwich/Peckham Rye, but stopped short because of rendering issues. It would be great to be able to import the lot and see if citizens can keep the data more up to date than the council, or perhaps even collaborate with the council and Trees for Cities?

Tagged , , , , , , , ,

Announcing OpenEcoMaps to geeks

I’m happy to announce that OpenEcoMaps is now stable and ready for use, albeit with a few wrinkles that I hope some more able hackers can help me iron out. OpenEcoMaps takes data about “eco” (green / sustainable) features stored in OpenStreetMap and turns them into KML files that are shown as overlays on the map, making it easy for people to find out where they can get a vegetarian meal, forage some wild fruit, spot a solar panel, recycle a can, pick up a car club car, or spend some money in a cinema.

You can use these KML files on your own map, or in Google Earth; you can embed the OpenEcoMaps map in your own web site; or you can just browse around the site.

At the moment there are packs of overlays for London and Exeter, but I can quite easily add other local areas with any combination of the features. I can also add in new features, they’re just little chunks of Python code to specify how OpenStreetMap tags are interpreted. Leave a comment if you want something in your area – I need a bounding box, the title for the area, and a set of layers specifying the title for the layer and the features you want in it.

I’ve been waiting for a reliable XAPI service before launching, and thanks to Mapquest I’m now able to say – use and abuse it!

Most of the data was added by various OpenStreetMap mappers over the years, it’s really fun to pull out something you think is very niche and find how much we already have in the database. But I’ve also been sourcing open data from organisations like the GLA and Southwark Council, particularly for renewable energy installations and food growing spaces. What’s interesting is finding out just how little these public bodies actually know.

Over the summer I’ve got a few events with food organisations where I’m going to present the project following my work with the council, the GLA and Sustain. That will be a nice opportunity to see how usable the average citizen finds the web site, and how useful it really is to them.

There are still a few features I think it needs before announcing it to networks of local green groups like Transition Towns. So if you’re a geek interested in this project please have a look at the TODO list on github and get stuck in.

Tagged , , ,

Growing pains & getting data out of OpenStreetMap

One of the killer features of OpenStreetMap, which makes it completely different to Google Maps and the rest, is that we provide totally free geodata. In fact it’s really the primary purpose of OpenStreetMap – the various maps shown on the homepage are just a tasty preview.

For those of us lacking the time, money and skills enjoyed by some of the cooler data users (i.e. unable to run a dedicated server with a PostgreSQL database and all the programs and storage space needed to maintain an up-to-date clone of the OpenStreetMap database) there are two main ways to grab OpenStreetMap data.

One is to simply download a defined area using the API. The “export” tab on the web site and editors like JOSM make this easy. The disadvantage is that you get everything in that area. If you wanted to get, say, all the power generators in the UK you would need to download and process gigabytes of data for the whole of the UK to pull them out.

Thankfully we have the wonderful eXtended API (XAPI, or “zappy”). This lets you define the features you want to download, and the area you want them for. So I can easily download a small file just containing power generators in the UK.

I have used XAPI for my OpenEcoMaps project. It’s great. It lets me build a web site on cheap shared hosting without needing ninja data processing skills. I love it.

Unfortunately, for the past few months XAPI has gradually become so overloaded that it is useless. I don’t think I have been able to make a batch of five calls for data from XAPI for weeks. OpenEcoMaps tries to download fresh data every hour, to no avail. Downloads either time out, return empty files, or return partial files that stop half-way through.

As a result I’ve not been able to launch OpenEcoMaps, and I’ve had to slow down conversations with local community groups, charities, councils and regional government about various exciting collaboration ideas.

I don’t begrudge hard-pressed volunteers who set XAPI up, especially when so much good work has gone into making the main web site, the default map, the wiki and the underlying database run so reliably.

But I wish and cross my fingers for the day when there is a resource – either from the community or a company – to run a reliable XAPI service. I’d even be willing to pay a small annual fee for access. In the meantime, sadly for me, the work I’ve been building up over the past few years networking with those other groups has ground to a halt because of my inability to get data out of OpenStreetMap. Suggestions are welcome.

Tagged , ,

Matchmaking open data geeks and local mappers

Two parallel worlds are starting to rub up against each other – open data enthusiasts and local activist groups. As Sam Smith has pointed out, embedding the power of open data in other worlds such as local activism has barely begun.

Maps are one medium where I’ve been trying to bring these worlds together.

Stepping into the ring

In the left corner we have people like Rob Hopkins, who has just written a great summary of Transition Town groups mapping wild food, local groups and visions of the future. This wonderful work makes use of relatively open tools like Google Maps, but (so far as I can see) they make absolutely no use of open data, and keep all of their data in their own separate mapping systems.

In the right corner we have open data crowds like OpenStreetMap, and after some prodding from me the Greater London Authority and the Department for Energy and Climate Change. Together we have stacks of open data on renewable energy generators, allotments, recycling bins and more. But so far we haven’t made it easy for activists who aren’t super-geeks to do interesting things with this data, nor to use platforms like OpenStreetMap to store data they gather.

This is a great shame because both camps believe in the value and power of co-operation and collaboration.

Here in Southwark (south east London) I have found several local groups, the council and the Greater London Authority all trying to map local food growing, or at least interested in getting the results. Why not all work together on one open dataset that everyone can then use?

With OpenStreetMap it is possible for everybody and their dog to gather data of interest to them, and put it all in one place. That way you don’t duplicate effort, and you benefit from other people’s work.

It should also be possible to share the tools so local groups don’t need a resident geek to reinvent the wheel. Google Maps enabled people to make maps of local fruit and nut trees with ease; sadly OpenStreetMap has required too many geeky power skills to do this.

Touching knuckles

Which is why I have been working on the grandly-titled Sustainable London Map (ta-dah!) with much-appreciated help from another Sam Smith, Shaun McDonald and Andy Allan. This offers two tools for local groups:

First, easy access to the data we hold. My tool generates KML files with nice pointy clicky icons for all sorts of data related to low carbon power, waste, transport, food and culture. It pulls fresh data out of OpenStreetMap every hour. You can use these KML files on your own map or desktop programme, and you can embed the map itself if you don’t already have one.

Second, a customised editor (using Potlatch 2) that focuses only on the features that the map shows and that makes the presentation of all the OpenStreetMap data a little less overwhelming.

If every community group, charity and government body in London used OpenStreetMap then we would all be contributing to one definitive map instead of all doing our own thing ignorant of each other.

I have extended a hand to friends and contacts in my local Southwark who want to map food growing and renewable energy generators. Through various emails and pub meetups I hope they will begin to use the maps on their web sites (as Peckham Power have done) and to use the customised editor to enter new data.

I have also started discussions with staff at the GLA (who lead on Londonwide food strategy and projects like Capital Growth) and Southwark Council. To my slight surprise, they have been very enthusiastic about the potential of this work. If our tentative first steps in Southwark bear fruit, there is interest in rolling this approach out across London.

Pulling my punches

Given that this is a hobby, competing with a life and my Green Party responsibilities, I’m taking it all quite slowly. I know there are good reasons whymany groups will want to stick with the tools they already have, perhaps because they don’t have the time to make the switch, or because we don’t yet offer something they need.

But if you’re involved with any mapping exercises for local community groups and would like to find out how you could make better use of open data, or if you’re an open map data geek interested in helping bridge the divide with local groups, get in touch by leaving a comment below.

Tagged , , , , , , , , , ,
Follow

Get every new post delivered to your Inbox.