I attended an interesting seminar at the Oxford Internet Institute today given by David Weinberger, a Fellow at the Harvard Berkman Center. He spoke about the "new" shape of knowledge in the digitised world, where it is able to escape the organisational restrictions previously imposed by our physical world. Whilst his talk was interesting, particularly in the way he drew trendy technological developments around some basic philosophical questions about knowledge, it was a bit evangelical for my taste. It reminded me of a previous, somewhat anecdotal blog entry I wrote on the value of blogs. So what follows are some reflections, worries and criticisms from his talk.

First I should sketch his main points for those not in the know :) Knowledge used to be organised in hierarchies such as Dewey's decimal system used by libraries, and the basic taxonomies of kingdoms, families, species, etc. used by biologists. This was necessary primarily (at least that was implied by David's talk) because we can only ever file a physical object away in one place; a book on military cooking has to go either in the military or cooking section of the library. But with digital information any one item can have multiple overlapping taxonomies. Moreover, he suggested, the less hierarchy we have and the more messy these taxonomies the better. The only justification he gave for the use of the word better was that the mess gives us a more fertile ground, with more data being contextualised in more interesting ways. I think his use of the word messy is a bit odd, since his best example of a very messy knowledge base was Wikipedia with all of its hyperlinks between articles. That's not messy to me, but I suppose he wanted to criticise the notion that a clean hierarchy designed top-down by someone like Dewey was flawed.

So we end up with a world of abundant digitised information loosely connected with highly subjective taxonomies, and this is good. We then filter this information ourselves, using tools that impose limited restrictions on our abilities. For example, rather than having Yahoo! bookmark the web a lot of people now use del.icio.us, which allows users to freely tag bookmarks. These tags intermingle and can be browsed creating a form of social filtering, as opposed to the editorial filtering exercised by Yahoo! We can find information that's good enough and filter the abundance of information socially rather than relying on questionable institutions and their experts to do it for us.

Ok, so that's a very basic sketch of his position. Here are my thoughts.

What's good enough?

A key point of David's talk was that, in this new era of widely distributed knowledge, the knowledge that we can find and use via social filters will almost always be "good enough". When I asked him about this he obviously understood the following point, but I fear he is overly optimistic about its practical implications.

What's good enough will depend upon the context, which will have its own standards and intentions. So for example, a lumpy lawn will be good enough for a rugby match but it will need to be rolled flat for a game of croquet. Moreover, if we want to know whether or not it is actually flat without being able to test every square metre of the lawn it's important to know the intentions of the groundskeeper; if he is about to quit his job and hates everyone who uses the lawn you'd be wise to check he did a good job!

The problem of standards for knowledge is easy to grasp. Let's say, for example, that I'm looking for some instructions for a tool I've just bought. I get some instructions off a blog and they're wrong. Now if I were technically minded I might be able to work out, using those instructions, how to use the tool anyway, so the knowledge would be good enough. But if I were inexperienced or incompetent the knowledge wouldn't be good enough, and by applying the inaccuracies in the knowledge I could do myself harm. These standards aren't a feature of the knowledge itself, but rather of the context in which it is used.

The problem of intentions is similar. At the moment I'm quite confused about the question of nuclear power in the UK, so I'm always looking for reliable data on the costs and environmental impacts of nuclear. I find a document on the web and have little idea of who wrote it, but I find the arguments persuasive. But hang on, who did write it? Here are two potential authors: the first is a politically motivated green who has deep-seated, partially socially constructed, intuitions about the costs of nuclear power but little evidence; the second is a dedicated citizen journalist who has read a variety of reputable sources, considered them carefully and come up with the same conclusions as the first author. I want something approaching objectivity here, I don't want to enter a public debate through street campaigning and lobbying based upon misleading, biased information.

I will also have intentions myself as a reader, which will relate to or include certain standards. I might read the Wikipedia entry on the science wars out of vague interest, or as part of my preparation for a debate, or even for an undergraduate essay on the philosophy of science. The article looks reputable, it covers a few angles and refers to the eminent thinkers in the dispute. But what if it doesn't meet the standards that my intentions demand? What if I want something that gives a good account of the scientific side of the story, seek out a highly biased polemic instead that fits my argument and run with that? I'm now operating on what would, by a higher standard, be judged misleading or even wholly inaccurate information.

So when we ask: What's good enough? We're opening a big can of worms. David thinks that people's habits on the internet are such that we need not worry, since they will generally put accurate information up or at least any bias or intention that might introduce inaccuracies will be obvious. People will naturally throw so much metadata out there that we will be liberated from the limitations of Dewey and co.

But I see no evidence for this, other than some circumstantial stuff to do with particular web sites working out OK at the moment. To make this claim work you'd need to demonstrate a psychological/sociological tendency towards adding data that will be good enough for people's purposes. That or defend a normative point, that the messy pile of leaves is simply better than a clean tree.

The politics of knowledge coherence

These worries are most profound when we consider the political implications. It's not that big a deal when we're deciding whether or not a pint of beer is good enough, or whether a photo on Flickr actually shows the person noted in the description. But when it comes to knowledge that we use for actions of consequence, such as feeding a child particular products because you're assured that they're healthy, or campaigning against nuclear power because you're convinced that it's a costly mistake, some kind of certainty is important.

Now I don't want to suggest that the institutional media, which David criticised a lot, are perfect here. I've read enough of Chomsky, Herman, Pilger and other leftie media critics to know how fallible, biased and corrupt even institutions like the BBC can be. But there can be no doubt that we think that, for example, a report in The Guardian is more reliable than a conversation with a random man in a pub. The reason is that we assume their intention is to investigate and convey the truth, and their standards are significantly higher. And despite my reservations I still read most of my news from The Guardian, FOX News (for the other side, and a laugh), Newsforge and New Statesman. I don't have time to be participating in social filtering schemes, hunting down reputable bloggers and the like. I just want an hour-long hit of news every morning through the RSS feeds and a couple of magazines.

It's also revealing to first point out a trend in the institutional media towards opinion pieces, polarisation and sensationalism. It sells, even if it betrays my trust in their intentions and standards. Nowhere is this more stark than FOX. This suggests I have some deep intuition about the role of the media and the standards which they should meet.

If millions of people trust FOX and absorb their pro-corporate religious fundamentalism, what makes David think that people will be much more discerning with knowledge spread around the internet? Blogs are already congregating into ecosystems, with their blogrolls and hyperlinking arguably reinforcing their own viewpoint rather than stimulating debate and challenging lazy assumptions. Webs of coherence are great as a theory of knowledge organisation and basic justification, but you need either virtuous social tendencies, political mechanisms that lock those tendencies in or some other standard against which to hold knowledge to account.

In other words, without empirical evidence to the contrary I'm inclined to think that the internet is neither inherently better or inherently worse when it comes to delivering knowledge that is good enough. It can be better - there are some excellent examples of citizen journalism - but there is also the worry that our access to 'good enough' knowledge will decrease with a slide from reliable information to masses of subjective views with no popular and widely-exercised framework to discern between them. I think this is already happening.

Range and representation

There's another side to this question, which is the range of knowledge you're exposing yourself to and the way in which it is represented. One of the supposed functions of a news editors (in any part of the media) is to cover a decent range of stories, mostly of the sort that their 'consumers' will want but also some that are deemed to be in the public interest. They're also supposed to ensure that their journalists give an accurate representation, based on first-hand accounts that can be corroborated.

Whether or not this works in the established media institutions is one thing. Whether the piles of leaves and social filters provide this is another, equally important question. By devolving editorial control to individuals and popular social filters we're trusting that people will still receive the appropriate range and representation of current affairs (whatever appropriate means). I've no doubt that a lot of people are now getting a much wider range of stories, and that a lot of the representation will be individual perspectives on the established institution's representations. So in this sense the piles of loosely connected leaves provide people with, as David said, a more fertile ground for coming to knowledge about (in this case) current affairs that is good enough for democracy. But it's also possible that social filters will develop into insular communities with their own extreme representations of a limited range of stories that distort in subtle ways the representations of an already distorted mainstream media outlet such as FOX News.

Again, this sets the internet up not as an inherently good medium but as a problematic but exciting one that needs further study to ascertain and, hopefully, correct its weaknesses.

We're all institutions now

Finally, a few people at the end raised the question of the role of institutions. David was pretty scathing, suggesting that social filters would replace hierarchical institutions in an ideal digitised world. But institutions don't need the formal trappings of, for example, a national newspaper or an encyclopedia business. Influential blogs are themselves now institutions, because they are generally respected and deemed to be reliable. Where lots of people share data-relations on del.icio.us they are creating de-facto institutions of a sort.

These institutions are also emerging from the old institutions. For example, the BBC and Guardian both link to blogs, Wikipedia and other new institutions, lending them some of the reputation that the BBC and Guardian (deservedly or not) enjoy. There are also institutions that have been empowered or enabled by the internet whose purpose is to act as a counterweight to the established, mainstream institutions, for example Indymedia and Media Lens, though they still have editorial filters. How will these different kinds of institutions play out in a world where editorial filters are redundant or are merely an archaic tool with limited uses? Without institutions like Media Lens scrutinising blogs the closed circuits of self-reinforcing subjective viewpoints will lack any real challenges. There's also a related worry, that the established institutions will unduly influence the pile of leaves so that these supposedly anarchic subjective views just mimic and/or reinforce the basic intentions set by the established institutions.

There will be extremely complex interplays between these institutions that import all kinds of other political and sociological considerations, such as the control of information with copyright, and the moderation of trolls and racists. I don't want to follow this line of thought any further.

My worries can be summed up as follows: it's not that institutions are being replaced with huge piles of massively-connected leaves, but rather than institutions are becoming more diverse. Old hierarchical institutions that used experts to gather knowledge and distribute it in taxonomical trees are becoming less important. New institutions based upon transient social/subjective connections, our ability to adapt these and create our own simply by finding new ways of filtering knowledge, are becoming more important. The old institutions supposedly operate according to virtuous standards and intentions, even if they really don't. These new institutions have no guiding principles so their web of knowledge is be messy and fertile, which is in a sense desirable but it also makes it extremely hard to know when knowledge is good enough.

With FOX I at least know that they're basically corporate shills. With Flickr, as a bloke from the OII pointed out, I might search for Walmart and be presented with images of happy people uploaded by Walmart. Articles on Wikipedia could be written by people with a wealth of experience, or a kid who has read the preface to an introductory textbook. Knowing the standards and intentions becomes much harder. These are issues we need to address if we want to go from naive evangelism to genuine reflection and action.