The Future of News: DIY visualisations

Next week, I’m headed to Princeton University to talk about the Future of News at the Center for Information Technology Policy. David Robinson has asked me to talk about data visualisation, which, along with a few projects at the day job, has given me an opportunity to think about and explore some areas of interest. During the conference, I’ll be blogging here and on the Guardian blogs, most likely a mix at Organ Grinder, our media blog, and also at our Technology blog.

I think one of the opportunities that we’re missing in journalism right now is that we’re not doing enough with freely available tools to experiment with editorial concepts. The plethora of free web tools allows us to see what works for journalists and just as importantly our audiences and communities. If we lower the cost of experimentation in terms of time and money as near as zero as possible, we can try something new almost every day. As I say, experiment. Learn. Apply lessons. Repeat.

Here’s something we can do today: Visualise the North Carolina and Indiana primary addresses of Hillary Clinton and Barack Obama. It took me about 10 minutes to do both, and about 10 minutes to write this post. Innovation at the speed of news.

UPDATE: The embed code for this visualisation is pretty flakey or at least doesn’t play well with Strange Attractor’s CSS. It’s going to take a bit of work. Fail forward.

Technorati Tags: , ,

Is New Media a threat to press freedom?

The UK National Commission for UNESCO has organised several events today to mark World Press Freedom Day. One of them is: Is New Media killing journalism?

What the hell does this have to do with press freedom? Really? Questions like this publicly expose the bias some journalists still hold against the internet and their online colleagues. I’m mad as hell about this kind of intellectually lazy nonsense, and I’m not going to take it anymore.

And while I know and have worked with some of the people on the panel at the Frontline Club, it’s difficult to see that panel making a strong case to support new media, especially the professional troll Andrew Keen. The only reason that he is still given place at a podium is because he reinforces the professional biases of journalists looking for easy answers to job losses and finding a convenient straw man in ‘new media’ – the ‘murderer’ – and the ‘mob’ replacing the ‘old professional caste’. His ill-informed views may be comforting to some journalists, but they are palliative care for a professional ‘caste’, arrogant and bewildered in its decline. Furthermore, I’m a journalist, not some brahmin. And I don’t perceive a challenge to the arrogance of some journalists as a threat to journalism as a democratic institution.

And Keen pitting ‘expert journalists’ versus the uninformed masses is a dishonest representation of what the vast majority of journalists are: Generalists. As a journalist, I know a little bit about a lot of things, but my knowledge of most subjects is superficial given the constraints of time. We do our best to make sense of a fast-paced, fast changing world, but we must rely on the expert knowledge of others and have the skill to interpret that. More journalists should specialise because the complexity of the subjects that we follow is increasing. For example, how many business journalists, let alone general reporting journalists, have the chops to explain the unwinding of collateralised debt obligations to our readers and explain how they are in part responsible for the credit crunch?

But instead of calling journalists to a higher standard, Keen bemoans the lack of intellectual standards amongst the masses who “think we know better than expert journalists. Rather than enlightenment, we want the self-expression and the democratised interactivity of blogs and wikis.” Blaming the consumer for why your product isn’t selling is a common tactic of hapless executives leading their businesses into bankruptcy.

But let’s get back to the question: Is New Media killing journalism? The threat to the traditional business model of journalism could be interpreted as a threat challenge to press freedom. It takes money to mount journalistic investigations. But the threat to the business model of traditional journalism is not solely the fault of so-called new media, which isn’t that new unless you’ve had your head up your arse for more than a decade now. In the US, newspaper readership has been declining since the 1970s. It started long before the wide spread commercial availability of the internet. Classified ad sales for newspapers in the US have been declining, with only the occasional break, since the 1940s.

Let’s phrase this question as it should, more honestly, be phrased. Is new media a threat to press freedom? Tell Iranian journalists whose papers have been shut down and have turned to the internet to continue their work that new media is a threat to press freedom. Tell Berkeley journalism student, James Karl Buck, that ‘new media’ is killing journalism and human rights when he used Twitter to tell Egyptian activists and the world that he had been arrested. Tell people in Burma the technology is a threat to press freedom when the media would not have had the pictures of the recent crack down had it not been for mobile phone technology and the internet. As Geoff Long writes on a blog (sorry Andrew, those damned amateurs again):

Whether it’s via cell phone, blog, picture sharing sites or old-fashioned email, the consensus is that more news got out, and got out a lot quicker, than during the last big uprising and crackdown in 1988.

If you are a supporter of press freedom, then it’s clear from example after example, that technology and new media have expanded, not erroded press freedom, often in the world’s most repressive countries.

This debate is not about press freedom, it’s about business models and technology. To cloak it as a threat to the democratic institution of journalism is dishonest and a distraction. The real question is how do we develop new business models to support the time consuming and therefore, costly, job of quality news gathering.

Instead, journalists fearful for their jobs and more importantly, their perceived positions of authority, have set up the ‘new media’ as the scapegoat. It’s an intellectually bankrupt argument that won’t stop their businesses from going bankrupt as well. I’m quite bullish about the future not only of press freedom, but also of journalism. However, it’s long past time to move on from this bloggers versus journalists, ‘new media’ versus journalism debate. It’s now a matter of urgency.

UPDATE: The amateurs on Mr Keen’s post actually have raised some valid and well researched points. Worth a look through the comments.

UPDATE: From Nico Macdonald’s notes, it looks like most of the journalists at the Frontline Club didn’t rise to the occasion to attack the straw man of ‘new media’ presented by Andrew Keen. That’s a good note on which to start the bank holiday weekend.

Everyblock for everyone

Thanks to Martin Stabe for linking to the news from Steve Outing that Adrian Holovaty will release the code behind Everyblock.com as open source. Everyblock is the Knight-funded project that Adrian launched last year. It’s an extension of his groundbreaking ChicagoCrime.org. Normally, I would just link to this via Del.icio.us, but this development is important enough to warrant a post. It’s Christmas in June, when Adrian said he will release the code as open source. This is a huge gift from Adrian and by extension the Knight Foundation, if only the industry will take it. Steve Outing has highlighted one application for Everyblock: Mapping classifieds.

When asked about the utility of mapping classifieds, Holovaty says that absolutely that’s a great use for Everyblock’s system. You can easily imagine the Everyblock concept applied to garage sale ads, lost-and-found listings, or real estate listings, for example. Mapping could be useful for other classified categories, too, but of course there are privacy issues to consider. (A private classified advertiser selling his car or bicycle might not want his ad showing where he lives, but since there’s a benefit to exposing that information, he should be given the option.)

There are other valuable points in Steve’s post. He is not alone in encouraging news organisations to begin thinking about geo-tagging. It’s not even that difficult to add geo-data to stories semi-automatically by parsing the data from the dateline, although the more local the story, the richer the data one would want to add. An increasing number of mobile phones include GPS functionality, and it’s not difficult to add this to story data. What about a script that would automatically add that data if the journalist used the data connection via bluetooth to file? We all should be considering adding a geo-data field to our databases. It’s a basic step that enables a vast range of journalistic and commercial applications.

And Steve and Adrian go one step further saying that geo-tagging is only one bit of meta-data that news organisations should be consider adding. One of the subtle changes the Guardian (my day job) has made to its CMS is to think about the information architecture of the site and the stories. Structured data, as Adrian has shown in several projects, allows news organisations to begin to make sense of vast amounts of information in novel and very useful ways. We can use our own meta-data to help show trends, make connections and add context to our journalism.

As Steve says, there are alternatives to Everyblock, and the code may not be applicable to all projects. However, I’m increasingly concerned about a ‘not made here’ complex amongst journalism organisations. You can see some of that in the comments on Ryan Sholin’s recent post about building a local news site from scratch. Some of the comments are critical of open-source projects Drupal and WordPress. I’m not a knee-jerk open-source advocate, and some of the criticisms are no doubt valid. But I think many open-source projects deserve consideration along with the custom CMS route. Morris Digital Works has done some ground-breaking work with Drupal. The New York Times has even gone as far as being a good open-source citizen by releasing some of their code on their Open blog. All the code that’s fit to printf(), as they say.

But I’m convinced that news organisations have more to gain than to fear from open source and projects like Everyblock. Open source can be another form of networked journalism. Instead of relying solely on your own development team, you can suddenly plug into a worldwide network of passionate developers.

The importance of pigheadedness

I just read an essay by Clay Shirky, Gin, Television and Social Surplus, about how the industrial revolution has resulted, after a brief period of societal gin-soaking, in a surplus of time and productive capacity which has been mopped up by TV sitcoms. Now, however, this social surplus is being put to use in things like Wikipedia, World of Warcraft and blogging. People are taking their spare time and energy and they’re doing something with it.

It’s a great essay, and I strongly recommend that you pop over and read it, right now, all the way through, because it articulates something that many of us know is happening, but which a particularly large chunk of the media hasn’t cottoned on to yet. It’s not the content of Clay’s essay that I want to further discuss, but one little line that has much broader ramifications:

The normal case of social software is still failure; most of these experiments don’t pan out.

Every now and again I’ll be talking to a client or a journalist or some random person at a conference, and they’ll ask me if I think that social software is a fad. Invariably they’ll have anecdotal evidence of some company, somewhere, who tried to start up blogs or a wiki inside their business, and it failed. That, they say, is proof that social software has nothing to offer business, and that if we give it a few more years it will just go away. Quod erat demonstrandum.

The problem with this interpretation is that these failures – which are common, but largely unexamined and unpublished because no one likes to admit they failed – are part and parcel of the process of negotiating how we can use these new tools in business. They are inevitable and, were they discussed in public, I’d even call them necessary as they would allow us to learn what does and doesn’t work. Sadly, we don’t often get a glimpse inside failed projects so we end up making the same mistakes over and over until someone, somewhere sees enough bits of the jigsaw to start putting them together.

There is a lot of failure in the use of social software in business, on the web, in civic society, but we need to see this as a part of the cycle, a step along on the learning curve. We can’t afford to stop experimenting, just because something failed once, or because it didn’t work out for someone else. And we can’t afford to take part in the Great Race To Be Second, either, because if you’re waiting to see how other businesses succeed (or fail) before you leave the starting line, you’re not going to be second, you’re going to be last.

From a business point of view, the nice thing about social software is that a lot of is is free or ridiculously cheap, so the monetary cost of failure is low and made up mainly of the cost of people’s time. There is no need to judge a social software project based on the same criteria as, say, a massive software deployment from a megacorp vendor that cost millions and took three years, yet these are the terms by which many businesses are judging their blog, wiki, or social networking experiments. And because the tech is so cheap, businesses can afford to run many small experiments to find out what works before they deploy tools more widely; indeed, they cannot afford not to.

But we also need to recognise that the biggest speed bump in social software projects is invariably going to be the social, not the software. The technology is improving every month, mainly because it’s being developed by small, nimble vendors who use the software they create and want it to be the very best it can be. But the tech is only a fraction of the battle. The rest, like Soylent Green, is made of people.

And this is where the problem with failure comes in. Generally speaking, people don’t much like change. They don’t even like choice all that much, although they’ll tell you that they do. They certainly don’t like failure, or anything that looks even remotely like it. (Especially in the UK, although I think that the US is a bit more tolerant.) And they don’t like trying again when things do go a bit wobbly.

Failure, real or perceived, is inextricably entwined with status and, frequently, if a project looks like it’s about to go bottom up, instead of figuring out how to save it, people figure out how to distance themselves enough to save face. In a business culture where rewards and punishments are focused on the individual, the teamwork and collaboration required to make a social software project a success can become too much of a risk. But if you’ve got the right skills and personality, you can turn that around.

To be successful at social software implementations in business you need firstly to have a solid understanding of how people work and relate to computers, tools, and each other. You need to understand how to introduce tools in a way that is non-threatening and which emphasises utility and benefits. You need to understand the political climate within your business, and know how to route around anyone who’s threatening to be obstructive.

Secondly, you need to be really pigheaded. If one team doesn’t take to a wiki, try working with another. If one blog fails, try to figure out why and then start another. Iterate. Change things. Experiment. Try again. After all, it’s only failure if you give up.

Shovelware 2.0

When I started in online journalism, we struggled with aspirations that far out-stripped our resources. We were small teams passionate about creating a new medium but still dependent and subservient to legacy media – newspapers and radio and television stations. We yearned to do original journalism but often had to settle for ‘re-purposing’ other journalist’s content. We did as much as we could that treated the internet as its own medium, that developed multi-media story telling methods that simply weren’t possibly in print or in linear, broadcast radio and television. But most of it was simply shovelware: TV and radio scripts transcribed and thrown up online and print stories chucked on the internet. Or as Whatis.com says:

Shovelware is content taken from any source and put on the Web as fast as possible with little regard for appearance and usability.

It’s sad to see that so-called integration sometimes isn’t really about integration at all. It’s about a maintenance of organisational and internal political status quo. It’s about maintaining the dominance of print and broadcast and the subservient, derivative position of the internet. It continues to miss or ignore the opportunities the internet provides for journalists, which now isn’t defensible in terms of audience numbers, advertising revenue or future prospects for growth. And as my friend and former colleague at the BBC, Alf Hermida, says, it just doesn’t work. The BBC is advertising for a “web conversion producer”. I wonder if this is a position to produce web-literate producers from television and radio journalists. But seriously, Alf says:

This is a flawed concept and risks undermining the reputation for excellent online journalism that the BBC News website has built over the past 10 years. In any case, we tried in the early days of the site when I was a daily news editor, and it didn’t work.

It also implies that online is an after-thought, picking up the scraps off the broadcast table, rather than considered an equal.

Now, I’m not arguing for internet primacy over other media. This is not a zero-sum game. The legacy media still make most of the profits in real money terms, despite the double digit growth rates in online revenue for the past few years. Just as I say that the internet and on demand digital medium need to be understood on the basis of their own strengths, television, radio and print still have unique strengths. As Steve Yelvington says, the internet is one of the centers for a successful media business. He adds:

My rule of thumb is a simple one: Use the right tool for the right job. The Internet’s strength is collaborative interaction; print’s strengths are linearity, focus and serendipitous discovery.

But as news organisations struggle, some for survival, they will fail if, due to organisational in-fighting, they repeat the same mistakes of the late 1990s. Those few of us in online journalism who survived the dot.com crash have seen this before. Unfortunately, while we have a decade or more of experience, we digital natives still don’t have the political capital when we go head-to-head with the powers that be in our own organisations. If media bosses want to engage in Shovelware 2.0, they can use that shovel to bury their own businesses.

Twitter interviews on ReadWriteWeb

I already added the post Real People Don’t Have Time for Social Media on ReadWriteWeb to del.icio.us because it talks about participation inequalities and relative time spent by people on various social media sites and services. The post has kicked off an interesting discussion in the comments as well as at the office. But as a journalist, one thing caught my eye. Sarah Perez ‘interviewed’ people on Twitter about how they spend their time using social media. Now, obviously, this isn’t a broader sample of people who simply don’t participate, but it does give a snapshot of social media usage and a range of participation.

I’ve used it personally if I have a tech question I’m stymied by or want to get a range of views on a movie or a restaurant. Suw jokingly refers to it as a query for the ‘lazyweb’.

However, there is definitely something useful here journalistically. Sarah’s use of Twitter also shows how using the service not only as a way to promote your content but also to create community could be used to add to your journalism. No, it’s not a random sample. But since when are ‘man on the street’ interviews?

Twitterquest

Technorati Tags: , ,

Real-time innovation in news organisations

Ryan Sholin has started a great conversation about how to create cultural change at newspapers.

The important part of the job isn’t speaking to the first 20 people on the conference call for an hour, it’s maintaining contact with the one person on the call who has the potential to Get It: Moving from the Paper business to the News business isn’t as simple as picking up a different skillset; it’s about changing the mindset of journalists.

It reminded me of a question I’m often asked about cultural change: How do you turn journalists into bloggers? The simple answer is that I don’t. I find journalists who happen to be bloggers or who show an interest in blogging, give them all the technical and editorial support that I can, and then I try to share that knowledge and success around the organisation.

How do I spot a good blogger? I ask whether the journalist is already aware of other bloggers writing in their beat. I try to determine whether they are willing to engage with other bloggers and people who comment on their posts. In short, are they ready to join the conversation?

Sharing the success stories helps spread the culture. As David Anderson of Fairfax Digital in Australia told me recently, you need success stories to tell your managers, and I would say that you also need success stories to win over journalists, who are professional sceptics. You have to spread culture up and down the organisation.

I’ve been fortunate. At the BBC, I had support at all levels for digital experimentation and, when I came to London, it was great to see evangelism from people like Richard Sambrook, the head of the Global News Division. At the Guardian, we’ve got digital evangelists all the way up to The Editor’s office. However, both the BBC and the Guardian still grapple with cultural change. And I couldn’t agree with Ryan more when he says you can’t mandate change from the top down.

Anyone who has worked with me will attest that sometimes I get frustrated at the pace of change in the industry. I really thought digital journalism would be further along by now than we are. The dot.com crash wiped out a lot of talent in the industry and set us back years. And I’ve crossed swords with a fair number of nay sayers. I’m not sure there is much we can do for the close-minded, and the need for change is urgent enough that arguing with the Andrew Keens in the industry just isn’t worth your or my time.

Industry scale change will only come with time. The industry is struggling because the depth of digital culture is still too thin and still so new. Don’t sweat that. Don’t even try changing your organisation wholesale. We might have the experience, but as Steve Yelvington says, we still don’t have the political capital. Many of you will run into middle management who ‘own’ the bureaucracy and have an investment in the status quo. They’ve spent several years supporting the Andrew Keens because it protected their position and power. Now, they have a new strategy: They are fighting over who owns change rather than focusing on actually creating change. While they’re fighting, us digital journalists need to get on with it:

  • Start small with an event or story-based project.
  • Bring the cost of the project down as close to zero as possible from a technical standpoint. Use open-source or free net-based services.
  • Try one new multimedia story-telling feature or engagement feature with each project.
  • Debrief. Learn. Repeat.

This is based in part on my interpretation of the Newspaper Next project. As Steve Yelvington says:

We need to think of making things that are good enough and not overshooting. We’re taking too long to create ‘perfect ‘ systems that don’t meet needs. We over-invest, over-plan and then we stick with the bad business plan until it all collapses. Come up with a good idea and field test. Fail forward and fail cheaply. Failure is not a bad thing if we learn from our mistakes and correct. Be patient to scale. Impatient for profits.

This is real-time innovation. Try journalistic projects with existing tools and learn journalistically and technically from that. That takes zero development and relatively little time. It’s about editorial creativity, not about development cycles or budgets. If you find something that works, then you know where to focus product development. What can you do with Twitter, blogging software, YouTube, Seesmic or FriendFeed to create a journalistic project and help build your audience?

Are we the signal or the noise?

I recorded this video for a project that the Guardian is doing with Current TV. I recorded it after reading a post by a friend and one of my heroes

, Steve Yelvington, in the wake of the recent conflagration over Barack Obama and his former pastor Jeremiah Wright. Steve asked whether we were listening. We could be interpreted as journalists, politicians, pundits as well as the public.

Today I see journalism falling into two traps. One is the passive abandonment of responsibility that sometimes comes along with the “objective” mode, and the other is the crass exploitation of divisive opportunities that you see from infotainers like Rush Limbaugh, Glenn Beck and Lou Dobbs.

And that brings us back to my point. Is anyone listening? And is the press helping us all listen? Are we working to further understanding?

Or are journalists just parroting words and perpetuating the racial divide that has scarred this country throughout its history?

It’s one of the things that many journalists don’t do enough of when they blog: Listen. That’s one of the important skills for a blogging journalist. Blogging is not just publishing my thoughts. I can do that in any old media. Blogging is about the conversation.

Why I’ve chosen to do the kind of journalism I do is that I see great potential in being able to foster civic discussion and participation using the internet. It hearkens to the ideals of journalism that I learned as a j-school student. I really don’t understand why more journalists don’t see it.

As I said in the video and the discussion that followed on Current, I want to find ways to expand who is taking part in these discussions and actually explore important issues. As a journalist, I can add some reporting to provide a for some of the issues, which isn’t to say that the participants can’t add their own reporting. There is such scope to explore the issues of the day and be in a constant, rolling, evolving conversation. It’s exciting territory to explore.

But too often, either through neglect or active provocation, the media are turning these online spaces into brawls. It’s not surprising. It mirrors talk radio, cable news shouting matches and some bizarre version of Jerry Springer for intellectuals. The media is just turning the internet into what it knows. Bring on the noise.

But isn’t good journalism supposed to amplify the signal, find it in the noise? Aren’t journalists supposed to help find the important data points, turning points to help people and themselves make sense of the world? It’s an abdication of our professional responsibility if we stop trying to find the signal and become the noise.

That’s not going to save our profession. It’s not going to help use cut through the clutter in this very busy media landscape. But it’s easier to try to shout above the crowd than to find the wisdom in it. It’s easier to be provocative than to be thought provoking. I don’t have much time for it, and increasingly, neither do our former audiences.

Is Google hijacking newspaper website traffic with new search?

From the Twittersphere, Robert Andrews pointed me in the direction of this post by Martin Belam, Google hijacks traffic from newspaper site search. Martin as always makes some good arguments on why this might be a threat to newspapers.

Whilst Google has dressed this up as being for the benefit of users, it does have some significant implications for the newspapers involved, and has the potential to dent their revenue. … By allowing people to do site searches whilst still on google.co.uk, Google is potentially reducing the number of page, and therefore advert, impressions that these newspapers may be getting. In fact, not only that, but Google is effectively hijacking the advertising that can be displayed by newspapers against search queries on their own site.

I agree that this might negatively impact newspapers’ revenue both in terms of display adverts and also when the newspapers themselves (including the folks that pay my wage, the Guardian) insert text adverts alongside their search results.

Where I might disagree is Martin’s argument that it negatively impacts user experience. He says that Google’s position is that they can provide search better than the news sites. Well, the sad truth is that whether it’s information architecture or search, most news organisations have been very slow to improve these parts of their services. Some news and media organisations have forced their users to use Google because their own search is unusable. They still are making the unmissable, unfindable.

I also see a number of newspapers forcing their users to follow a print paradigm that their drive-by readers may not be familar with. I guess it’s useful for newspapers to allow people to filter their knowledge based on authors, section and branding. It’s useful for those people who are familiar with those things, but increasingly, I believe that many people coming to a site from some random link on the internet aren’t familiar with those things and wouldn’t find that type of filtering useful and may find site architecture based on those considerations baffling. It’s sad that in 2008, we’re still building news sites for us and not our audiences. News editors can’t see the forest from the dead trees and build sites based on their print reading behaviours and their intimate knowledge of their desk structure instead of information needs of their audiences. When you look at online audiences for national or international titles, the great majority are not going to have any familiarity with your print product. Using print product paradigms as a basis for site architecture is a mistake.

Hey maybe I’m an edge case. Or maybe not. (Go to about 2:35 in the discussion of The State of the News Media 2008 by On the Media.) I only read physical newspapers when I fly. I rarely buy newspapers, and my news consumption is a lot more promiscuous. I don’t believe that any news source provides me with the complete picture so I fill in the blanks on my own.

Technorati Tags: ,