Author Archives: Suw Charman-Anderson

05.04.11
by

LinkedIn as a source of traffic

Earlier this year I did some work for OldWeather.org, a citizen science project that is transcribing weather and other data from old ships logs. As part of their website progress assessment, I hand-analysed their web traffic referrers to see where people were coming from and whether we were reaching our core communities. One of the things I found was that whilst Facebook sent over two orders of magnitude more visitors than LinkedIn, LinkedIn was responsible for much higher quality visitors. Visitors from LinkedIn visited an average of 17 pages per visit, staying for 34 minutes with a bounce rate of 33%, compared to Facebook’s 1.8 pages per visit, 1:41 minutes on site, and 79% bounce.

The quality difference is stark and indicates that for OldWeather.org, perhaps a bit more promotion in LinkedIn might be in order. But is LinkedIn capable of the same volume of visitors that Facebook can provide? Facebook still provides a far higher overall share of time on site compared to LinkedIn, although on some sites (this one included) a single page view isn’t all that useful in terms of the site being able to fulfil its remit. Lots of single-page-view visitors aren’t as valuable as fewer multi-page-view visitors.

According to Business Insider, recent changes to LinkedIn has upped their ante quite significantly.

Out of nowhere, Business Insider started seeing real referral traffic from LinkedIn last month. […]

LinkedIn product manager Liz Walker tells us the traffic is coming from a bunch of sources – mostly new products like LinkedIn.com/Today, newsletters, and LinkedIn News.

It seems to me that, if these visitor quality stats and this new trend in volume hold true, then LinkedIn is successfully shifting from being a site often marginalised in social media outreach strategies to one that should be central. After all, with traffic it’s not just the volume you should be interested in but the quality of visitors as well.

04.27.11
by

Zephoria Inc.: About to find out how social media really works

Dramatis Personae

Zephoria Inc.: An “internet marketing consulting company based in New York focused on helping companies maximize their online exposure through search engine optimization, web analytics, and marketing focused web development” who don’t have a clue about how social media actually works.

Zephoria: danah boyd’s online nickname which she has been using since 1998.

Tumblr: A sort-of blogging platform with staff who have really put their foot in it this time.

The Chorus: Bloggers, Twitterers and other random persons who will show Zephoria Inc and Tumblr just how social media really works.

The Story

danah blogged this evening about how she had been using Tumblr as a place to collect random bits of stuff, with the URL zephoria.tumblr.com. Zephoria has been her ID online for ages:

I’ve been using the handle “zephoria” online since around 1998 when I started signing messages with that handle while still at Brown. It’s actually a funny blurring of two things: zephyr and euphoria. Zephyr was the name of the instant messaging service at Brown and the name of the dog that I lived with in 1997, two things that I loved dearly. And talking about euphoria was a personal joke between me and a friend.

But suddenly, her Tumblr blog has been moved to zephoria1.tumblr.com, seemingly without any discussion or permission given on her part. Her original zephoria.tumblr.com is now used by Zephoria Inc, whose main home is Zephoria.com (compare and contrast to danah’s Zephoria.org).

Zephoria Inc say that they named themselves after “Greek mythology’s west wind”, though they seem to just have a problem spelling ‘Zephyr’:

zephyr | ?zef?r |

noun
1 poetic/literary a soft gentle breeze.
2 historical a fine cotton gingham.
• a very light article of clothing.

ORIGIN late Old English zefferus, denoting a personification of the west wind, via Latin from Greek zephuros ‘(god of) the west wind.’ Sense 1 dates from the late 17th cent.

So they can’t even get their name right.

It seems that Zephoria Inc asked Tumblr to release danah’s subdomain, and Tumblr did just that. danah says she didn’t get any sort of notice from Tumblr, although David Karp, who appears to be in some way related to Tumblr although his Twitter profile doesn’t state as much, said on Twitter:

Hi Danah. We never reassign domains w/o notifying users first. Our support team reached out two weeks ago and didn’t hear back.

Per your last msg, your login/account are NOT dead. It looks likes you registered with a secondary email address?

Our team sent over the details. Please let me know if there’s absolutely anything I can do to help. I’m sorry for the trouble. 🙁

So it’s unclear exactly what went down… however, given that Tumblr have pulled this kind of stunt before it isn’t entirely clear that Tumblr actually did go to what one might call reasonable lengths to get in touch with danah.

Furthermore, Zephyr Inc… sorry, Zephoria Inc, have not been playing nice regarding other usernames/URLs according to danah:

A few years ago, I learned that there is a technology consulting company called Zephoria.com. And apparently, they’ve become a social media consulting company. In recent years, I’ve found that they work hard to block me from using the handle of zephoria on various social media sites. Even before the midnite land grab on Facebook, they squatted the name zephoria, probably through some payment to the company.

The irony is that Zephoria Inc not only does SEO (hm, how does “Zephoria Inc are a bunch of bullying fucktards” work for you guys?) and web dev, they also say that they do social media. Well, here’s a tip for you: If you stomp on one of social media’s most intelligent, dedicated, beloved people, you can expect to get stomped on back. Not just by danah, but by all her friends and everyone who holds her in high esteem.

A search for ‘zephoria‘ on Twitter right now shows just how much people resent bullying fucktards who think they have a right to a username just because they want it. (Hint: you don’t). More people will have heard of Zephoria Inc now than ever would have without this kerfuffle, and they all think that it’s a company that they would never do business with because they don’t play fair. It’s like a variant of the Streisand Effect, where a company does something utterly stupid which serves only to draw attention to its own stupidity.

Hopefully, Tumblr will come to their senses and give danah her URL back. And hopefully Zephoria Inc will either apologise and mend their ways, or go out of business. I’m hoping for the latter, because I just don’t need yet more fuckwits bringing social media into disrepute.

But there are two main lessons here:

1. If you are a service providing users with username-based URLs, be very, very careful how you handle requests to free up usernames even if they are dormant. You need dialogue and to come to a fair agreement which isn’t simply based on “Biggest bully wins”.

2. If you are a business and someone else has your username, suck it up (unless they are username squatting). If you are starting a business, do make sure that your business name isn’t already in use by someone else, or be willing to use a variant. if you’re going to rely on your ability to bully others into submission, you may find out that you have bitten off more than you can chew.

Zephoria Inc. have made a major error in bullying danah, as the very social media they purport to understand will now ensure that their Google search page turns up lots of content that discusses just what a bunch of dishonest, bullying charlatans they really are. Well done, chaps. Couldn’t have done better SEO if you’d tried.

UPDATE: Some more info via Betabeat.

UPDATE 2: danah has updated her blog with what has now happened. This is the key excerpt from that update:

10:39PM: I just got off the phone with John Maloney [President of Tumblr]. We had a lovely conversation which began with him apologizing for what he described as a human error in customer service and saying that he looked into the issue and has reinstated my account. He explicitly stated that they are working hard to have strong customer service processes where things like this don’t happen and that he feels terrible that it did happen. He said that Tumblr has only had four issues like this in the past and that they are committed to making certain that legitimate active users do not face these issues. He did say that they work hard to not allow squatting (and he argued that the Pitchfork case was one of squatting, not active use by the individual).

04.27.11
by

Collaborative reporting

I read Highs and Lows of “Post Mortem” Collaboration Between Frontline, ProPublica, NPR, by Carrie Lozano over on Mediashift with interest, not least because collaboration has been a specialty of mine for many years now. Ever since I first started working with social media over seven years ago I have focused on collaboration, so a project that marries collaboration and journalism is of course going to pique my interest.

This piece, first in a series by Lozano, sets the scene, but doesn’t go into any detail about how journalists from Frontline, ProPublica and NPR actually got down to the day-to-day nitty gritty. That’s what I’m really interested in, because the collaboration tools available today make working together really easy, if – and only if – people are willing to learn and adjust the way that they communicate.

There are basically two types of collaboration.

Asynchronous collaboration:

  • The actions of the collaborators are spread out over time 
  • Materials are gathered or created and made available to the other team members who access them whenever they wish to
  • Collaborators can be spread out over different time zones
  • Conversations can occur slowly as there can be a delay before each participant is able to reply
  • Tools include: wikis, blogs, microconversation tools (i.e. Twitter-like tools), Google Documents, file sharing services like Dropbox

Synchronous (or nearly synchronous) collaboration:

  • The actions of the collaborators are taken in concert
  • Materials are created together, in real time, either by simultaneous editing or by taking turns in a timely fashion
  • Collaborators are usually in overlapping timezones, even if they are not physically in the same location 
  • Conversations happen smoothly as collaborators can response almost instantly
  • Tools include: instant messenger, chat, Google Documents, microconversation tools

 

The trick is in knowing what kind of collaboration you need and how to swap seamlessly between modes as required. A lot of people who don’t understand collaboration wind up using email as their primary tool, despite email being very badly suited for the work. If you’re going to collaborate with other reporters, you must set up your collaboration systems and ensure that everyone is familiar about what to use, when and how, before your project truly kicks off.

Lozano says:

there’s a learning curve to working this way as basic issues pose challenges, from how to communicate to how to share information to how to simultaneously report for different platforms. Trying to understand how it all works can be confusing, even with a front-row view, so imagine what it’s like when you’re truly in it.

Indeed. You can’t change your workflow in the middle of a big project: trying to do so will cause problems because people are focused on their work, not on their methodology. So you have to sort out ahead of time how people are going to share their research, their notes, their files, their ideas, their rough drafters and their final copy. And you need to make sure people actually use those tools, rather than clinging to familiar ways of working.

“Chaos is the ultimate form of investigative reporting,” Stephen Engelberg, managing editor of ProPublica, told me in the midst of “Post Mortem.” “You have to acknowledge that inefficiency is part of the process.” He was explaining the vicissitudes of an investigative piece, where a story can take unexpected turns all the way down to the wire. Joint reporting is not so different.

Collaboration can cause issues if people are unclear on the tools and how to use them, but social collaboration tools can and do create new efficiencies, despite projects going in unexpected directions. If everyone, for example, shares their research and keeps an eye on what others are finding, then that can save a lot of duplicated effort. Understanding what everyone else in your team is doing and how to reduce duplication is a key part of collaboration. It’s not just about doing your own thing and letting others follow on behind, but about considering how your work fits in with other people’s, and how you can all save each other a bit of time.

You also have to teach people how to work inside the collaboration tools, so that the act of writing/researching/planning becomes a de facto act of collaboration, rather than making collaboration an add-on that people do after they’ve finished their final draft. I have worked on a number of ‘open’ projects, where clients can see my notes, research and drafts in all their states of disrepair, and it’s a very different way of working. Emotionally, it’s quite hard because if you’re used to handing over only your final draft, you can feel quite vulnerable when someone can see your messy first draft, but it’s well worth it as early feedback is easier to incorporate.

At the most basic level, think about the logistics. With “Post Mortem” it was often a struggle to get everyone in the same room. At times, there would be 15 busy, bicoastal people on a conference call. It was always a juggle keeping everyone in the loop, but with that many people, I’m certain there were some who wouldn’t have minded being excluded.

Meetings and conference calls are essential to any collaborative project, but they should be reserved for discussion and decision making, and should not be used for updates. Keeping people in the loop should be done online, via wikis, blogs, and other such tools, all of which can be kept confidential. (Remember: your work blog does not have to be public!)

I have had many clients who have found that using a wiki to give people updates, set the agenda, and disseminate notes from calls and meetings cut the length of those calls and meetings by at least half. And let’s face it, there’s nothing more tedious than sitting on a call waiting for your turn to bore everyone to death with what you’ve achieved that week! But you do have to think about these processes ahead of time and ensure everyone is in the habit of adding their update and reading everyone else’s before the call. Once people realise that doing so cuts down the tedium, however, most are happy with the new process.

I shall be interested to learn more about this journalist collaboration, the tools they used and how they managed the processes of sharing information. But there’s one thing that all journalistic organisations can and must learn right now:

People outside of your organisation and industry already know a lot about collaboration: asking their advice will save you some pain.

I often worry that the industry is so convinced it is special that it needlessly eschews external expertise, instead preferring to reinvent the wheel over and over and over again. You can see this in the continual re-examination of the use of blogs in journalism, despite the fact that we’ve been having that discussion for the best part of the last decade and have pretty much got it figured out. To paraphrase William Gibson, the future is here, it’s just being ignored by half the industry.

Indeed, if there’s one key lesson for journalists to learn about collaboration, it’s to ask the experts who’ve been working in this field for years. Having worked on collaboration projects in many diverse sectors, from investment banking to pharmaceuticals to PR to built environment to science to media, I can promise you that journalists are no different to anyone else. Humans are reassuringly similar no matter what they do, so the lessons learnt about collaboration in pharma are just as applicable to journalism as anything else.

I hope that the exploration of collaborative journalism between Frontline, ProPublica and NPR will throw up some interesting and previously unknown lessons. But I fear that many of their problems will have been be ones they could have avoided with the right preparation. However, we shall have to wait and see!

04.20.11
by

Standards in journalism (and comments)

Via Kevin, I came across this piece by James Fallows of The Atlantic: Learning to Love the (Shallow, Divisive, Unreliable) New Media. As soon as I saw that headline, my feathers ruffled. So you think new media is worse than traditional media eh? Well, how come debates that pit blogs against journalism never talk of the scum-sucking pond-dwelling tabs, eh? How comes it’s always the worst of blogging vs the best of journalism, eh? Eh?

Oh, Mr Fallows, I apologise. I did you wrong. Fallows hasn’t, as I had assumed, written some lazy tripe based on a false dichotomy. Far from it. He’s taken an intelligent and insightful look at the claim made by “everyone from President Obama to Ted Koppel” that there has been a “decline in journalistic substance, seriousness, and sense of proportion.”

It’s a really tempting position to take, that standards in journalism have slipped. It’s a position I have some sympathy with, because it jibes with the frustrations I feel on a daily basis when I see inaccurate reporting, sensational headlines and so much PR that I’m surprised that someone at CERN isn’t studying the destabilising effects of political spin on sub-atomic particles.

But Fallows argues eloquently that have we heard this argument before, at pretty much every major inflection point in journalism.

As technological, commercial, and cultural changes have repeatedly transformed journalism, they have always caused problems that didn’t exist before, as well as creating opportunities that often took years to be fully recognized. When I was coming into journalism, straight from graduate school, in the 1970s, one of the central complaints from media veterans was precisely that the “college boys” were taking over the business. In the generation before mine, reporters had thought of themselves as kindred to policemen and factory workers; the college grads in the business stood out, from Walter Lippmann (Harvard 1910) on down. A large-scale class shift was under way by the time of Watergate, nicely illustrated by the team of Bob Woodward (Yale ’65) and Carl Bernstein (no college degree). The change was bad, in shifting journalists’ social sights upward, so they identified more with the doctors and executives who were their college classmates, and less with the non-college, blue-collar Americans whose prospects were diminishing through those years. And it was good, in equipping newspapers and TV channels with writers and analysts who had studied science or economics, knew the history of Russia or the Middle East, had learned a language they could use in the field.

And we can’t just round on digital and blame it for the shift in the way that people consume news and the way that the internet allows them to indulge their interests without ever eating their greens. But we can, he demonstrates, learn something from some of the digital journalism outfits, like Gawker, that so many traditionalists look down upon.

Now, I confess, I can’t even begin to paraphrase the rest of Fallows’ article. Like many Atlantic pieces, it’s long and to try to summarise everything could only do it a disservice. I can only say that this is a great piece, well worth reading and rereading.

Derail
One bit, about Gawker’s Nick Denton, stood out as requiring further consideration, but not just in this context of standards:

In the first New York profile, in 2007, Denton had said that an active “commenter” community was an important way to build an audience for a site. Now, he told me, he has concluded that courting commenters is a dead end. A site has to keep attracting new users—the omnipresent screens were recording the “new uniques” each story brought to the Gawker world—and an in-group of commenters might scare new visitors off. “People say it’s all about ‘engagement’ and ‘interaction,’ but that’s wrong,” he said. “New visitors are a better indicator and predictor of future growth.” A little more than one-third of Gawker’s traffic is new visitors; writers get bonuses based on how many new viewers they attract.

This is fascinating. The received wisdom, and certainly my position for many years, has been that comments can be valuable in terms of encouraging audience loyalty and return users. Over the last several years I’ve been refining my ideas of when comments should and shouldn’t be used on a news site. We have so many examples of how comments can turn toxic, putting off both readers and advertisers, that one would be a fool to think that commenting is some sort of loyalty silver bullet. It is clear that commenting requires serious thought: when is it enabled, how it is used, how/when journalists should engage (hint: most of the time), and how can users be encouraged to behave in a positive and civil manner.

Denton is right. Comments for comments sake is a dead end. And most news outlets have no comment strategy, have given no thought to when and why they might enable comments, and so rarely use comments in a productive way that readers simply aren’t used to the idea that such a collaboration might even be possible. The news industry is also so tribal that they are almost incapable of taking advice or help from anyone that they see as an outsider. That’s a pity, because outside of the news industry is where most of the expertise sits, even now.

We need to be much more sophisticated about how we use comments, much more thoughtful and much more experimental, because we already know that free-for-all comments too easily go awry. Like a pristine blanket of newly fallen snow, it’s only a matter of time before someone comes along to write their name in wee and ruin the view for everyone else.

03.24.11
by

Sacrificing web history on the altar of instant

As I said in my last post about Twitter’s lack of a business model, I’ve been doing some research lately for a think tank. My research has basically consisted of three things:

  • Looking back on the media coverage of an event that happened in early 2010
  • Looking back at the way bloggers reacted to said event
  • And having a quick look at Twitter for reactions there too

Pretty simple stuff, I think you’ll agree. My assumption was that I would be able to tap into Google News; Google Blog, Icerocket and maybe Technorati; and Twitter’s archives. Then I’d be able to scrape the data using something like Outwit Hub, chuck it in Excel and Bob’s your uncle.

Oh, how sadly misguided, how spectacularly wrong.

Now before you talk about how ephemeral the web is and how no one should rely on it for anything, that’s only partly true. A lot of stuff on the web stays on the web, and given how much of our digital selves we are putting on the web, we do need to think about archiving and how we preserve our stuff for the future. But this post is not about archiving, but about accessing what’s already out there.

The first thing I did when I started my research was try to go back in time in Google News to early 2010 and search for news articles about the particular story – the eruption of Eyjafjallajökull – I was interested in.

But Google’s News search results are fuzzy. I wanted to search for the news on particular days, e.g. all the news about the Eyjafjallajökull on 16 April 2010. Do that search, and you’ll be presented with lots of results, many of them not from 16 April 2010 at all, but 17 April or even 18 or 15 April.

I  wanted to refine the search by location, so restricted it to Pages from the UK. Fascinatingly, this included Der Spiegel, Business Daily Africa, Manila Bulletin, FOX News, Le Post and a whole bunch of media sources that, when I last looked, weren’t based in the UK.

So I now have search results which are not limited to either the date or the place that I want. But even worse, results are clustered by story, which might seem like a good idea, but which in reality is lacking. Firstly, these clusters of similar stories are often not clusters of similar stories at all, but clusters of stories that appear to have some keywords in common but which are often actually about slightly different things. I can see the sense in attempting to cluster stories together for the sake of cutting down on duplication for the reader but equally, sometimes I just want a damn list.

Whilst doing my research, I also found that Google News is not, as I had thought, a subset of Google Web results. If you do the same searches on Google Web you get a slightly different set of data, obviously including non-news sites, but actually also including some news sites that aren’t in Google News, and not including many that are.

So far, so annoying, but Google isn’t the only search engine in the world… Except, Google pwned search years ago and innovation in search appears to be almost entirely absent. Bing does news, but a search on Eyjafjallajökull tosses up just three pages of results, and you can’t sort by date. Yahoo News finds nothing. A friend suggested that my local library might have a searchable news archive, but the one I looked at was unworkable for what I wanted.

I’m sure there are paid archives of digital news, but that wasn’t within my budget and, to be honest, given how much news is out there in the wild, there should be a good way to search it. I even tried the Google News API, but that has exactly the same unwanted behaviours as the website.

But hey, things will be better in the blog search, right?

Years ago, in the golden era of blogging, Technorati worked. Their site used to be really great, and I loved it so much I did some work with them. These days, I’m not quite sure what it’s for. It’s certainly not for search, given it finds nothing for Eyjafjallajökull. Icerocket is a better search engine, and you can refine by date, but it finds nothing on our target date, which is surprising as it’s a day or so after Eyjaf popped her top and the flight ban was well underway and, well, you’d think someone on the internet might have had something to say about it.

So, we’re back to Google Blogs. It lets me restrict by date! And specify UK-only! And it coughs up one page of results. Really? We have 4.57m bloggers in the UK and only 35 of them wrote something? I’ve always had my suspicions that the Google Blog index was poorly formed, but Google Blogs is the only choice I have, so I just have to put up with it. At least the results are in a neat list and all on the target date, even if some of them are clearly not from the UK, or even actually blogs, for that matter.

Now then, Twitter. We all know that Twitter’s archives have been on the endangered list for some time, but although they aren’t deleting old Tweets, accessing them is very difficult. Despite providing you with dates going back to 2008 in their advanced search page, you get an error if you try to search for April 2010: “since date or since_id is too old”.

SocialMention is a new search site that I’ve started to find really useful. They search across the majority of the social web and allow you to split that down by type. So I can search for ‘Eyjafjallajökull’ in ‘microblogs’ and get realtime results, but I can’t go back in time further than ‘last month’.

So, we’re back to Google again, this time Google Realtime. It only goes back to early 2010, so lucky for me that my target date is within that period. But the only way I can access that date is by a really clunky timeline interface – I can’t specify a date as I can in Google’s other searches.

Furthermore, there’s no pagination. I can’t hit ‘next page’ at the bottom and fish through a bunch of search pages to find something interesting – my navigation through the results is entirely dependent on the timeline interface. Such an interface will and does entirely outwit Outwit, which can normally follow ‘next’ links to scrap date from an entire search. I doubt it knows how to deal with the stupid timeline interface.

After all this searching and frustration, I’m left with this question:

What has happened to our web history?
The web is mutable, yes, but there’s an awful lot of fire-and-forget content that, generally speaking, hangs around for years. Individual blogs may come and go, but overall there’s a huge pool of blog content out there. Same for news. Twitter is a slightly weird case because it’s a single service with a huge archive of historically interesting data which it isn’t letting just anyone get at. Not even scholars. Twitter may have given its archive to the Library of Congress, but even that’s going to be limited access if their blog post is anything to go by:

In addition to looking at preservation issues, the Library will be working with academic research communities to explore issues related to researcher access.  The Twitter collection will serve as a helpful case study as we develop policies for research use of our digital archives. Tools and processes for researcher access will be developed from interaction with researchers as well as from the Library’s ongoing experience with serving collections and protecting privacy and rights.

The Library is not Twitter and will not try to reproduce its functionality.  We are interested in offering collections of tweets that are complementary to some of the Library’s digital collections: for example, the National Elections Web Archive or the Supreme Court Nominations Web Archive. We will make an announcement when the collection is available for research use.

I’m not an academic researcher, so whether I’d even get access to the archive for research is up in the air. (I can’t find any updates as to the availability of Twitter’s archive via the Library of Congress, so if anyone has info, please leave a comment.)

I think we have two problems here, one already briefly mentioned above.

1. Google has pwned search
For years, Google has been the dominant search engine, and in some ways they’ve paid a price for this as publishers of all stripes have climbed on the Google haterz bandwagon. My suspicions are that Google’s fuzzy search results are a sop to the news industry, because Google should be capable of producing a rich and valuable search tool that allows the user to see whatever data they want to see, in whatever layout they want. Maybe, after all the stupid shit the news publishers have thrown their way, Google thinks that building in failure to their news search product will insulate them from criticism from the industry.

But I don’t think that this absolves Google of responsibility for the lack of finesse in historical search. After all, which bloggers are gathering together to demand Google not index them? And Twitter users who don’t want to be indexed by Google can go private with their account.

But Google dominance does seem to have caused other search engines to wither on the vine. It’s almost like no one is bothering to innovate in search anymore. Bloglines used to be a pretty good blog search engine, but it has gone the way of the dodo. Technorati is now useless as a search engine. Bing is a starting point, but needs an awful lot of work if it’s going to compete. News search is completely underserved, and Twitter… really, Twitter archival search is non-existent.

Are Google really so far ahead that they can’t be touched? Are they really so great that no one is going to bother challenging them? The answer to the first question is clearly no, they aren’t that brilliant that their work can’t be improved upon, not just in terms of the search algorithm which has come in for a lot of criticism lately, but also in terms of their interface and the granularity of advanced searches. And I’d be deeply disturbed if people thought that the answer to the second question was yes. Google are the incumbent but that makes them vulnerable to a smaller, more nimble, more innovative competitors.

2. Historic search has been sidelined to serve instant
What’s going on right now? That’s the question that most search engines seem to be asking these days. Most have limited or zero capacity to look back on our web history, focusing instead of instant search. The immediacy of tools like Twitter and Facebook is alluring, especially for brands and companies who want to know what’s being said about them so that they can respond in a timely fashion.

But focusing on now and abandoning deeper, more nuanced historic searches is a disturbing trend. Searching the web’s past for research purposes might be a minority sport, but can we as a society really afford to disenfranchise our own past? Can we afford to alienate the researchers and ethnographers and anthropologists who want to learn about how our digital world has changed? About our reactions to events, as they happened rather than remembered years later? There is value in archives, but not if they are locked up, and the key thrown away by the search engines.

We cannot afford to sacrifice our history on the altar of instant. We can’t just say goodbye to the idea of being able to find out about our past, because it’s ok, we can see just how pretty the present looks. The obsession with instant risks not just our past, it also risks our future.

03.18.11
by

Twitter: Building a business-critical tool, then breaking it

I remember four years ago, when Twitter was still a blossoming new service, the outages that they used to suffer. Within just a few weeks of joining, I realised what a great tool it was and how important it was to me. Like many others who endured ongoing disruptions to Twitter’s service, I publicly stated I was willing to pay. Indeed, people were begging Twitter to let us give them money, to have some sort of way of paying for a service we had so quickly learnt to love. Twitter, inexplicably, pooh-poohed the idea, much to our frustration.

Over the last four years I have watched Twitter grow and fail to find any sort of business model that you could hang a hat on. Unlike Flickr, or Viddler, or Dipity, or WordPress, or LinkedIn, or countless other services, I can’t choose to give Twitter my money through subscribing to a premium account. For four years, Twitter have failed to earn any money from me, despite my willingness to pay.

i still have no idea what Twitter’s real business model is. Promoted Tweets and trends seem like a weak and risky ad-based model. I had thought they were going to let the ecosystem grow some awesome apps, add-ons and clients, buy the best, and then use sales of that as one income stream, but instead they bought Tweetie and then killed it dead.

For a while I thought they’d spot the popularity of services like TwitPic or TwitVid and build a premium offering around media sharing. Or maybe they were going to archive the world’s Tweets and use them in some clever way to research market demographics or something. But no. Not only have they trashed their archive, their search is useless.

And now they are slowly killing off the very developer community that made them great – indeed, made them at all usable – in the first place. Many have written about Twitter’s Ryan Sarver telling developers not to bother working on new clients already, so I shan’t dig further here. But it’s another nail in the coffin for all the goodwill and love that had built up around Twitter.

Whether deliberately or by accident, Twitter has created a service that people, businesses, NGOs, governments and grassroots groups now need in order to communicate with their constituencies. And many organisations, and many different types of organisation, rely on Twitter in one way or another. They also rely on a number of 3rd party services to help them understand how well they are doing, and what they are doing, on Twitter.

But Twitter’s rate limits – which stop 3rd party apps from abusing their API – are now starting to affect the use of Twitter as a serious communications tool. I am in the process of writing a research report, and yesterday I wanted to do a bit of research into the usage of Twitter by some well-known organisations. One of the tools I was using was Twitalyzer, and I came across this error:

Twitter is refusing to process any more of our requests right now! Twitter imposes certain rate limits on all partner applications and we have hit one of those limits.

Now, when I check, I get this message too:

The Twitter Search API appears to be largely OFFLINE today. On Friday, March 18th we started seeing a dramatically elevated rate of search-related errors during our processing. We are talking with Twitter but have not found resolution yet. All accounts and account processing appears to be affected. We are doing everything we can to resolve this issue but for now it appears out of our hands.

Now, it’s one thing to get errors like that if you’re doing an ego search, but if you’re using Twitter for professional purposes you need it to be up and running all the time, and that means that up and running for 3rd party tools as well as for Twitter.com. That means having an API that is reliable, that people can depend on not to die or get rate-limited into oblivion.

It’s not just Twitalyzer that’s having problems. Tweetstats had problem with a massive queue (which I guess is their way of getting round the rate limiting), and Kevin has reported problems with Nambu, a desktop client. Slowly, it seems, Twitter is killing off its ecosystem with API changes, rate limiting, poorly developed clients/apps, annoying new features (the ‘dickbar’) and restrictive API policies. Whether this is deliberate or not, I wouldn’t like to say. But it is shortsighted.

There is still an opportunity for a premium Twitter account, one that I and, I am sure, many businesses would happily fork out decent cash for. A Twitter account that guaranteed me reliable service, some way of ensuring that I don’t get rate-limited when using 3rd party apps, and perhaps even some additional premium services that I haven’t dreamt up yet. That, yes, I would still pay for.

Now, you might say that Twitter is caught between a rock and a hard place: They don’t have the scale of Facebook, and they don’t have the income streams either. With no credible business model, further scaling could be difficult, and investors may be militating for some sort of return by any means possible. This could be pushing them into shortsighted strategies to scrape some dosh in any way they can. Now that’s speculation, of course, but I’m still struggling to make any sense out of what they actually are doing, or see how it translates into a decent business model.

Certainly, I have to think that Twitter are playing a dangerous game with their users’ goodwill. People keep saying that they have lock-in because there’s no credible competition, but where is Friendster now? Or MySpace? Yeah, they still exist, but as shadows of their former selves.

Twitter can’t take their users for granted, because they can and will go elsewhere. Twitter have made a phenomenal tool, one that both businesses and individuals find useful enough to want to pay for, yet bit by bit they are breaking it. Not big. Not clever. And not a strategy for long term business sustainability.

03.03.11
by

Ada Lovelace Day: 7 October 2011

Cross posted from FindingAda.

As announced on the front page of the Ada Lovelace Day site a few weeks ago, the date of this year’s Ada Lovelace Day has moved to Friday 7 October 2011. Please put it in your diary!

I didn’t take the decision to change the date lightly. We’ve had two years of ALD being in March, and it was starting to become a bit of a tradition, so the idea of moving it to later in the year has worried me a bit, as I don’t want to lose momentum. But by early January it had become clear that things just weren’t going to be ready in time.

Although I have had some fabulous help from some wonderful people, the responsibility for getting things moving still lies with me, and the last six months has seen me incredibly busy with work. We’re in the middle of a recession, so I feel grateful for having such a full diary, but the knock-on effect has been that I’ve not been able to give this year’s Ada Lovelace Day the love it deserves.

It turns out that March is a supremely bad time of year to have a recurring event. Despite trying to get things moving towards the end of last summer, I didn’t make much progress and before you know it, it’s Christmas and everyone’s really busy, and then New Year has come round and suddenly things aren’t ready and it’s all getting a bit tight. Add a trip to India in February to the mix and deadlines throughout March and it became clear to me that something had to change.

The March date was always arbitrary, picked because I was too impatient to wait any longer! The October date has been picked because it’s far enough away that it gives us a chance to get our ducks in a row, but also because (hopefully!) it doesn’t clash with school and university calendars. I’d very much like to do a bit more outreach this year, and would like to have more resources for teachers, pupils, university lecturers and students. A date that’s in term-time, but not too near Easter or in exam season is a more important consideration now than it was two years ago.

There are other changes afoot too: I’ve also shifted the mailing mailing list from Yahoo to Mailchimp, so provide us with more flexibility. Please do join up – there’s a form in the sidebar of FindingAda.com. I’ll be sending out monthly updates once we have a few people subscribed, with more updates closer to the big day. You’ll be able to manage your subscription and unsub at any time you like, so take the plunge and subscribe today!

Finally, I do need your help to spread the word about the new date, so please do blog, tell your friends, Twitter, and Facebook followers! Ada Lovelace Day: 7 October 2011.

01.05.11
by

Hacking the BBC

I spent several months last year working with a team of people, including Kevin, on a retrospective of BBC Backstage, which has now been released as a free ebook. As I wrote for the introduction:

BBC Backstage was a five year initiative to radically open up the BBC, publishing information and data feeds, connecting people both inside and outside the organisation, and building a developer community. The call was to “use our stuff to make your stuff” and people did, to the tune of over 500 prototypes.

This ebook is a snapshot of some of the projects and events that Backstage was involved in, from its launch at Open Tech 2005, through the triumph of Hack Day 2007 and the shot-for-web R&DTV, to current visualisation project DataArt. We take a diversion to Bangladesh to see how a Backstage hacker helped the World Service keep reporting through the horrendous Cyclone Sidr, and look at the impact of the ‘playground’ servers, used inside the BBC.

Backstage’s mandate, throughout its history, was for change. It changed the way people think, the way the BBC interacted with external designers and developers, and the way that they worked together. So what remains, now Backstage is no more? The legacy isn’t just a few data feeds and some blog posts. Backstage brought about permanent change, for the people who worked there, for its community of external developers and for the BBC. What better legacy could one ask for?

Jemima Kiss has written a great piece on The Guardian about it:

Night has finally fallen on the visionary and quietly influential five-year project that was BBC Backstage, a collaboration of ideas, experiments and talent that informed and defined some of the corporation’s best technology work.

Now set to be replaced by a cross-industry developer network – a repository for data from many media organisations and tech companies – this special corner of the BBC devoted to developers has been wound down.

[…]

Woolard talks of Backstage in three phases: creating a space to make this kind of experimentation and open innovation possible; engaging the developer community; and a third stage that takes these findings and this attitude of openness further across the BBC and its output. He points to last year’s BBC 2 series Virtual Revolution, which explored the impact of the web, and was heavily influenced by the R&D TV project led by Rain Ashford, which also filmed wide-ranging interviews with high-profile technologists and allowed viewers to cut and shape footage for their own use.

Now, says Woolard, it is normal to talk about openness, innovation and working with external developers – and he claims the BBC is “fully technology conversant” in what it needs to do.

Backstage was born on the same day as the Open Rights Group, on 23 July 2005 at the Open Tech conference. I was rather busy bootstrapping ORG whilst Backstage was getting off the ground, but I kept my eye on it nonetheless.

Backstage and ORG had a lot in common beyond their birthday – a desire for more openness, a fervent dislike of DRM, and a strong DIY ethic. We also shared a community: many of our supporters were also Backstagers. In many ways, we were two sides of the same coin, with ORG campaigning publicly for change, especially around DRM, and Backstage working the inside line. I like to think we had a nice pincer movement going on.

Backstage had some amazing moments, most notably HackDay at Ally Pally, which I sadly missed due to being on a plane at the time. Spending time last year reading over old blog posts, it was great to be reminded of what a vibrant community Backstage built and what groundbreaking work they did, especially with events. Most importantly, they redefined expectations around open data and standards.

Congratulations to everyone who was involved in Backstage, and huge thanks to the Hacking the BBC team!

11.04.10
by

Sky News got the argument it wanted

Last week, Sky News announced the closure of its discussion forums.

Simon Bucks, Sky News Online’s associate editor, wrote:

We did this after a lot of thought and consideration. Although the boards were very popular, a small number of people had hijacked them and reduced the level of debate to meaningless abuse.

He continued:

At Sky News we welcome robust debate about the news, but we want it to be of a high standard. I am afraid that too often on the discussion boards threads which started intelligently would degenerate into mindless name calling.

The closure comes a couple of months after Sky Sports quietly closed their forums, saying:.

The forums have been a popular part of the site for several years but we are no longer able to provide the sort of service users expect from Sky Sports.

Some of the commenters on the Sky News announcement aren’t very happy about this turn of events. User TryAgain1234 said:

400+ replies and hardly a response from Simon, goes to show exactly how interested Sky really are in the comments of their customers. He can’t even be bothered to respond to his own blog.

Any news community manager worth his/her salt will tell you that the involvement of the journalists in the comment threads on their blogs is essential to the debate. The same is true of forums: If you are running a news forum, having your news journalists engage with the discussion can help keep the tone of the forum polite. Of course, this is predicated on the journalists in question keeping a civil tongue in their heads – and not all do.

Another important influence on how a community develops is how the people running it react to the different behaviours that their commenters exhibit. News communities often struggle because comment threads on contentious issues are highlighted, rewarding bad behaviour. That’s because of an editorial miscalculation: Because contentious threads get lots of comments, they are mistaken for successful threads, and are so promoted in order to get even more comments. The metrics are purely quantitative. By any qualitative measure, most discussions around hot issues are utter failures, devolving into slanging matches and providing no value to readers, participants, the news organisation or its advertisers. Indeed, vitriolic comments can put advertisers right off a site.

When news communities go bad, it’s often because they’ve been mismanaged or not managed at all. Commenter Sphinx said on the Sky News blog:

meanwhile over at a differentsky posters are getting used to a forum where the admin does respond to things and does care.

If Sky News have not been paying full attention to their community, then they only have themselves to blame when things go south. You can’t just leave people to it. As human beings we are used to living within constraints, and the idea that the web is a place where they are not needed is a myth. Communities need limits, and those limits need to be communicated, discussed and thoughtfully enforced.

Ultimately, you get the community that your marketing deserves. If you market your forums as News Fight Club Online, you’re going to get exactly that. Asking people if they are ‘looking for an argument’ sets up an expectation in the user of extreme hostility, so they will react intemperately to the slightest thing.

I am entirely unsurprised by the closure of Sky News forums. I could have predicted its demise in 2007, when Sky started running these idents.

10.13.10
by

Are journalisms start-ups being appropriately funded?

If you’ve been paying close attention, you might be aware that myself, Kevin and our friend Stephanie Troeth are working on a journalism start-up called NoThirty. Kevin and I have been thinking of ideas for a journalism project for about two years now, talking over different concepts and throwing out the ones that didn’t seem feasible. In January we started work in earnest, bringing Steph on board as our co-founder. We are now at the stage where we need to find funding so that we can get our prototype built.

I have always been keen on the idea of using grant funding wherever possible, certainly in the very early days. Grants are hard to get, slow to arrive and come with strings attached. Almost all money has strings and the key thing is to understand which strings are acceptable and which are deal-breakers.

Last week, we were alerted to two possible grants that we could apply for, one independently and one via a university partner. As I read through the grant details, I was struck by how some of their requirements would immediately undermine our start-up’s business models.

I’ll talk first about JISC, the Joint Information Systems Committee, which supports the use of technology in research and higher education (as we are not in HE/FE, we’d be applying as a partner of a university). They are currently running a programme aimed at improving and expanding the use of digital archives through collaboration between educational bodies such as universities, private business partners and the general public. NoThirty would fit into the remit of this programme, but JISC require “software outputs” to be open source and for there to be enough supporting information for that software to be reused. They also require content to be open access.

Because JISC are focused on an entirely different sector to the one we live in, I don’t say the above as a criticism. Indeed, I fully support their drive to ensure that publicly funded work done by educational bodies should be made available to everyone. But it became clear that their grants are not aimed at people like us, in the position we’re in, and their requirements would scupper us.

Our main aim is to create a sustainable business, so any requirement to open source our code has to be considered very carefully. Whilst we could release libraries rather than the full code base, until we are further down the road we won’t know if that is an appropriate route for us to take. Committing right now to open source/open access could turn out to be commercial suicide and it’s not a decision we can make yet.

Note that I am fully in favour of opening up as much as possible, as I believe that it helps create a healthy ecosystem around one’s service, but until you know the exact shape of what you’re building it is hard to predict what can be made open source/open access. There isn’t an open-closed dichotomy -both can live quite happily side by side in the same business – but one has to be able to make choices based on what is best for the health of that business.

The second funding source we considered was the Knight News Challenge which opens for applications on October 25th. But looking through the FAQ, which is the closest thing I can find to terms and conditions, I came across the same requirement for us to go open source at the end of the grant:

By “open-source” we mean a digital open-source platform that uses a code base that can be used by anyone after the grant period to either replicate your project in their community or to build upon it. You will own your platform, but you will have to share under GPL or Creative Commons licensing.

Again, I understand why they are doing this, but in this case, I feel that Knight is throwing the baby out with the bathwater. Where JISC exists to encourage the use of technology in education, Knight exists to help create new journalism organisations and a healthy journalism ecosystem. But by insisting on full open source, not just libraries or other separable parts, they immediately limit the type of projects they attract.

Most of the people I’ve spoken to about it attribute this insistence on open source to be a result of EveryBlock’s acquisition by MSNBC.com. EveryBlock got $1.1m from the Knight News Challenge over two years to fund development and were bought for, rumour has it, $5m to $6m. Whether Everyblokc is the cause or not, Knight’s open source clause causes problems.

If your project is a public service journalism tool or project, opening up your code makes sense. But if you’re looking at creating a financially sustainable business, then making your site/service/platform/product easily replicable by anyone could be a daft thing to do. If you have created something genuinely new and compelling, the result of making it easily replicable could be that someone with more money and bigger guns comes along and wipes the floor with you.

How does this motivate journalism entrepreneurs? Entrepreneurs work very hard to make their dreams a reality, they take risks not just with their money, but with their time, reputation and their career. What’s the pay-off for all that risk? In the rest of the start-up world, it’s the dream of a big exit, a windfall that make the hard work worth it. Most start-ups never get there, but you still want to have that hope.

Taking that option away from the very beginning will put off journalism entrepreneurs who have an eye on commercial viability. It stunts the growth of the start-up community. It damages the news ecosystem over all and will have knock-on effects for years to come as projects that could have made good businesses remain scribbles of the back of a napkin because of a lack of suitable, early-stage support.

I think Knight are being short-sighted by insisting that grantees go open source. If the only projects they fund are public service projects, what happens when the grant runs out? How will those projects become financially sustainable? How will they provide employment for journalists? We desperately need to find new business models for news and there are untold thousands of journalists searching for work. This means we need to foster start-ups that could become commercially viable, that could develop new business models, and that means they could wind up in a big exit.

I know what some of you are probably thinking now: If a project is so commercially viable, let the angels and VCs fund it. That’s what they do, after all. But I don’t think that’s an answer. We’re going to need a lot of experimentation, a lot of learning, a lot of slow growth before we get to the point of understanding what the new commercially viable business models are for news. Clearly it’s not a problem that is easy to solve, because if it was, we would have solved it by now. I love the quote from Barack Obama in his recent Rolling Stone interview:

[T]ypically, the issues that come to my desk — there are no simple answers to them. Usually what I’m doing is operating on the basis of a bunch of probabilities: I’m looking at the best options available based on the fact that there are no easy choices. If there were easy choices, somebody else would have solved it, and it wouldn’t have come to my desk.

There are no easy choices in journalism innovation. If there were, we wouldn’t be in this mess.

I certainly don’t want to generalise about the motivations of angels and VCs, but generally speaking I don’t think that they have the stomach for the kind of long-term investment and support that news ventures are going to need. Private equity investors seem to prefer start-ups with simple concepts, potential for mass market adoption and a quick route to profitability.

Journalism innovation certainly isn’t simple – all the simple stuff has been done already. It’s not mass market either – the fragmentation of the market is one of the key problems that any start-up would have to overcome. The fact that audiences are time- and attention-poor and have a myriad options to choose from when it comes to how they spend that time and attention has pretty much destroyed the ‘monetise eyeballs’ business model, so any new model has to depend on niche markets. And as for profitability, given the potential need for extensive research, experimentation and change, I don’t think that many journalism start-ups will find profitability within three years, and it’ll probably be more like five. How many VCs would be happy to fund a start-up that is complex, niche, and slow to profitability?

It turns out that, indeed, VCs aren’t flocking to fund media start-ups. Matthew Craig-Greene recently wrote on The Media Briefing that (original emphasis):

Private-equity firms are not really investing in young, entrepreneurial media start-ups.

Whilst cash invested in media buyouts has grown nearly six-fold since the late nineties, investment in fast growing, maturing media businesses has less than doubled and, worst of all, venture-style investments in new, disruptive media technologies and service models has remained absolutely flat.

So far in 2010, the only significant venture capital deals in the European media sector to grab my attention have been Rockola.fm, a Spanish online music and radio business, and pr2go, an innovative, online PR agency with a distinctive “pay as you go” pricing model. The two investments together total less than 2 million.

If grants like those from Knight are inappropriate and private equity investors don’t have the guts for it, who exactly is going to fund and support innovative journalism start-ups? Matthew suggests that perhaps large media organisations should create investment funds, but many of them are preoccupied with their own survival and lack the skill to both spot promising start-up and let them develop in their own direction.

Where does this leave NoThirty? Well, we’ll keep looking for grants that have more acceptable strings and we’ll almost certainly be trying alternative funding models early next year. Keep your eye on the blog for news.