Metrics, Part 1: The webstats legacy

Probably the hardest part of any social media project, whether it’s internal or external, is figuring out whether or not the project has been a success. In the early days of social media, I worked with a lot of clients who were more interested in experimenting than in quantifying the results of their projects. That’s incredibly freeing in one sense, but we are (or should be) moving beyond the ‘flinging mud at the walls to see what sticks’ stage into the ‘knowing how much sticks’ stage.

Social media metrics, though, are a bit of a disaster zone. Anyone can come up with a set of statistics, create impressive-sounding jargon for them and pull a meaningless analysis out of their arse to ‘explain’ the numbers. Particularly in marketing, there’s a lot of hogwash spoken about ‘social media metrics’.

This is the legacy of the dot.com era in a couple of ways. Firstly, the boom days of the dot.com era attracted a lot of snakeoil salesmen. After the crash, businesses, now sceptical about the internet, demanded proof that a site really was doing well. They wanted cold, hard numbers.

Sysadmins were able to pull together statistics direct from the webserver and the age of ‘hits’ was born. For a time, back there in the bubble, people talked about getting millions of hits on their website as if it was something impressive. Those of us who paid attention to how these stats were gathered knew that ‘hits’ meant ‘files downloaded by the browser’, and that stuffing your website full of transparent gifs would artificially bump up your hits. Any fool could get a million hits – you just needed a web page with a million transparent gifs on it and one page load.

This led to the second legacy: an obsession with really big numbers. You see it everywhere, from news sites talking about how many ‘unique users’ they get in comparison to their competitors to internal projects measuring success by how many people visit their wiki or blogs. It’s understandable, this cultural obsession with telephone-number-length stats, but it’s often pointless. You may have tens of thousands of people coming to your product blog, but if they all think it’s crap you haven’t actually made any progress. You may have 60% of your staff visiting your internal wiki, but if they’re not participating they aren’t going to benefit from it.

Web stats have become more sophisticated since the 90s, but not by much. Google Analytics now provides bounce rates and absolute unique visitors and all sorts of stats for the numerically obsessed. Deep down, we all know these are the same sorts of stats that we were looking at ten years ago but with prettier graphs.

And just like then, different statistics packages give you different numbers. Server logs, for example, have always provided numbers that were orders of magnitude higher than a service like StatCounter which relies on you pasting some Javascript code into your web pages or blog. Even amongst external analytics services there can be wild variation. A comparison of Statcounter and Google Analytics shows that numbers for the same site can be radically different.

Who, exactly, is right? Is Google undercounting? StatCounter overcounting? Your web server overcounting by a factor of 10? Do you even know what they are counting? Most people do not know how their statistics are gathered. Javascript counters, for example, can undercount because they rely on the visitor enabling Javascript in their browser. Many mobile browsers, for example, will not show up because they are not able to run Javascript. (I note that the iPhone, iTouch and Android do show up, but I doubt that they represent the majority of mobile browsers.)

Equally, server logs tend to overcount not just because they’ll count every damn thing, whether it’s a bot, a spider or a hit from a browser, but also they’ll count everything on the server, not just the pages with Javascript code on. To some extent, different sorts of traffic will be distinguished by the analytics software that is processing the logs, but there’s no way round the fact that you’re getting stats for every page, not just the ones you’re interested in. Comparing my server stats to my StatCounter shows the former is 7 times the latter. (In the past, I’ve had sites where it’s been more than a factor of ten.)

So, you have lots of big numbers and pretty graphs but no idea what is being counted and no real clue what the numbers mean. How on earth, then, can you judge a project a success if all you have to go on are numbers? Just because you could dial a phone with your total visitor count for the month and reach an obscure island in the Pacific doesn’t mean that you have hit the jackpot. It could equally mean that lots of people swung past to point and laugh at your awful site.

And that’s just web stats. Socal media stats are even worse, riddled with the very snakeoil that web stats were trying to mitigate against. But more on that another day.

Professionalism

First we caused the twin evils of poor communication and inability to learn from each other through our systematisation and bureaucratisation of the world of work. We devalued relationships and trust as twin pillars of human endeavour. Then we made it worse by sticking plaster on the wound, adding layers of “professional” intervention on top in the form of “internal communicators” and “knowledge managers” in our attempts to make things better. We buried the people trying to do things under increasingly collusive layers of “grown ups” pretending that this is the way things have to be.

And then… (Euan is a great read – if you’re not already subscribed, he’s well worth it.)

Professionalism is, at best, a veneer of objectivity. At worst is a false persona that distances us from our colleagues, complicates collaboration and erodes trust. Social media turns all this on its head – instead of being “professional” we can be ourselves, we can have genuine relationships with colleagues that promote trust and understanding. We can finally acknowledge that we are real people with real emotions and that those emotions matter.

danah boyd and digital anthropology

There’s a great interview with digital anthropologist danah boyd in The Guardian. I love danah’s work. We so desperately need more people like danah who take a calm and evidence-based view of the way the internet and social tools are changing society (or not, in some cases). She proved to be an essential source for my work on digital natives earlier this year, and her work should be essential reading for everyone in recruitment and HR.

The article says:

Lately, [boyd’s] work has been about explaining new ways of interpreting the behaviour we see online, and understanding that the context of online activity is often more subtle – and more familiar – than we first imagine.

Last week she outlined some examples at the Supernova conference in San Francisco, including the case of a young man from one of the poorest districts of Los Angeles who was applying to a prestigious American college. The applicant said he wanted to escape the influence of gangs and violence, but the admissions officer was appalled when he discovered that the boy’s MySpace page was plastered with precisely the violent language and gang imagery he claimed to abhor. Why was he lying about his motivations, asked the university? He wasn’t, says Boyd: in his world, showing the right images online was a key part of surviving daily life.

This is possibly an extreme case, but an important illustration. There are so many examples of people who have been fired or had job/univiersity offers withdrawn because of their behaviour on social tools. As a society we need to be really careful about how we handle the occasional mismatch between what we see and what we want to see. Sometimes it’s just more complicated than it seems and people deserve to have that complexity examined before people leap to a conclusion.

Social tools bring into the light behaviours that were previously hidden and we risk making very poor HR decisions if we don’t examine the nuance of each scenario individually. Is it right that employers expect a potential hire’s Facebook page to reflect the employers’ values rather than the reality of the new hire’s life? How long do youthful transgressions linger? After all, how many managers making hiring decisions have an entirely unblemished past? The fact that their mistakes are buried in the mists of pre-technological history doesn’t mean that they didn’t make the same ones that we witness young people making online now.

Incentives in social media

I found myself explaining to a client the other day why incentives don’t work very well for encouraging people to get involved in social media. Indeed, incentives can have the very opposite effect so must be handled with extreme caution.

This excellent video from Dan Pink explains very succinctly why incentives do not work for anything other than simple mechanical tasks, and goes on to examine the importance of autonomy.

You should also read Johnnie Moore’s blog post Incentives, innovation, community which adds yet more flavour and context, including lots of quotes from and links to studies in the same area. And you might also like my post on incentives from earlier this year.

It’s unsurprising that these flaws ‘business operating systems’ affect the way that social media projects are rolled out, as companies try to remake social media in their own image. But it’s also interesting to (sometimes) see how the more autonomous processes of social media can rub off on business culture. Might social media even be a powerful enough force to change the default settings under which business operates? I hope so.

Google’s real-time search ups the misTweet ante

Google has announced that it is going to be indexing the web in real time:

Now, immediately after conducting a search, you can see live updates from people on popular sites like Twitter and FriendFeed, as well as headlines from news and blog posts published just seconds before.

[…] You can also filter your results to see only “Updates” from micro-blogs like Twitter, FriendFeed, Jaiku and others.

[…] Our real-time search features are based on more than a dozen new search technologies that enable us to monitor more than a billion documents and process hundreds of millions of real-time changes each day. Of course, none of this would be possible without the support of our new partners that we’re announcing today: Facebook, MySpace, FriendFeed, Jaiku and Identi.ca — along with Twitter, which we announced a few weeks ago.

This announcement should make people with twitchy Twitter fingers pause. There was once a time when a mis-posted Tweet could be deleted in time to ensure it never made it into Google’s cache (although never fast enough ensure no one saw it in their timeline). Google hasn’t explained how they will now deal with deleted updates, but my own experiment this morning showed that deleted Tweets are not deleted from Google in a timely fashion (if at all).

This is good and bad news. On the one hand, Google Cache has allowed me to do a bit of forensic Twitter searching to piece together deleted conversations. There will be times when it will be an important tool for holding public figures accountable for what they say in public. On the other hand, everyone makes mistakes. Shouldn’t we be able to delete and forget them?

However Google ultimately decides to deal with deleted content, it’s a timely reminder not to update in haste.

Notes of caution and notes of hope

Stephen Baker writes an interesting piece over on Business Week sounding a note of caution about social media snake oil (and publishes some paragraphs that didn’t make the final cut on his own blog). The comments take Baker to task about the case studies he selects, but I think the point he makes still stands: It’s very easy to become a well-known name in social media regardless of your actual knowledge and experience, and quite a different thing to achieve results.

The problem of social media carpetbaggers is something I’ve mentioned before, but it’s a topic worth revisiting regularly because it’s not one that’s going away. People can be suspicious of consultants at the best of times and now that the job title “social media consultant” draws the same reaction as “estate agent” or “used car salesman”, it’s clear that the carpetbaggers are having a strong and negative impact on the perception of social media.

Therein lies the problem. Social tools can be incredibly powerful, but they have to be used well to stand even the slightest chance of success. If you have a crappy email client, you just have to learn to live with it. A crappy social media project is not only something that people can reject out of hand, it’s also likely that when it fails it is social media that is blamed, not the implementation.

Baker suggests that there is “danger of a backlash”. I’d say that the backlash is already happening – I see it already in the scorn some people heap on not just consultants but the tools themselves.

We saw exactly the same thing happen after the Dot Com Crash. Companies that had invested in expensive web projects, many of which were doomed from the outset due to being patently stupid ideas, failed to look at their own poor decisions and instead wrote off the web as a bad idea. “Internet” became a four letter word. (If you tried raising biz dev money in autumn 2002, you’ll know that!) The baby was thrown out with the bath water.

Seven years later, companies that had been quick to throw their digital talent under the bus have found themselves way behind competitors who reacted more sensibly to the end of the boom. Those who invested wisely in the web and ensured they had good digital people on board have flourished. The nay sayers are still running to catch up.

So here are two basic truths about social media:

* Social media is not a panacea. It cannot perform miracles. It cannot turn a sow’s ear into a silk purse. It can go horribly wrong horribly easily.

* Social media is not a waste of time. It can be transformational. It can empower your staff and your customers. It takes time, effort and understanding to get it right.

Companies making bad decisions now about social media are going to have a lot of running to do in five years’ time when they suddenly realise how far behind they are.

Why does a blog look like a blog?

Smashing Magazine has an article titled The Death of the Blog Post, wherein UX designer Paddy Donnelly examines a trend amongst web designers to play with their blog’s design and layout in what he calls a “blogazine” – a blog with a magazine-style layout. Donnelly’s main point seems to be that he, and other designers, find traditional blog designs boring, and feel that that each post deserves to have its own design to service its own needs, rather than have to fit in with a single blog-wide design.

I can understand why this is deeply attractive to designers. The creative freedom to tailor a page’s design perfectly to fit the text must be something designers often crave. And the examples he gives, particularly those from Dustin Curtis, look lovely. But the idea of designing each post afresh is only going to work for a very tiny minority of bloggers with the time and skills. For the vast majority of bloggers, this is just not an option.

But more than that, conflating blog and magazine is a really bad idea.

In unpicking why, we have an opportunity for some important lessons for enterprise. The first is that your blog design really, really matters. There is no excuse for you not to have a beautifully, professionally designed blog that is readable, accessible, and flexible enough to be read on different monitors or devices. If your blog is just slapped onto your corporate website with the same navigation, styling and layout as the rest of the site you should get it redesigned right now. No excuses.

The next lesson is relevant not just to enterprise, but also to web designers shifting from site design to blog design: Blog design patterns matter.

When you look at a well-designed blog you will see a number of features that I call “blog furniture”. There are many pieces of blog furniture to choose from, and not all blogs use all pieces, but most use a combination of:

  • Calendar
  • Search
  • Categories
  • Archives
  • Recent posts
  • Recent comments
  • Meta information (e.g. the admin sign-in link, RSS feed link)
  • RSS feeds from other sources, e.g. Delicious, Twitter, or news headlines
  • Badges from third party sites, e.g. Flickr badges
  • About the Author text, photo or link
  • Blogroll or list of external links
  • Tag lists or tag clouds

These are really important not just because they are useful, but because they provide the visual cues that tell visitors they are somewhere different from the rest of the site, somewhere more personal, more conversational, more informal. Take those cues away, and you risk confusing your readers, even if only momentarily.

If I pitch up on a page that looks just like the rest of the site – or, indeed, nothing like any other page on the site – then it’s going to take me a while to understand what it is and what it’s for. When we arrive on a new site, we give it less than a second to impress us. If the visuals conflict with the content, for example, we are expecting to see a blog but we are presented with something that looks like a magazine, we are less likely to hang around. The fact that it looks pretty isn’t going to make up for that moment of disconnection. (In this precise case, designers may be the exception, but that also means they are profoundly unable to judge whether or not a page causes a conflict of expectations.)

Thirdly, RSS matters. A cornerstone of the blogging world, RSS strips out all design and present, very simply, passages of text interspersed with any graphics. Donnelly’s post looks awful in RSS. Compare and contrast:

From the website

The Death Of The Blog Post - Smashing Magazine

From the RSS feed

NetNewsWire (1536 unread)

A blog post that reads in a disjointed way, with too many graphics, in your RSS reader is going to be a post you don’t bother to finish. Beautiful layouts that rely on the juxtaposition of text and image to make their point are likely to fail horribly in RSS.

I would say that if you’re creating a site with lots of bespoke pages, no blog furniture, which loses its coherence in an RSS reader, you’re not actually writing a blog at all: you’re using blogging software as the backend of a website. Now, there’s nothing wrong with that and I’m glad that such talented designers are flexing their online creative muscles. But let’s not confuse our spades and our shovels.

Over the last ten years blogs have evolved conventions because those conventions are useful. There is no reason why those conventions should hamper design, but you throw them out at your peril.

The other Two Cultures

What are the implications of reducing bureaucracy? Bill Vlasic of the New York Times asked that question in his piece about how General Motors is trying to get rid of needless form-filling and shed its “hidebound, command-and-control corporate culture”. GM is trying to shift from a company where dissent was marginalised to a culture of openness and honesty.

I agree wholeheartedly with Johnnie Moore, when he says:

My feeling is that what appears to be happening at GM needs to happen in a lot more places. It often seems to me that everytime we experience a crisis, the solution is to write more rules. […]

The intention is good, but the practical effect is to engulf people in explicit, complicated systems and reduce their freedom – based on an unconscious assumption that everyone is not to be trusted. We give ascendancy to people who are really great at theory and effectively degrade practice. I think its rooted in the idea that one person or a group of people can effectively oversee a system and control how it works with written instructions.

In order to get things done people have to find elaborate work arounds for the rules, often with anxiety. The result: it’s actually harder to create real trust the human way, using our judgement and instincts.

This reminds me of theories of management that I stumbled once on Wikipedia, Theory X and Theory Y, which were proposed by Douglas McGregor in the 60s. In Theory X, management assume that employees are “inherently lazy and will avoid work if they can”. In Theory Y, managers assume that employes “may be ambitious and self-motivated” and enjoy their work.

Whilst reality tends not to fall into two neatly opposing mindsets, the framework is still useful, especially when think about how social media fits into a corporate culture. One could extend the theories thus:

Theory X companies are inimical to social tools, because they simply do not trust staff. Concern that people will ‘abuse’ the tools in some way leads to attempts to control employees’ access to them. The company’s public blog winds up with an editorial committee, only approved managers are allowed an internal blog, and access to sites like Wikipedia or services like Twitter are curtailed. Social media projects generally fail in these cultures, if they are ever started in the first place.

Theory Y companies, on the other hand, are ready to trust their staff to do the right thing. Social software is made available to all, small talk and social uses of the tools are allowed (sometimes even encouraged), and people build stronger relationships with colleagues which increases trust and ability to collaborate. Departmental silos are broken down, communication across time zones and locations improves, duplication of effort is reduced. Social media projects generally succeed in these cultures.

Of course, in reality, corporate cultures are not homogeneous. One department may have a much more open, collaborative and sharing culture than another. The question is whether Theory Y cultures are nurtured and growing within a wider Theory X company, or are they seen as aliens to be disposed of?

(The other Two Cultures, in case you’re wondering, are CP Snow’s.)

ATA: Who are your favourite social media bloggers?

So I reckon it’s time for a bit of audience participation here on The Social Enterprise, so I’ve decided to create a new “Ask The Audience” category. I shall, unsurprisingly enough, periodically and at random ask you a question about your thoughts on social media. Simples!

Today’s question:

Who are your favourite social media bloggers?

Who are the trusted old voices whose opinions you value? Who are the up-and-comers that provide you with insight? Which social media blogs can you simply not live without?

Let me know in the comments!

CWSE Roundup – 27 Nov 09

To make sure that you don’t miss out on the blogging that I’m doing over on my new Computer Weekly blog, The Social Enterprise, I’m going to do weekly round-up posts so you can see if anything takes your fancy. Obviously I’m a bit late for last week, but I’m sure I’ll hit a rhythm soon.

Monday: Joining the Social Enterprise
Tuesday: CoTweet: Twitter tools get collaborative
Wednesday: Is tendering right for social software projects?
Thursday: The role of dopamine in social media
Friday: Merlin Mann’s Time & Attention talk

Please do pop over to The Social Enterprise, take a look around, and let me know what you think.