The Atlantic’s Derek Thompson has published a handful of graphs which he says tell us about the popularity of “viral publishers” on Facebook and Twitter, and how important Facebook is compared to Twitter based on volume of shares/likes. It’s true that the graphs do give us some very interesting insights, but they aren’t the ones Thompson thinks they are.

Thompson’s graphs are based on data from Newswhip’s Spike database. The first problem is Thompson’s sloppy use of terminology. His first graph says that it shows Facebook likes, but in the text he uses the word shares, but likes and shares are not the same thing. Liking something on Facebook is basically just giving that thing a thumbs-up, it’s a very lightweight interaction. A share is much more emphatic and gives you the opportunity to comment on the item you’re sharing. Both apparently show up in timelines, although Facebook is, as usual, spectacularly unclear on the precise differences regarding when a like will show up and when a share will, but either way, they aren’t the same kind of action.

Newswhip’s graph shows total likes and shares for each content source, whereas Thompson’s graph says that it shows “overall likes” and provides significantly higher figures than Newswhip: ~27,000,000 vs 20,878,994 for The Huffington Post, for example. This is because he has actually plotted “Total FB Interactions”, a figure from Newswhip that includes likes, shares and comments on Facebook.

This might seem like nitpicking, but when you have words like “like” and “share” being used to designate very similar but different actions, with different social meaning, you cannot just use the words interchangeably. And you can’t just chuck in comments to the mix without saying so.

Here are the two graphs for you to compare:

Thompson’s Facebook graph

Newswhip’s Facebook graph

The next pair of graphs are for Twitter. Thompson’s say they are for Twitter mentions, whereas Newswhip’s graph is for “tweets and retweets of articles”. This time, Thompson’s figures appear to be about the same as Newswhip’s, so must refer to both mentions and retweets.

Thompson’s Twitter graph

Newswhip’s Twitter graph

Thompson then goes on to take Newswhip’s total article count for each publisher and use it to calculate the total shares per article on each platform. Upworthy‘s article count is just 225, so its shares per article is ridiculously high compared to every other source. Even TwentyTwoWords, which is in second place after Upworthy, has significantly more shares per article than other, bigger sites.

That’s a big red flag for me, indicating that something odd and statistically dubious might be going on. Looking at their Facebook pages gives you a sense of how many shares, likes and comments their articles are getting. Upworthy’s are highly variable, from 51 shares, 322 likes and 10 comments to 11,934 shares, 32,800 likes and 1,260 comments. TwentyTwoWords timeline posts vary from 2 shares, 17 likes and two comments to 75 shares, 66 likes, and 15 comments. So what we’re looking at, as one commenter on Thompson’s piece says, is a few runaway hits pulling up Upworthy and TwentyTwoWords’ figures.

Thompson gives us the mean recommendations (shares/likes/comments and tweets/retweets) per article, but to draw more robust conclusions we would need to know the median number of recommendations for each site. We also need to see the range, so that we can see how runaway hits are statistically skewing the distribution.

But still, given the meme-y nature of their content, it’s no surprise that Upworthy is popular. Pointing out that internet meme-based content is particularly popular with internet audiences isn’t an insight, it’s a tautology.

Thompson concludes his piece with a huge non-sequiteur, that “Facebook is huge. Much bigger than Twitter. […] Even the biggest sites on Twitter are much, much, much bigger on Facebook.”

Well, duh! Anyone who didn’t know that Facebook is bigger than Twitter has to have been living in a cave for the last few years. Facebook has 1,189 million monthly active users whereas Twitter has 232 million monthly active users. More users means more potential for sharing. We would expect Facebook’s activity to be some five times larger than Twitter activity but we don’t, we see that it is ten times larger. That is at least in part because Thompson is comparing apples and oranges.

Facebook likes, shares and comments are not equivalent to Twitter tweets and retweets. It’s not even clear to me that it’s meaningful to compare them, because of the different levels of engagement required to complete each action. An original posting to Facebook or Twitter is about equivalent in effort, because usually these days it’s just a matter of clicking a button on the original source post or copying/pasting an URL. Resharing that within Facebook is more akin to retweeting on Twitter, and neither liking nor commenting on Facebook has an equivalent on Twitter.

In order to properly compare activity types on Facebook and Twitter, we need to compare similar behaviours, so we can compare originating posts, or sharing or retweeting, but have to cut out likes and commenting on Facebook. Newswhip’s numbers don’t allow us to do that.

What this data does tell us is, however, much more interesting than Thompson’s  analysis might lead us to believe. Knowing what kind of content plays well on Facebook and Twitter gives us a fascinating insight into the tastes of their users. Facebook likes polarised, outrage-inducing or meme-y content, and is rather uninterested in sports. Twitter likes non-partisan news, tech news with a bit of polarised news, a few memes, and a lot less of the outrage. Twitter is also not massively keen on dedicated sports sites.

And if the shares per article data has any grounding in reality – which at this point I don’t have enough data to assess – then you can also see how well highly partisan, fringe content plays on both platforms in comparison to those sites’ sizes. Russia Today, Breitbart, Alternet and The Blaze are far from being balanced or neutral news outlets, but their bias allows them to punch above their weight compared to more moderate sources such as The Atlantic, CNN and the New York Times. That too is fascinating as it points to very vocal, politically partisan subcultures within both platforms.

If we wanted to, we could look at the demographic research for all these sites and get a much deeper insight into the psychographics of users than you can get from the usual Twitter/Facebook analyses. However, that takes a bit more effort than is required to chuck a few graphs up and draw superficial and suspect conclusions from fuzzy data.

Finally, what this data doesn’t and can’t tell us is whether Facebook is driving ten times more traffic to content sites than Twitter, given that content is being recommended ten times more often than on Twitter. Indeed, it’s well known that people are happy to re-share content without clicking on the links, and in my own experience, there are differences between how willing people on different platforms are to click on links and the dwell times and bounce rates for traffic from different platforms. On one project, oddly, LinkedIn provided the best traffic with dramatically longer dwell times and lower bounce rates.

Ultimately, it doesn’t matter if Upworthy has ten or a hundred times more shares on Facebook than Twitter if that doesn’t translate into traffic and revenue.