Category Archives: History

09.22.13
by

Listen to the dawn of data journalism: Univac, the first Nate Silver

In one of my data journalism presentations I look back at the history of data journalism, and one of the key dates is 4 November 1952: the first US election when a TV network, CBS, used a computer to analyse and predict the election results.

CBS used the room-sized, vacuum-tube powered Univac.  The idea came from Remington Rand, the makers of the Univac, because sales weren’t as strong as they wanted. In a creative bit of marketing, they approached CBS to use the computer to help analyse the election results. Of course, CBS also saw a marketing opportunity and mentioned Univac in their election coverage ads. Last night, in a lovely bit of luck, I heard one of those ads.

I often listen to old radio dramas while I’m making dinner and, much to my surprise, last night when I was listening to the 13 October 1952 episode of the western radio drama Gunsmoke on Old Valve Radio, I heard CBS run an advert touting how “Univac, the electric brain” would be assisting Edward Murrow and his team on election night coverage the following week. My jaw just dropped to hear this message announcing the dawn of computer-assisted reporting, as it was called in the US, 70 years before the term data journalism came into vogue. You can hear the ad yourself on Podbay.fm, and, if old western radio drama isn’t your cup of tea, fast forward to 13:43 to hear about how the CBS election team would be backed up by “Univac, the electric brain”.

For a lot of data journalists, the CBS-Univac partnership is a famous and well known bit of history, but if you aren’t familiar with the rest of the story, it is fascinating. Both Remington Rand, makers of the Univac, and CBS saw value in the arrangement, as Wired explained when they looked back at the day in 2010.

The eight-ton, walk-in computer was the size of a one-car garage and accessed by hinged metal doors. Univacs cost about $1 million apiece, the equivalent of more than $8 million in today’s money. The computer had thousands of vacuum tubes, which processed a then-astounding 10,000 operations per second. (Today’s top supercomputers are measured in petaflops, or quadrillions of operations per second.)

Remington Rand (now Unisys) approached CBS News in the summer of 1952 with the idea of using Univac to project the election returns. News chief Sig Mickelson and anchor Walter Cronkite were skeptical, but thought it might speed up the analysis somewhat and at least be entertaining to use an “electronic brain.”

They had no idea how quickly it would speed up analysis, and early on the evening with only about 5 percent, or 3.3 m of the total 61 m votes, counted Univac had a prediction. The Computer History Museum has the printout of a prediction the Univac team sent to CBS via teletype at 830 pm. “It’s awfully early, but I’ll go out on a limb.”

However, just as traditional political pundits heaped scorn on stats wizard Nate Silver in 2012, CBS’s editors were not willing to join the Univac team out on that limb. On air, CBS hedged:

In another story about the election by Ars Technica, we learn that the Univac team figured out that they had entered the New York results incorrectly, overstating Stevenson’s votes by a factor or 10. They ran the numbers again, but they got the same result. Univac predicted that Republican Dwight Eisenhower would win the Electoral College 438 to Stevenson’s 93 votes, and the computer set the odds of an Eisenhower win at 100-1.

As the night went on, Eisenhower gained momentum. The final vote was 442 to 89 to Eisenhower, and Univac’s early prediction was off by only one percent.

While I’ve known about this story for a while, it was great to hear the advert of CBS advertising how it was going to use an “electric brain”. I also learned something new about Univac. It was programmed by computer pioneer Grace Hopper. Her team fed the computer with statistics from previous elections, and she actually wrote the code that allowed Univac to make its prediction. Sadly, her contribution was not mentioned in reports at the time.

06.23.10
by

Everything you need to know about the internet | Technology | The Observer

Kevin: John Naughton has an excellent meditation about the internet and the pervasive search for easy answers. We're living through a revolution. Get used to it. As he writes, disruption is a feature not a bug. "By implementing these twin protocols, Vint Cerf and Robert Kahn created what was essentially a global machine for springing surprises." It's an excellent piece that looks at a number of the disruptive trends online such as how to deal with information abundance, which upends traditional economics that attempts to deal with the allocation of scrace resources.
03.24.09
by

An interview with Ada Lovelace

This morning, I went to the Science Museum to talk to Ada Lovelace herself about Charles Babbage, his computing machines, and her vision and brilliance.

Ada was a most fascinating lady, and I hope that because of today, more people will know not just about her, but about all the other amazing women in technology.

I’d like to thank Steph Troeth and Steph Booth for helping me with Ada Lovelace Day. Both of them helped me to figure out what shape the campaign was going to take and were then invaluable in kick-starting it. Without their help and encouragement I’m not sure that Ada Lovelace Day would have happened at all. Thanks also go to Vicky Riddell at the BBC for deciding to run with the story on the BBC News Channel and doing such an amazing job of getting so many smart techie women on the news.

I also need to thank two men, firstly Tony Kennick, who very kindly cobbled together the Ada Lovelace Day Collection mash-up, and who put up with my last-minute-ness with grace and good humour.

And secondly, my wonderful husband Kevin who has provided me with endless support and help over the last three months, who shot the video above, and who came with me to BBC Television Centre this evening and helped me calm my nerves before my interviews.

Thanks are also due to everyone who has taken part. Ada Lovelace Day was a community effort, with everyone playing an important role in making it the success it is.

But it’s not over yet! We have another 15 hours before the day that is Ada Lovelace Day is finally over as midnight arrives in the Baker Islands, and we have a lot more blog posts still to be added to our Collection.

Even then, it’s not over. We have our first event booked at NESTA for 10th June – on which more to come – and I have a few other ideas up my sleeve too. So don’t go away – keep FindingAda.com in your RSS reader, or follow us on Twitter, and keep up-to-date with our news.

Technorati Tags:

07.20.08
by

What has prevented newspapers from being successful in the digital age?

In the daily flood of links that stream by me via RSS or Twitter, I noticed a post by Mark Schaver, the computer-assisted reporting director of the Louisville Kentucky Courier-Journal, in which he challenged the view of newspaper executives as short-sighted and out-of-touch. He pointed to a couple of projects in the US, Videotex and Knight-Ridder’s early investment in Netscape (then Mosaic). Mark said that calling news execs short-sighted and lacking in vision is overly simplistic.

What I am saying is that powerful economic forces, forces that are vastly more complicated than the simplistic drivel about newspaper curmudgeons and their resistance to change, are behind the news industry’s malaise today.

I agree with him. It is overly simplistic.

However, Videotex is a fine example of a disastrous technical project driven by the newspaper industry. The system was too slow, cost too much and didn’t provide anything that couldn’t be found easier in some other form. As often happens in the US, the FCC failed (or refused) to set a standard, hoping that the market would sort it out, and NTSC – the North American television standard – on which some of these projects were run provided too low of resolution to read text on televisions unlike the Ceefax system on the higher resolution PAL video standard in the UK. Maybe it was ahead of its time, and it’s definitely before my time. But I’ve never heard anyone in the industry hold up Videotex as an example of how to do a technical project.

Knight-Ridder was forward looking. They grasped a lot of the innovations early, partially because of their presence in San Jose. They even moved their headquarters from Miami to San Jose to plug into the new media revolution. In 1990, Robert Ingle, executive editor of the San Jose Mercury News wrote a memo that sounds eerily similar to the strategy that most newspapers are following now:

Give information to readers however they wanted it, integrate the print and online operations, and dream up new forms of advertising.

Knight-Ridder were part of the New Century Network, which was supposed to position the newspaper industry for the 21st Century. But there is a but. As BussinessWeek reported of the Network on its closure in 1998:

In a ballroom at the Newspaper Association of America convention in Chicago, a thousand bottles of champagne emblazoned with ”New Century Network: The Collective Intelligence of America’s Newspapers” awaited the hordes expected to come to toast the watershed new-media joint venture. When fewer than 100 people showed up, Chief Executive Lee de Boer made an abbreviated speech before retreating. ”They built a business and nobody came,” says David Morgan, president of the online ad agency Real Media Inc.

The reception was the first public humiliation for New Century Network, but only one in a series of blunders that culminated in the company’s abrupt shutdown on Mar. 10 (1998). Created in 1995 to unite newspapers against Microsoft Corp. and other competitors girding to woo electronically advertisers and readers, New Century Network came to embody everything that could go wrong when old-line newspapers converge with new media.

Knight-Ridder should have been leaders in how to do it right. As Matt Marshall wrote in 2006 as Knight-Ridder was on the eve of ceasing to exist:

The real irony of this situation is that for 15 years KRI was, by far, the most innovative newspaper company in the country, including its early experiments in teletext and having the first online newspaper (the Mercury News on AOL in the mid 90s).

But as Matt says in the title of his post, sometimes innovation is not enough. Newspapers continued to be newspapers, just online, as he and most of us have said over the last decade. It is proving for some newspapers a fatal mistake, although one that many of us saw years ago. And I’d agree with Matt that it’s easier to imagine a new entrant making the changes necessary to survive in this new world rather than an established newspaper.

As my friend and former colleague Alf Hermida points out from Readership Institute data, people do not have the same connection with their local newspaper websites that they do (or possibly did) with their local newspapers.

Obviously, something isn’t quite working when it comes to newspapers, ‘new’ media’ and innovation. As Mark Schaver is correct to point out, this is probably not for lack of trying at some newspapers. I know that a lot of journalists are exhausted and frustrated by reorganisations, restructurings and new strategies. I ask the following question not pretending that I have all of the answers but because I’d really like to hear people’s experience: What has prevented newspapers from being successful in the digital age?

Two years ago, Steve Yelvington wrote a post after hearing someone refer to “NCN nostalgia”, NCN being New Century Network. He said a few things that might speak to my question:

  • “But there was something else at work: technology was evolving faster than anyone’s business vision.”
  • “The notion that a we-tell-you news cartel would be relevant in a conversational universe may already be obsolete.”

The newspaper industry hasn’t adapted to the pace of news online or the pace of technological change. More than that, I think Steve is right that business vision hasn’t kept pace with technology. In the wake of the newspapers ‘are worth fighting for‘ discussion kicked off by Jessica DaSilva, Pulitzer winner John McQuaid said:

Meanwhile, the default attitude of newspaper management is still caution and probity. And if you point a gun to the head of caution and probity and say “innovate or die,” don’t expect wonderful things to happen. Instead, expect buzzwords.

Newspapers have only recently woken up that the real competitive threat isn’t from other newspapers or print media, not even from TV but from new digital businesses that might not have even existed a few years ago. Even though Robert Ingle and others saw the competitive threat 18 years ago, there has not been a sense of urgency until the last 18 to 24 months.

However, unlike John McQuaid, I would argue the over-cautious nature of journalism change is not just about boardroom conservatism. Print newsrooms are some of the most conservative places you’ll find. Journalists are paid sceptics (some might say cynics), and they approach their own business with that mix of scepticism and cynicism.

Some things have changed since Robert Ingle wrote his prescient memo on his Apple ][ in 1990. In the 1990s, tech was expensive, and I heard a lot of journalists argue that the internet was a money sink not a money maker. There was some truth to that, but very few disruptive technologies have a clear business model at the beginning. Did Google have a magical money-spinning idea with search? No, not until AdSense. But now, smart technology buys and clever use of open-source technologies can bring the cost of failure down to almost the petty cash level. Just look under the hood of Google’s massive data centres and you’ll find lots of commodity hardware lashed together with a lot of open-source technologies.

The newspaper industry also still seems to be thinking in industrial terms. Too many of the strategies I see are huge, heavy, expensive strategies instead of light-weight, nimble and low cost digital strategies. By the time the strategies are in place, the state of the art and, more importantly, audiences have already moved on. More importantly, you can attack the business model problem from two fronts. You can find new ways to make money, but you can also find new ways to make high-quality, compelling content with less money and not just with less staff.

Things are changing. A few newspaper companies are making the investments in flexible, scalable technology to prepare them for the future. They are getting serious about developing new income streams. They are freeing their content and taking it to where the audience is instead of forcing the audience to come to destination sites. But for some newspapers, it’s too late.

What would you do and what are you doing to ramp up the pace of change at your company?

Technorati Tags: , ,

05.14.08
by

Future of News Princeton: Paul Starr

I’m at the Future of News workshop at Princeton University. I’ll be speaking about data visualisation tomorrow, but Princeton’s Paul Starr kicked things off. This is a bit of a ‘rush transcript’ as broadcast would say. I’ll go back and refine it over the day.

We are on treachourous ground. Many who have tried to anticipate the future of news, including leaders in the industry have failed. Hasn’t been for lack of effort. Back in 1970s, news industry invested millions of dollars into teletext or video text, electronic ways to supplement existing ways of delivering news. They failed to see the future coming. In 1992, news industry leaders failed to see the internet in the future of news. Network meant to them broadcast networks. They saw overall trends, but they said that the broadcast and print media faced no serious threats.

News media are in the midst of transformation greater than anyone in the history. Many in old media organisations are reeling from the impact. The future of news comes from instability of the present. The future of news is coming not from single change but serial changes. Consider earlier changes: Postal network, telegraph, radio in 20s and TV in 1940s. They all had a compressed period of development with with minor innovations following.

The succession of changes now are different; innovations are piled on top of innovations. Political and legal decisions could upset things as they have developed.

To consider the future of news, we can try to extrapolate trends that are underway. We can analyse changes in major social frameworks. We can brace ourselves for the unexpected.

He looked at trends and how they fit together shift in social frameworks. He looked at the peculiar features US news media. Central institution in news media: general interest newspapers. 1930s, broadcast networks radio and then TV. Both newspapers and broadcast benefitted from government largesse. They became hugely profitable. Leading national news media supported public interest. They used other revenue streams to cross subsidise news: Classified ads for newspapers and entertainment for TV.

It paid for international news gathering. High profits, high standards, high prestige. Up to early 20th, newspaper industry highly competitive. by middle 20th century, most cities had one major newspaper. Warren Buffet, newspaper business used to be an easy way to make huge returns. One great newspaper said, I owe my fortune to monopoly and nepotism.

Mr Starr quoted Buffet at length from a letter to Berkshire Hathaway investors:

For most of the 20th Century, newspapers were the primary source of information for the American public. Whether the subject was sports, finance, or politics, newspapers reigned supreme. Just as important, their ads were the easiest way to find job opportunities or to learn the price of groceries at your town’s supermarkets.

The great majority of families therefore felt the need for a paper every day, but understandably most didn’t wish to pay for two. Advertisers preferred the paper with the most circulation, and readers tended to want the paper with the most ads and news pages. This circularity led to a law of the newspaper jungle: Survival of the Fattest.

Thus, when two or more papers existed in a major city (which was almost universally the case a century ago), the one that pulled ahead usually emerged as the stand-alone winner. After competition disappeared, the paper’s pricing power in both advertising and circulation was unleashed. Typically, rates for both advertisers and readers would be raised annually – and the profits rolled in. For owners this was economic heaven. (Interestingly, though papers regularly – and often in a disapproving way – reported on the profitability of, say, the auto or steel industries, they never enlightened readers about their own Midas-like situation. Hmmm . . .)

As long ago as my 1991 letter to shareholders, I nonetheless asserted that this insulated world was changing, writing that “the media businesses . . . will prove considerably less marvelous than I, the industry, or lenders thought would be the case only a few years ago.” Some publishers took umbrage at both this remark and other warnings from me that followed. Newspaper properties, moreover, continued to sell as if they were indestructible slot machines. In fact, many intelligent newspaper executives who regularly chronicled and analyzed important worldwide events were either blind or indifferent to what was going on under their noses.

(The full 2006 letter from Buffett can be found on Berkshire’s site. It’s a PDF.)

Coming off of period of monopoly profits, there is no way that (newspapers and broadcast could duplicate those returns online. In the new environment, they are no longer protected from competition. On the internet, Craigslist and others are cutting into their ads and aggregators cutting into their readers.

Broadcasters also coming out of period of limited competition. Once, a broadcast licence was a licence to print money. Cable and then the internet caused decline. From spectacular height broadcast has fallen, and no one sheds a tear.

not only loss of audience but development more disturbing, long term decline in attention for news. Robert Putnam look at casual role in generational change. Those come of age during WWII and Cold War, high level of engagement. Younger generation, decline in participation in civic society. Also, younger generation decline in news consumption. Despite rising level of education.

How media contribute to these trends, Putnam sees television itself as reason for declining civic engagement. Princeton research found that TV first increased civic engagement that only decreased with multiple channels.

Recent State of the News Media report every sector of news media apart from two decline – online news and ethnic sector. Since 2001, decline in work force 7-10%. Number of American foreign correspondents has dropped 188 to 141. Broadcast networks have shut bureaux around the world. Amount of foreign coverage has declined from 27% of total in 87 and 97 to 24% in 2004. Internet provides access to international news such as BBC. Declining number of foreign correspondents, decrease boots on the ground.

But there has also been a decline also in state and local reporting. Tom Rosenstiel said: More and more reporters are following fewer and fewer events. All news organisations are becoming niche players.

Another trend, ideological. In 19th Century most of press is partisan, and there has been a recent return to partisan for confluence of factors. Until 1980s, partisan tendencies in broadcast held in check by regulation – FCC fairness doctrine. There were a limited channels and for competitive reasons they had to appeal to broad ideological cross-section of the population.

Abandon Fairness Doctrine 1987, rise of Rush and Fox News, a complete media system enveloping its audience. No liberal equivalent to right wing talk shows. Liberals on TV, but no liberal equivalent to Fox and right wing talk show. Part of broader trend.

Next he said:

Newspaper web site can get back into the business of breaking news and broadcast website can provide much more depth. Different rhythms of media are lost when media migrate to the web.

Each news organisation has an identity tied to legacy medium but these fade into the past. Broadcast and print will combine into single hybrid media. Distinctions are breaking down. One of most important distinction breaking down is between media and their audience. Journalists can no longer put themselves between public and public figures and opinion makers.

The future. the underlying financial and competitive pressures will increase.

They are living off aging audiences and obsolete business models. As older audiences die off, all of these news organisations face a mortal threat.

However, bloggers rely on old media. “Just as any parasite doesn’t want to kill off their prey”, but unclear if bloggers and citizen journalists can be profitable enough to support a viable news gathering function. News organisations still remain profitable, and they can eliminate 70% of cost structure, that which they use to support the print publication.

It is not a question of which organisations survive but how new framework affect what they do and how they do it. What are basic features of this emerging framework? One comes from Yochai Benkler: Mass media public sphere and networked public sphere.

Mass-commercial media had industrial model reliant on large audiences and large amounts of capital. The system has constraints on feedback.

The declining price of computation, production and storage has put into hands of up to a billion people around the world means of cultural and information production. Opportunities for feedback increased. Benkler says that internet has evolved systems for information synthesis and analysis.

There is a good interview with Benkler on Kottke that expands on these issues.

Big media has two capacities that might not be replaced: 1) They can mount long term investigation 2) They can stand up to government (see Pentagon Papers). Rather than replacement, networked public sphere may overlap and interact with mass-commercial media. Josh Marshall’s TPM bring down Senate Majority Leader Trent Lott, but Josh Marshall depended on main stream media to deliver knock out blow.

Early history of the internet may have spoiled us into ultimate direction it will take. Lessig and Zittrain say that there is an ongoing struggle between open internet and those who want to lock the internet down. In connection of news media, cell phones and handheld devices become more secure platforms for news delivery than desktop computer (I think to myself what about Android or open mobile platforms? I guess it depends on open networks.)

Second set of issues, economics, if MSM no longer able to cross subsidise and Benkler’s collaborative network public sphere can’t produce, there will have to be other business models. In the US, only public radio provides quality radio news. That is one possible replacement.

What must not be lost in transition, values must not be lost. The press must remain independent and committed to a social mission.

Q: Issue of partisanship in news media. One hundred years ago, penny press quite blatantly partisan.

A: Disengenous of me as co-editor of American Prospect to criticise partisan press. Partisan press in 19th century marked an error of high civic participation. Not in itself a bad thing. Unruly aspect of democracy.

Q: Old media, more directly regul-able (easier to regulate). What is your opinion of the BBC?

A: The BBC is a great institution. Model of public service broadcast journalism in Commonwealth countries financed by tax. US has historically opposed to any special taxation for press, TV and radio. We never had support to finance anything like the BBC. Shielded from everyday influence of politics. It has established itself as authoritative source of news. Somewhat surprising, it has expanded its reach beyond the UK.

Q: Pentagon Papers quintessial reason for why we need press. But think not good example. If I had them, I could put them and no one could take them down. (Possibly a good example is WikiLeaks.)

A: What you are talking about is putting investigative reporters on story for several months. Can non-market model sustain that type of investigations is a question. Lot of reasons to worry whether that is true.

Q: You seem to say that more is better in terms of number of reporters, number of bureaux. Is there a level?

A: I can’t say what optimal level. But this is a period of time when Americans need to know more about the world given the circumstances we face.

10.20.04
by

Tracing the Evolution of Social Software

Christopher Allen writes an interesting essay on the history of social software.

The term ‘social software’, which is now used to define software that supports group interaction, has only become relatively popular within the last two or more years. However, the core ideas of social software itself enjoy a much longer history, running back to Vannevar Bush’s ideas about ‘memex’ in 1945, and traveling through terms such as Augmentation, Groupware, and CSCW in the 1960s, 70s, 80s, and 90s.

By examining the many terms used to describe today’s ‘social software’ we can also explore the origins of social software itself, and see how there exists a very real life cycle concerning the use of technical terminology.

As you might expect of a blog entry, his post is fleshed out even further by a huge raft of comments and trackbacks which are also well worth exploring.