Xtech 2006: Matt Biddulph – Putting the BBC’s Programme Catalogue on Rails

Matt’s talking about the BBC’s experimental Programme Catalogue. It’s amazing. Absolutely great piece of work.

One million contributors, 1.1 million contributors, going back to the 1920s.

The BBC has some 80 year’s worth of archives, which have been catalogued. Only catalogued stuff that was archived, so no record of stuff that was broadcast but then never archived. The default format for archiving audio was, until the 80s, vinyl. They even have stuff on wax cylinder. And it was basically down to Matt Biddulph, with help from Ben Hammersley, to take their database and do something cool with it. So now you can search for anything and you’ll get as much information as possible, including:

– programmes

– xml

– tags

– by date

– search on keywords

– contributors

– feeds

So here’s the Dr Who search, which tells you how many programmes have been made, and when. And the episode page for New Earth, which includes broadcast details, a very ‘terse’ description, categories which the Beeb has been using to organise its categories for years, list of people involved and an RDF feed. So then I can search on any particular person and see what other entries they have, and can see a cool little graph showing frequency of appearances from 1930 to 2006.

There’s also the ability to do maps of who’s appeared with whom from the FOAF feed.

It’s linked into the rest of the web, for example, Wikipedia so that you can correlate data using the API at developer.yahoo.com.

The site was built in two months in Ruby on Rails. No ‘really good ideas, just really good practice’.

Matt is now going through some techie stuff regarding how he created the site.

Can do a fulltext search. People are spending significant amount of time on the site and people are linking to it not just because of the programmes, but because of all sorts of reason, perhaps talking about an event that happened in the 70s.

Built it really quickly, in two months. But deployment in the Beeb is difficult because the tech is dealt with by Siemens, so they had never seen Ruby, didn’t know about Rails, didn’t work on a fast turnaround. So five months after Matt delivered, the site was still not deployed properly.

The archives people really bought into the project, because their budget always gets cut ‘because they are librarians’. So then people high up picked up the word ‘metadata’, and the archive guys immediately said ‘we have metadata’. They got into the idea of making their stuff available.

Matt worked offsite, communicated by blog where he said what he was doing, and they would send him bugs. Had hardly any meetings, once every two or three weeks.

On the Web 2.0 meme map, does well. Very long-tail, because most of the stuff in the archives doesn’t get much attention. BBC must put its content online at some point, but they don’t have any way of knowing what people want. This gives them an idea of what people want.

Google searches coming in are people searching for really strange and obscure stuff, so there’s a lot of stuff in the database that isn’t anywhere else.

It is perpetual beta, and falls over a lot. Exactly one URL for everything on the site, and they are canonical identifiers. Not a beautiful user experience, but is very rich, very clickable.

[I have to say, this is what the BBC should be talking about and promoting. This is just the cat’s whiskers, and the Beeb are missing a trick by not shouting about this from the rooftops.]

Xtech 2006: Jeffrey McManus – Building a Participation Platform at Yahoo!

Yahoo! wants to open up to third-party developers.

Building a developer ecosystem requires:

– providing compelling products for developers, including APIs, documentation and a community

– disseminate information

– provide support

Mash-ups, not a useful term, won’t be talking about mash-ups in a year’s time, because it’ll just be normal to mix two or more sources of data. although Yahoo! did adopt the motto ‘mash-up or shut up’.

Creating communities are important.

Has included photos in slides from Flickr, which is of course a Yahoo! property.

Timeline:

– Feb 05, search APIs. Made the APIs more openly available

– May 05, Developer RSS Index; Music Engine Plug-ins.

– Jun 05, simple maps API

– Aug 05, comparison shopping API

– Nov 05, Flash maps API; AJAX maps API. No reason not to do both, so did both.

– Dec 05, trip finder API; Javascript Developer Center, JSON support.

– Jan 06, Open-source Javascript UI library and design patterns library. Design patterns library is Creative Commons licensed.

– Feb 06, PHP developer center, serialised PHP support

– Apr 06, updated maps API

– May 06, updated Javascript UI library

Had dozens of developers in June 2005, and now up tens of thousands, because they have opened up and provided APIs that people want.

Maps mashups. For a lot of US locations, Yahoo! has better resolution maps than others. US is hard for mapping but great for satellite.

Demos a site showing Bay Area mash-up of public transport stops and Yahoo! map, but done with no coding, it’s all XML.

Sponsored walk map for Walk America, using Flash API and geoRSS. Also Running Maps, which allows you to plot your route and it tells you how long it takes to walk somewhere.

Rollyo, allows you to create customised searches amongst specific sites, as a ‘search roll’.

Flickr. Related Tag Browser, you type in a tag and it tells you which related tags have been used on recently uploaded photos. Also a Flickr friend network visualisation tool.

Yahoo! Widgets – tiny apps for your desktop. Used to be Konfabulator, and made the product free. Easy to create, and can crack open the widgets to see how they work.

Most third-party users of Yahoo! APIs do so under a non-commercial licence. Commercial exceptions are make on a one-off basis. Working to make it easier to obtain a commercial exception.

Note: I don’t think I’m going to blog all the really techie talks. If you want to see other people’s notes, then try PlanetXtech.

Xtech 2006: Paul Graham – How American are Startups?

Sitting in the Grand Ballroom of the Hotel Grand Krapolinsky Krasnapolsky in central Amsterdam, at yet another conference. This is the third in three weeks, and I’ll be glad when next week comes and I don’t have to think about writing a presentation. Well, for a few weeks at least.

First up:

Paul Graham – How American are Startups?

I
‘m here to talk about start-ups. Well, if I was giving a talk about start-ups in the US, there’d be a lot more people here, so maybe that says something, or maybe I’m reading too much into it.

Could you recreate Silicon Valley elsewhere? With the right 10,000 people, yes. It used to be that geography was important, but now it’s having the right people. You need two kinds of people to create a start-up: rich people and nerds. Towns become start-up hubs when there are rich people and nerds. NYC could not be a start-up hub because there are lots of rich people but no nerds. Result: no start-ups. Pittsburgh has the opposite problem – lots of nerds, no rich people. Uni of Washington yeilded a hi-tech community in Seattle, but Pittsburgh has a problem with the weather, and no beautiful old city, so rich people don’t want to live in Pittsburgh.

Do you need rich people? Would it be best if the gov’t invested? no. You need rich people, because they tend to have experience and connections, and the fact that it’s their money makes them really pay attention. The idea of gov’t beaurocrats making start-up decisions is comic, it’d be like mathmaticians running Vogue… or editors of Vogue running a maths journal. Start-ups funded by burocrats would be competing with start-ups run by rich people who have their own money on the line.

Start-ups are people, not the buildings. Creating business parks doesn’t help start-ups, because start-ups do not use that kind of space. Where a start-up starts it stays, so all you need your three guys sitting round a kitchen table. If you can get rich people and nerds together, you can recreate Silicon Valley.

Smart people like smart people and will go where they are, so universities are good. First rate compsci depts are important to this, preferably one of the top handful in the world, and has to stand up to MIT and Stanford. Professors consider one factor only – they are attracted by good colleagues. So if you can attract the best people then you will create a chain reaction which would be unstoppable. Just takes about half a billion, which is within the reach of any developed country.

But you need a place where investors want to live and students want to live when they graduation. It needs to be a major air travel hub so people can travel easy. Investors and nerds have similar taste because most investors used to be nerds. Taste can’t be too different, but also can’t be too mainstream.

Like the rest of the creative class, nerds want to live somewhere with personality, that’s not mass produced. To create a start-up hub, you need a town that doesn’t have mass development of large tracts of land [so, no Milton Keynes then?]. Most personality is found in older towns. Pre-war apartments are built better, and people like them more.

You can’t build a Silicon Valley, you let it grow.

Any town’s personality needs to have a good nerd personality. Nerds like towns where people walk around smiling, so not LA because people don’t walk around, or NYC where people don’t smile.

Nerds will pay a premium to live where there are smart people. They like quiet, sunlight, hiking. A nerd’s idea of paradise is Berkeley or Boulder. The start-up hubs in the US are very young-feeling, but not new towns. Want a place that tolerates oddness. Get an election map and avoid the red bits.

To attract the young, you need a city with a live centre. None of the start-up hubs have been turned inside out like some Americans have. Young people do not want to live in suburbs.

Within the US, Boulder and Portland have the most potential. They are both only a great university short of being a start-up hub.

The US has some significant advantages:

1. Allows immigration. Would be impossible to reproduce Silicon Valley in Japan, because most of the people there speak with accents and the Japanese don’t allow immigration. It has to be a Mecca, and you can’t have a Mecca if you don’t let people in.

2. Won’t work in a poor company. India might one day produce a Silicon Valley, because it has the right people but it’s still very poor. US has never been as poor as some countries are now, so we have no data to say how you get from poor to Silicon Valley. There may be a ‘speed limit’ to the evolution of an economy.

3. The US is not (yet) a police state. China might want to create a Silicon Valley, but their tradition of an autocratic central gov’t goes back a long way. Gets you efficiency but not imagination. Can build things, but not sure it can design things. Hard to have new ideas about tech without having new ideas about politics, and many new tech ideas do have political implications so if you squash dissent you squash new ideas. Singapore suffers a similar problem.

4. Need really good unis, and their US has those. Outside of the US, people think of Cambridge in the UK, then pause. The best professors seem to be all spread out, instead of concentrated, so that hinders them because they don’t have good colleagues, and their institutions don’t act as a Mecca.

Germany used to have the best universities, until the 30s. If you took all the Jews out of any university in the US, there’d be a huge hole, so as there are few Jews in Germany, perhaps that would be a lost cause.

5. You can fire people. One of the biggest obsticles are the rigid labour laws in Europe, which is bad for start-ups because they have the least ability to deal with the beurocracy. Start-ups need to be able to fire people because they need to be flexible. EU public opinion will tolerate people being fired in industries where they care about performance, but that seems limited to football.

6. Work is less identified by employment in the US. EU has the attitude that the employer should protect the employee. Employment has shed these paternalistic overtones, which makes it easier for start-ups to hire people, and easier to start start-ups. In the US, most people still think they need to get a job, but the less you identify work with employment the easier it is to start your own company. All you have to do is imagine it.

A year after the founding of Apple, long after they had sold their stuff, Steve Wozniak was still working for HP. When Jobs found someone to give them venture funding, on the condition that Woz quit, he initially refused.

7. America is not too fussy. If there are any laws regarding businesses, you can assume that start-ups will break them because they don’t know them and don’t care. They get run out of places that they shouldn’t be run out of, like garages or apartments. Try that in Switzerland and you’d likely get reported. To get start-ups you need the just right amount of regulation.

8. US has a huge domestic market. Start-ups usually begin by selling locally, which works because their is a huge domestic market. In Sweden, for example, the market is smaller. The EU was created to form an international market, but everyone still speaks different languages. But it seems as if everyone educated person in Europe now speaks English, if present trends continue, more will.

9. Funding. Start-up funding doesn’t only come from VCs, but also from business angels. Google might not have got where they were without angel funding of $100k. All you need to do to get that process started is to get a few start-ups going, but the cycle is slow. It takes five years for a successful start-up to produce a potential angel investors. You need angels as well as VCs.

10. America has a more relaxed attitude to careers. In the EU, the idea is that everyone has a specific occupation, but in the US things are more haphazard. That’s a good thing. A start-up founder is not the type of career a high-school kid will choose – they choose conservatively. Start-ups are not things you plan, so you are more likely to get them in a society that allows career changes on the fly.

Compsci was supposed to provide researchers, but in actual fact most students are there because they are curious.

Americas schools might be a benefit – they are so bad, that people wait til college before they make their decisions about what their career will be.

This list is not meant to suggest America is the best place for start-ups. It should be possible not to duplicate, but improve on Silicon Valley. what’s wrong with SV?

– It’s too far away from San Francisco. You either live in the Valley, or commute from SF. Would be better if the Valley was actually interesting, but it’s the worst sort of strip development.

– Bad public transportation. There’s a train, which is not so bad by American standards but by Eu standards, its’ awful. So design a town for trains, bikes and walking and cars last, but it’ll be a long time before the US does that.

– Have lower capital gains taxes. Low income tax isn’t so much of an issue, but capital gains is. Lowering the tax instantly increases returns on stocks, as opposed to real estate, so to encourage start-ups, have low cap gains. But decreases in cap gains disproportionately favour the rich. Belgium as cap gains of 0.

– Smarter immigration policy. People running Silicon Valley are aware of the shortcomings of the US immigration system, and since 2001 it has become very paranoid. What fraction of the smart people who want to come to the Valley want to actually do get in? Half? The US policy keeps out most smart people and puts the majority in crap jobs. A country which got immigration right coudl become a Mecca for smart people simply by letting them in.

So basic recipe for a start-up hub is a great university and a nice town. You could improve on Silicon Valley easily. Just let people in.

EBU: Wrapup

Last week, Kevin and I spent a couple of days in Warsaw, at the European Broadcast Union‘s Radio News Specialised Meeting. Kevin has done a sterling job of taking notes from the sessions, (although I think he has a few sessions still to write up) so I don’t want to rehash what happened, but I did want to talk about a few highlights for me.

It was, for both of us I think, a really good conference. The delegates came from across Europe and were all either public broadcasters or freelances. The atmosphere in the meeting was welcoming and open, and although I am neither a radio journalist nor a public broadcaster myself, I was made to feel as if I had as valuable a contribution to make as anyone else there.

Probably the stand-out session for me was The Dart Center for Journalism & Trauma‘s Mark Brayne, who gave a talk on how journalists and their employers deal with the aftermath of trauma, whether that be war, terrorism attacks or gruesome accidents. It’s very easy to forget, for those of us engaged in types of journalism that don’t require us to go out into the field, that there are people who end up having to deal with some harsh realities and the consequences of that are potentially life damaging. It was fascinating to me to see how the others in the room grappled with issues such as the ethics of sending a freelance into a war zone, and then to see the way that the freelance’s life is affected by those decisions.

My contribution was to the ‘citizen journalism’ panel, although by the end of our session we had pretty much all agreed that ‘citizen journalism’ is a divisive term and should be called something else. I prefer to use the phrase ‘participatory media’, because firstly it removes the implication that citizens and journalists are two different things, and secondly because it removes the erroneous concept that people engaging in these sorts of behaviours are trying to, or even want to be, journalists. It is also a more technologically agnostic phrase, not implying the written word as ‘journalism’ so often does. After all, much participatory media is photos, video or audio, so to think that it’s just blogging is seeing but a fraction of the story.

In a dramatic contrast to the WeMedia fiasco, the big media people in the room were really interested in different types of participatory media technologies such as blogs, podcasting, photosharing, videoblogging etc; in the different behaviours shown by those engaging in particpatory media; and in the different scales at which these sorts of projects can work. Vin Ray from the BBC’s College of Journalism asked me for the mindmap that I had thrown together for the session so here it is.

We also had some really good contributions from Holger Hank, Head of Multimedia at Deutsche Welle, who spoke about their blogs: a US election blog, one covering an assent of Mount Everest, and now some World Cup blogs, the most popular of which is the Spanish one. DW also run The Bobs – The Best of the Blogs – which gives awards to journalistic blogs in nine languages. One past winner was a Chinese blog about dogs, which was a subtle commentary on the way people are treated in China using dogs as metaphors. He also talked about how blogs are taken up differently by different cultures, for example, the way blogs are viewed in South America is very different form North America, and he explained how German blogs aren’t as original or self-confident as American ones.

Arthur Landwehr, Chief Editor at SWR and self-confessed ‘non-expert’ talked about the trends in blogging that he observed whilst working in the States. I’m not sure I agree with his assessment of US political blogs, but his discussion of religious blogs was fascinating. It seems that some churches in the States are seeing blogging and podcasting as a serious threat, because people are podcasting sermons and the congregation are listening on their commute to work and not coming to church. Ex-church goers, particularly rebel Mormons, are also using blogs to criticise their churches, having found a way to voice their opinions and talk about their experiences that they didn’t have before.

Rob Freeman, Head of Multimedia at the Press Association, demonstrated a number of mash-up sites, showing what you can do with crime statistics and Google Maps, for example.

It was a shame that we didn’t have very long for questions, but I got the sense that people were very curious about what could be achieved and how it could be done. Many of the delegates had not previously come across or thought about participatory media, so for them it was entirely new area. I wish I could have had time to demo NowPublic to them, but there simply wasn’t the chance.

After the end of the conference, which was nicely paced with not too many sessions and an adequately long lunch that involved a decent amount of food supplied by our gracious hosts, Polskie Radio, we had the opportunity to talk further. I had some relly great converations with several people, but the one that stands out was Urban Hamid. Whilst we were being shown around Warsaw on the guided tour that had been organised for us (and which was great fun) we had a really cool chat about Creative Commons and the benefits to freelances of releasing archival video or audio footage for others to reuse and remix.

I’ve had similar conversations with journalists before, and their reponse is often ‘But if I give my stuff away, how can I make a living out of it?’, but Urban immediately understood that by giving your stuff away under, say, a non-commercial licence, you can bring your work to the attention of a lot more people, and thus get yourself more work. It was a real pleasure to talk to him, and I am hoping soon to hear about his new CC-licenced video archive!

Overall, I came away feeling relieved that not everyone in the media is as clueless and small-minded as those whose ‘leadership’ we were subjected to two weeks go. There has been created, by a small group of press and bloggers, a false sense of antagonism between the two camps, yet if you were looking for that tension last week in Warsaw, you would not have found it. Indeed, so much did I enjoy talking with the journalists I met that the EBU conference turned into a much needed tonic against the bitterness of the previous week’s stupidities. I just hope that the other delegates feel as welcomed into my world as I was into theirs.

EBU: Covering Iraq

Reporters without Borders say that the war in Iraq is the deadliest for journalists since World War II. They report that 93 journalists and media assistants have been killed in Iraq. By comparison, over 20 years, 66 journalists were killed covering the war in Vietnam.

This session probably more than any other really unveiled the ethical dilemmas and life or death decisions that journalists working in war zones and oppressive countries face. It makes my job, sitting behind a desk in London now, or when when I was in the field in the US, look cushy.

Continue reading

EBU: Covering traumatic stories

I asked Suw about blogging this session, and she said that it was an important transparency exercise. It might help humanise journalists and help people understand what we do.

As Mark Brayne, with the Dart Center for Journalism & Trauma, points out, you wouldn’t send a journalist to cover finance in London without knowing the difference between FTSE and the NASDAQ. You wouldn’t send someone to cover the English premiership without knowing the difference between the rules of American football and the sport the rest of the world calls football. However, journalists are sent into life-threatening situations without knowing anything about dealing with trauma.

Continue reading

EBU: Enough international news?

Does the media give enough space to international events? Tip O’Neil, a former speaker of the US House of Representatives is credited with saying: All politics is local. And for most people, all news is local. People care about what happens in their back yard much more than halfway around the world.

But in the 21st Century, there is no doubt that events halfway around the world have impact. Unrest in the Niger Delta, impact global oil prices. As we just noted, cartoons published in Denmark set off protests across the Middle East and in Pakistan and Afghanistan.

Newsgathering is expensive. International newsgathering more so. Does your media give you enough international news? And for us in the media, how do we pay for it?

Continue reading

EBU: Does the media have the right to offend?

Lisbeth Knudsen, the managing director of Danish Radio led things off with a quote from Bob Dylan: “Something is going on. You don’t know what it is. Do you Mrs Jones?”

She said that the world probably still doesn’t know what is going on in the wake of publishing Muhammad cartoons. DR published the cartoons. It was not merely a statement on freedom of speech, she said, and added, that they felt that it was important to let people know what they were talking about.

Continue reading

Building bridges in Warsaw

Last week after months of redhot of rhetoric and heightening tensions, bloggers sent a letter to the Great Satans in the meainstream media ‘proposing new solutions’ in their long standing conflict. Oh wait, I’m conflating the WeMedia conference with the Iranian nuclear crisis.

Suw and I are proof positive that bloggers and the mainstream media can get along. This week, we’re off to another conference. Yes, spring has sprung, and the conferences are in full bloom. We’re off to a European Broadcasting Union conference in Warsaw where the theme is: Public Service Journalism and the Art of Building Bridges.

Continue reading

I’m listening

I was at the WeMedia conference where Suw was an online curator. Our friend Kevin Marks thought her role was, “pointing out the old media dinosaurs in the museum”.

As Ian Forrester points out, my position here is pretty tricky and slightly dangerous. As I have said, I work for the BBC. I am on the BBC’s blog steering committee as one of the ‘bloggers’ who doesn’t represent one of the major divisions in the corporation. I don’t say that to say, look at how important I am. This is about telling you where I’m coming from. Transparency, which as Dan Gillmor told some folks at an internal BBC briefing, journalists need to do more often.

I’m also a journalist and have been in one way or another for more than 10 years now. I think that journalism is important in a Jeffersonian sense of the functioning of a democracy, but I don’t confuse the importance of what I do with any outsized sense of self-importance.

Dan said that while he wasn’t at WeMedia last Wednesday that his impression was that it was: “Journalists vs. Bloggers conversation No. 7396”. I’m going to stick my professional neck out and say that is the impression I also got from a lot of participants, including Rebecca MacKinnon and Dorian Benkoil, here at Corante.

Bloggers are bored with this false dichotomy, and as for this journalist, I am too. There are lots of opportunities for colloboration, and as for the bloggers that I know and work with, I’ve never found bloggers to be bullies. I found that my relationship with bloggers, citizen journalists and DIY, participatory media folk of all stripes is just like any relationship: Treat people with respect and professionalism and you get the same back in spades.