Future of Web Apps, a week or so later

Since the Future of Web Apps, I’ve been a wee bit on the busy side, flying to Washington DC, enjoying the snow, and doing an absurd amount of shopping (mainly of the geek variety). In between all that, I’ve been thinking a lot about the presentations I saw.

I’ve done a lot of conferences over the last year, and had started to worry that I’d got a bit jaded and bored by the whole sitting about listening to people opine schtick, but the Future of Web Apps was just a great conference. Ryan and Gill Carson did a sterling job of getting a group of fascinating speakers together, along with 850 delegates and wifi that didn’t crap out.

Whilst there were a lot of really good talks, Joshua Schachter from Del.icio.us was definitely a stand-out for me. I’ve historically had my problems with Del.icio.us, but I was very impressed with Josh’s honesty and openness about the lessons he’s learnt, many of which I think are relevant not just to those developing web apps, but for anyone doing any sort of project that involves the public in any capacity.

Some gems:

Watch for interesting behaviour in the system. If people do things you didn’t expect or intend, that’s neat.

Don’t look for problems you don’t have, because someone else who is passionate about it will solve it better than you can.

Watch what you find your time doing. If you spend a lot of time building a feature that no one uses then that’s a waste.

If you’re doing something different, then understand where you’re breaking away from normal behaviour.

David Heinemeier Hansson was also really interesting, with a view on programming that differs quite significantly from that of many people I know. The way he puts it, Ruby on Rails is about minimising the number of times you have to reinvent your wheels.

Not reinventing the wheel is something that people should think much more about, not just in programming but in all aspects of life and business. We all have learning curves, but we need to make them as flat as possible and learning ways to cut down on repetition and reinvention. I liked quite a bit of the stuff that David said:

Look for patterns to abstract, conventions […]. Look for relationships between things. Conventions are there for the 80% of the time. When you work inside those conventions, you don’t need to do work, because you can get the conventions to do it.

Flexibility is overrated. The assumption is that flexibility is good, but you are trading valuable parts of your system away to get flexibility, and usually it’s not worth it. We should stop chasing flexibility.

It’s not about whether something is possible or impossible, it’s about whether it should be encouraged or discouraged.

Create invitations to do better, reminding you that you should be consistent, that you should be creating tests, and that you get benefits from doing things the right way.

I still have a lot of thinking to do about all that I heard, and it’s rare these days that I come away from a conference with my head full and my enthusiasm all a-bubble. I think it’s because this was a conference for developers, and I’m not a developer.

Blogging conferences, for example, rarely leave me feeling that I’ve heard something new or interesting, because most of what’s said is what was being said last year, or two years ago, or before even then. Conversations with bloggers, yes, that’ll do it. Blog posts too, even though I read so little these days. But rare is the blog conference that has be bouncing with newly-learnt gems.

Maybe this is about comfort zones. Sometimes your comfort zone gets so comfortable that it gets dull. Put yourself outside of that, into strange new territory, and suddenly it all looks exciting again.

All that said, though, as I sit here writing this, I’ve just discovered that BlogTalk Reloaded is scheduled for October 2 – 3 in Vienna. Will I be there? Absolutely. And I’ll be hoping that it leave me as excited about blogging and social software as BlogTalk 2.0 did in July 04.

Good Night, and Good Luck

Tuesday night, Suw and I saw Good Night, and Good Luck at the Uptown Theater in Washington, an old movie palace that opened in 1933. It is one of my favourite places to see a movie anywhere. A couple of years ago, I saw a restored 70-mm print of Lawrence of Arabia there on the huge curved screen. It reminded me of what movies were all about at one time: Grand spectacle.

But Good Night, and Good Luck was almost the opposite of that huge epic blockbuster, a film so understated, so anti-Hollywood that it seemed at times swallowed by that massive screen.

Murrow v McCarthy

The film followed the battle between Edward R. Murrow and the paranoid Communist hunter Senator Joe McCarthy. It started off with Murrow’s speech to the RTNDA in 1958 in which he said that TV was a powerful medium but risked becoming nothing more than a box of lights and wires and then flashed back five years.

There were actually two battles in the film. One between Murrow and McCarthy and the other between CBS head Bill Paley and Murrow. I was impressed with the development of Paley’s character in the film. He wasn’t portrayed as some mindless, corporate goon so focused on profit that he put Murrow in a straitjacket.

When Murrow reminded Paley that he had promised a firewall between corporate and editorial, Paley said that Murrow had to remember the other employees of CBS that he might jeopardise by going after McCarthy.

Maybe it’s just the info-junkie in me, but the film left me wanting to know more about the main characters, more about the history. Possibly that is what a good film or story or blog post does: Stimulate curiosity, leave unanswered questions. If you’ve seen the film, let me know what you think.

‘Crusading journalism’

I found the journalist in me feeling a mix of awe and discomfort at Murrow’s closing commentaries on See It Now. Murrow was an amazing writer and journalist. His writing and delivery were inspiring.

But as Murrow was reminded in the film: “We report the news. We don’t make the news.” It’s often been something that I have struggled with as a journalist. Sure, I have my own views and opinions, but I also believe strongly that my job is to report and let my readers, viewers or listeners to make up their own minds. I’m still uncomfortable with commentary, editorialising.

Murrow didn’t mince his words. He saw something that made him feel very uncomfortable in Joseph McCarthy and his anti-Communist witch hunts. And he tapped into the terror that many felt that their lives could be ruined if they were accused of being Communists.

After the broadcast, Murrow’s team grabbed the newspaper’s to see their reviews. The New York Times called it “crusading journalism at its finest”. It was fearless.

Dysfunctional relationship

Could journalists in the US do this now? I doubt it. It’s not just that the US is so divided. I sensed a trust that Murrow’s audience had in him to tell them the truth, even if it was the truth as Murrow saw it. Murrow challenged his audience doing pieces on not only McCarthy but also the shameful conditions that migrant workers suffered. He took on big tobacco, the situation in the Middle East, just to name a few.

Can the American media challenge its audience without being challenged itself? I don’t think so. Allowing oneself t be challenged takes a strong relationship, and right now we in the media don’t have that kind of relationship with our audiences.

Some people these days say that in this new world of journalism, our job is to stimulate and facilitate a debate. At the Beeb, we call it the global conservation. To play that role, I really want to be able to challenge and be challenged. But that is going to take developing a new relationship with my audience.

This is where blogs come in for me. To do this, I’m going to have to listen as well as talk. Blogs allow us in the media to do that. If only we would.

Don’t need a weatherman

Suw and I are in Washington DC mainly so that I can get my UK visa for the next year, and for me, it’s also a chance to see old friends. I was based in Washington for six and a half years from the Clinton impeachment through to George Bush’s second inauguration last year.

We landed last Thursday evening to warnings of impending doom. A snowstorm was on its way, more precisely a nor’easter. For those of you not steeped in American meteorological lore, a nor’easter is when a storm comes up the east coast of the US. In layman’s terms, the warmer, wetter air from down south slams into the cold air up the coast and voile, lots and lots of snow. DSCN0267.JPG

The last really big one happened in Washington on President’s Day weekend of 2003. It dumped a couple of feet of snow on Washington. My car was buried for a week behind a four-foot wall of snow left by the plows.

Weather weenies

Washington really can’t cope with snow, well, that’s putting it mildly. The town freaks out with even the mere rumour of a threat of inclement weather. I just don’t get it. It’s not like the city doesn’t have wintry weather.

And Washington really doesn’t know how good it’s got it. I grew up west of Chicago, and I have childhood memories of the Blizzard of ’79 when something like four or five feet of snow got dumped on us. My father, who is six feet even, had to go up on the roof in the middle of the storm to shovel off the chest-deep snow to keep it from collapsing.

Unfortunately, he dumped a good chunk of it right by the front door. I was about the only one small enough to squeeze into the front door until sometime in April.

Talking about the weather

Sorry for prattling on about weather. It probably has something to do with my storm chasing days as a cub reporter in western Kansas. Well, that in the fact that talking about the weather was one of the icebreakers I used when interviewing laconic Kansans who viewed me with deep suspicion. As they said, often: “You’re not from around here are ya?”

But weather is a real obsession for people. I almost enrolled my father in a 12-step programme for addiction to the Weather Channel. He had this habit of beginning every phone conversation by telling me the temperature of where I happened to be at that time, whether that was London or Washington. It was useful when planning what to wear for the day, but slightly freaky that my father knew my local forecast better than I did.

Here in Washington, local TV stations had their network of local weather watchers who sent in pictures of how deep the snow was in their back yards. I wonder why newspapers haven’t picked up on this or created spaces for their communities to talk about weather.

Sure, when I go into a weather site I want to know weather, quickly. But I wonder why news and weather sites don’t create more tools, more spaces to bank on this natural talking point.

Oh well, Suw and I survived the Blizzard of ’06. In Washington, it managed to coat the city in a beautiful blanket without really causing that much disruption. Plenty of Flickr pics to follow.

FoWA: Building a web app on a budget – Ryan Carson

What’s the big deal?

You don’t have to be big anymore. Everyone remembers Boo.com. They raised 130m and have nothing to show for it. Now, you don’t need to be anymore and that’s a huge change. It’s changing the entire landscape of the market with a team of 1 or 2.

So why now? Why hasn’t this happened before? Couple of basic reason: broadband is widespread; people are more comfy with web apps like Gmail; hardware is dirt cheap, don’t have to spend 5m on one Sun server like Boo.com did; open source software so there’s no need to go for an expensive .net platform.

What’s enterprise?

Mass market, 1000+ users. Meant to be apps used by 1000s of people. Not necessarily a lot of disk space or bandwidth but should be able to handle a lot of people.

What’s ‘on a budget’?

Under 30k. Minimum you can bring an enterprise web app to market today on. If Carson Systems can afford it, then you can too.

DropSend is used for sending and storing large files. Common problem – can’t email a 10meg file, can’t explain FTP. DropSend is the answer to that. Can store large files online.

It was a problem that Ryan had.

95,000 users in under two months. Five colo servers in San Francisco. Desktop apps for Mac and Windows. Have an API which will be released soon. PHP, AJAX, MySQL.

The most important thing

Make sure your idea is financially viable. use common sense, will people pay for it? Be cautious about your projections. A lot of guesswork. Acquisition my ass – aim for profit from the beginning. Acquisition is not a sure thing. If you don’t know it’s going to happen then you are betting a lot on that and it’s not smart. If it happens, then great. Forget about the whole ‘is this another bubble’ crap.

The budget

Once you’ve decided that your app is financially viable, set a budget. We had a hard time with this. How do we know how much this costs? We’ve never done it before.

So budget depends on project, but I’m going to explain about DropSend and be really honest about figures.

Branding and UI Design: £5,000

Slightly cheap, but did a deal with the designer and got a friend bonus, but our designer went to Bath, lived at our house and did it in a month. If you get UI wrong you’re screwed, but if you get it right then you’re sorted.

Development: £8,500

Good deal, pretty cheap. Offered a small percentage of equity in the product instead of a higher price.

Desktop apps: £2,750

Found someone who did Win and Mac, but next time will find specialists.

XHTML/CSS: £1,600

Really good deal. Guy was amazing, nice semantic code, not telling you who he is.

Hardware: £500

Hosting/Matinenance: £800 per month

Can afford outages. They’re not great, but difference in cost of having an app that’s up most of the time, or up 99.9% of the time is huge, so ‘most’ was fine. The hosting co helped architect the system, put it together, deploy it, and maintain it. Still cheap. That’s for five boxes. If you have one box, you don’t need the same services.

Legals: £2,630

Accounting: £500

VAT, invoices, integration with Quickbooks, etc.

Linux specialist: £500

To set up the dev box, virtual host etc.

Misc: £1,950.

Taking a trip, laptops crapping out etc.

Trademark: £250

If you spend time on your branding then you want to trademark your logo. Simple. Don’t need a lawyer unless you’re worried about a conflict. Do it early, before your finish your branding.

Merchant Account: £200

NatWest said taking recurrent payments was too risky. So had to go with Halifax-Bank of Scotland.

Payment processor: £500

Total: £25,680

Need to make sure that you have cashflow. So Carson Workshops funds DropSend, because it’s hard to work full time on a web app without bringing any income in.

So the reality of this is that it took us a year to save the cash. Carson Systems is 2.5 years old, and for the first year I worked my ass off to save up. If it does take you time, you can spend that time learning, picking up tips, maturing as a company.

How to build your team on a budget

You need good people but you can’t afford them. Couple of ideas – go for rock stars, go for quiet talent. People who are well known are expensive and busy. So find someone that you trust, someone who is not well known but good. They’ll be cheaper.

Offer a percentage of equity (2 – 5%), so if you get acquired you give them a %.

Ask people for recommendations. If you work with someone who’s crap, it’ll cost you money because you’ll have to redo all their work. You can outsource. We tried India, initially. All the dev for DropSend was supposed to be done in India, but it didn’t really work out for us. Some of it was down to my inexperience as a manager.

Scalability on a budget

Crux of the issue when it comes to doing enterprise web apps. You have to be able to support a large amount of users but to start with you have no hardware to do so.

Buy just enough hardware to launch. Basecamp launched with one server. We were originally thinking about getting IBM blades, but it was stupidly expensive. But wait to see if your app is successful before you throw money at it.

Build scalability into the architecture of your software. Hard to predict, but things like running out of diskspace. Can you just plug in a disk and it’s happy? need to not have to unplug for a week.

Plan, but don’t obsess. I obsessed for weeks, dreaming about hosting, it was terrible. Don’t do that, it sucks. Worry when your app succeeds.

How to keep it cheap

First one is don’t spend money unless you have to. Unless you’re sure you need something, do not spend money on it.

No stationery. We wasted £1000. Dumb.

No shiny new machines. I know you want that MacBook Pro. Don’t do it. You don’t need it. I survived for a year on a crappy beige PC with a CRT monitor that took up half the room.

No luxuries. When you have the cash, spend it, but in the meantime bite your lip and survive.

No frou-frou features. We had lots of ideas but the more ideas you have the longer it will take to developer, the more expensive it is. Launch with fewer features, but launch quickly. when you do launch your product is going to be easier to use. Less features is good, in general. Obviously it has to work, it has to do something.

Before you spend £25, ask yourself if you have to.

Make deals. Give a small % away. Barter, e.g. offer free advertising to your suppliers on your blog. Us IM [and VoIP] not phone calls. Do as much as you can yourself. All the wireframing I did myself in Flash. I did all the bookkeeping and copy writing.

Shop around. First quote for hosting was £12,000 a month.

Pessimism has its place

You will go 10% over budget

You will go 3 months over schedule

Solution? Plan on it, put it in to your cashflow.

Holy crap! Lawyers are expensive

Terms of Service £1000

Can’t copy someone else’s service agreement. Need a Terms of Service.

Contract for freelancers, £800

Can’t just run on trust

Privacy Policy, £15

Clickdocs, which are legal docs which are dirt cheap. Can also get a ToS for your website if you want to.

Barter with your lawyers – every lawyer needs website. They always give you a one hour consultation, so always take them up.

Cheap software is your friend.

Basecamp for project management, can get the free plan if you’re only doing one project.

Trac for bug tracker

Skype and AIM for meetings

Subversion for version control.

LAMP or whatever other OS platform you want to use.

Cheap hardware is your pal

£200 Linus box dev server run from the office.

How not to spend money on marketing

Use blogs, word of month, viral. But do it without annoying people. If your app is viral that helps.

Writing. Great way to raise profile. If you have good ideas you can write for magazines. If you write a web app for business, email, say, Business 2.0 and offer to write for them.

What about Venture Capital

If I’m ranting about enterprise web apps and a budget, why wouldn’t you just go with venture capital?

You might need it. I’m not gonna say that everyone should not have VC. If you need to scale or expand very quickly, if you can’t afford to wait that year, then VC is a good answer if you’re willing to give away a huge chunk of your money. But you need a really good reason these days. If you can launch for under £30k, why give away 25 – 30% of your company?

Essentials

– Don’t spend money unless you absolutely have to

– Bring down the cost of services for bartering

– Cut your features so you can build quickly and get it out the door

– Be realistic, if not pessimistic, about your cashflow

– Plan for scalability but don’t obsess

FoWA: Ten reasons why you need to build an API – Shaun Inman

API facilitates requesting, manipulating and exchanging data. Successful API obscures the storage format of the requested data as well as the details of the retrieval process. Don’t want your API to break if you break your storage.

Everyone has an API these days. Who is using APIs? Well, who isn’t?

Websites, widgets, desktops apps, bloggers.

– Increase brand awareness

Lay people don’t care about APIs. But you’re empowering people to do something with your data. People like to talk about things that empower them, so if someone is using your API in a way that’s new, they’ll talk about it, build more buzz.

– Allows people to use their own data

Feel more comfy if you can pull your own data out and take it with you wherever you want.

– Build goodwill with developers

– A perfect excuse for a community

Pulls people together around a common feature central to your app.

– Solving programming problems with an API in mind can improve code quality

Less work, less revisions.

– Simplify internal reuse of data

Present your data in different ways.

– Allows others to extend the functionality of your application

Means people can do things that you weren’t going to do, or didn’t think of.

– Allows alternate input mechanisms

E.g. Ecto or MarsEdit for blogs.

– Unanticipated applications of your data

Sort of like the Grey Album, an unanticipated use of the White Album and the Black Album. Chicago real-time crime maps.

– Turn your program into a platform

FoWA: Happy programming with Ruby on Rails – David Heinemeier Hansson

Silver bullet for developers – motivation. We should be focusing on productive and motivation in our development, in the work we do. Not enough to be just working in an interesting domain, or customers who appreciate what you do, but you also need to enjoy the mundane things of what you do.

I’m most motivated when I’m happy. Motivation comes from happiness. Not far down to say that if happiness, is the key ingredient for motivation we should optimise for happiness to optimise for productivity.

Used to be a programmer who didn’t like programming, liked the end result but not the process of programming. Didn’t find that exciting. But found there was one key factor that me me happy about doing the actual work and for me that’s beautiful code.

Beautiful code is one of those things that’s hard to explain, it’s about feeling good about a single statement, expressing what you want to say in a way that’s aesthetically pleasing, that’s understandable, that’s right.

Your app is not a unique snowflake. It is not special. Hard for people to deal with the fact that they are not special. Most parts of most apps are just like everyone else’s. Most people do the same most of the time.

So Ruby on Rails optimises for the 80% of the workloads. Can’t optimise for special, can abstract special. So that’s not the work that we focus on with RoR, because it’s too hard.

To get people on board with this they have to believe that 80% of what they do is the same as everyone else, and the last 20% is special, is the snowflake.

One technique for making beautiful code is the notion of convention over configuration. Configuration is the annoying part of making software. First you do the work, then you tell the system how you did the work, then you tell another part of the system how you did the work. Programmers don’t like repetition. So beautiful code does not repeat itself. It express what it needs to express just once.

[Gives example of code.]

Look for patterns to abstract, conventions say in mapping. Look for relationships between things. Conventions are there for the 80% of the time. When you work inside those conventions, you don’t need to do work, because you can get the conventions to do it.

Configure once, the move on.

Flexibility is overrated. Assumption is that flexibility is good. Trading valuable parts of your system away to get flexibility, and usually not worth it. Should stop chasing flexibility.

Constrains are liberating. If you think about what most people are doing most of the time, you’ll get systems which are consistent.

Devil shows you the easy way, where something gets done today but tomorrow is tomorrow. On the other hand, the angel tells you to test things.

Not about whether something is possible or impossible, it’s about whether it should be encourage or discouraged. At this point I realised that PHP is the devil, because PHP was constantly giving me temptation to do the ugly thing. The whole language just encouraged me to be a slob. Yes, I could fight that, and that works until the pressure’s on, and then you listen to the devil.

Ruby on Rails is the angels, we’re trying to embed the angel on RoR. We’re making it easier to do the right thing, the clean thing, the pure thing, than the hack. You have to go out of your way to do something ugly.

Create invitations to do better, reminding you that you should be consistent, that you should be creating tests, and that you get benefits from doing things the right way.

We do this by allowing common patterns but showing you there is a better way to deal with these common patterns.

Conventions

Invitations

Opportunities

Expectation

Want to set up a community of expectation that people are interested in doing the right things, the clean things.

[Another code example. Hard to take notes on code without the examples.]

Extracting common usage. Reduce mental effort. Sometimes the beautiful thing is to get out of the way. Sometimes, the beautiful thing is not to try to abstract, but to be a snowflake, do the unique thing.

What do you need to use RoR

– Feel the hurt. If you don’t feel the hurt, don’t hear the devil whispering in your ear, you’re not ready for RoR yet.

– Appreciate agile. That a functional spec is evil. Appreciate tests.

– You can skip the vendor. ‘I’m not here for you’. Open source because you want to help yourself. Rails is solution to problems faced by the contributors.

Does it scale? Yes.

FoWA: Native to a web of data – Tom Coates

Spent the last couple of years working for the BBC, but now working for Yahoo!, but I’ve only been there a couple of months os if I say something controversial then that’s not official policy.

Presentation will be online afterwards.

For some reason I was put down to talk about user interface, and when people think about Web 2.0 and user interface, they think about rounded corners and gradient fills.

Blogger started this entire thing. I could do a whole talk on design topes for Web 2.0, but I’m going to talk about product design at a higher level, what it means to design for the web.

I’m going to talk about what the web is changing into; what you should be building on top of it, and architectural principles.

What is the web changing into?

Web 2.0 is clearly a buzzword, and a conference, but it’s also marketing to see the idea that things are changing, You could argue that it’s a shift and an attempt to get money out of VCs. Buzzwords are an attempt to create order, to make sense of what’s going on amongst some tremendous change.

Tim O’Reilly’s article What is Web 2.0. And Marcus Angermeier’s representation. But there are many difference, in design, infrastructure, server, business models, everything is changing. This idea of Web 2.0 can’t support the no. of ideas going on beneath it. The church is too broad, so I’m going to concentrate on one layer, a web of connected stuff.

Think in terms of APIs and data. We are moving from a world of web pages connected by links, we are now in a world of data connected by APIs.

The web is data sources, services for manipulating, and ways of connecting them together. Conflating content and data together.

Mash-ups, terrible word but interesting idea, shows that things can be cut together

Yahoo! Astronewsology, splices news and star sign, e.g. things that happened to Capricorns, or explore a news story and compare what should have happened to someone and what actually did, which is useful when looking at obituaries.

Can use non-standard data sources and hybridise them in ways that makes each data source better, so you can navigate one in terms of another.

Two things aren’t so interesting, but add a third or fourth, and it gets good, and you get a network effect of services. Every new service you create can build on top of every other service in existence, and every service which adds data to the ecosystem adds value to the other services.

This creates massive creative possibilities.

You get accelerating innovation because people don’t need to invent the same thing twice

increasingly competitive services

Increasingly componentised services

Increasingly specialised services

There is money to be made. Can you se APIs to drive people to your stuff. Worked on a project at the BBC to produce info about programmes, which if the BBC opened it up, people could do interesting things with. Amazon do it well.

Make your work more interesting by providing APIs because other people go and do stuff with it. Can use syndicated content as a platform for other things. Can turn your API into a pay-for service.

If you’re not part of that ecosystem, if you’re not opening up, you end up in a backwater, doing all the work yourself, disconnected from the environment around you, and that’s not going to work anymore.

Choosing what to build.

So if you buy this vision that if you’re part of the ecosystem you’ll be caught up in it and pulled forward, the question then is what kind of things do you want to build?

Ask, what can I build to make the whole web better? How can I add value with the aggregate web? Users are getting benefit from the connected web, and if you can’t add value to that you are nowhere.

Can you open up a dat source? Can you help users put their own data online in reusable ways? Can you help annotate? Can you help organise? Can you get their data into the ecosystem?

Important to data that is architecturally important to people’s lives that companies want to own it – location, identity, calendaring, names spaces etc.

So companies want to own that space. They want to own the data.

Also, exploring and manipulating data, improve it, navigate it, add metadata to it. Navigation is important, with increasing amounts of data being put up online, e.g. can you help people find video amongst all the TV that’s going up online.

Architectural principles

Need to read Matt Biddulph’s The Application of Weblike Data.

In this world, the core components are

– data sources

– std ways of representing data

– ID and URLs

– distribution

– interaction and enhancing data

– rights frameworks and financial

Believe that there is money here and this is going to happen in a solid way over the next 10 years.

1. Add value to the aggregate web.

2. Build for normal users, developers, machines (some of these may be synonymous). Normal users just want to use it, developers want to find hooks, and machines want predicability so makes sure that you’re consistent.

3. Start your design with data and not with pages, data is reusable asset, it’s the core part of the system so must get it right. You can do these processes together, so you need to be able to represent the data properly too. Design the data in weblike ways then manifest it as pages. Want to be comprehensible enough to be explorable and comprehensible for human beings.

4. Identify your first order objects and make them addressable, i.e. the core things that people refer to, e.g. films. Programs, not the time they are broadcast or the series, but the episode. That gives you the maximum options for aggregation and organisations. Events, which only happen once. People. Then make that addressable. Give it a unique and well structured URL so that anyone on the internet can point at it. If you do that then you’re in a good situation already, you’re winning because you’re part of an aggregate web of data because you’re comprehensible by search engine, service aggregator like Technorati, communities such as Digg or Del.icio.us can refer to it. All of these things are connected data resources connected to this data source.

5. Readable, predictable, guessable, hackable URLs. A good URL should be a permanent reference to a resource which must not change, and should represent the thing it’s referring to well, solidly, reliably, for ever. If your first order object has subordinate pasts, then ok to use hierarchy in URLs, but the URL that represents the definitive object should be in one place. Should not be underlying tech, particularly if that tech might change. So get reid of .html or .aspx or whatever, because you might move away from that technology and that would make it break. Should reflect the structure of the data, e.g. events only happen at one event os their structure can have the date in, so long as it’s a one-to-one correlation that’s ok. And there’s a temptation to make things human readable so much so that it breaks the URL, e.g. radio broadcasts shouldn’t have the date in because they are rebroadcast and the date breaks the URL. Let people guess the URL. More predictable and guessable the better. Readable is good, but don’t need to overdo it. Exposing identifiers is not always bad, if it’s the right sort. URLs are powerful, being moved into the UI of the page. Good URLs are beautiful and a mark of design quality. Don’t do bad ones because I’ll cry.

6. Identifiers are not enough. Lots of things that are on the web that aren’t well represented, things that maybe didn’t originate on the web, e.g. people, events, films. You need a way to correlate them, so if you can correlate with other identifier schemes then do so. Films are a good example, there are lots of other urls for specific films. Need to make it easy to talk about the same thing.

7. Build list views and batch manipulation interfaces. Need to find ways for people to navigate data and manipulate it. Three types of page: destination page; lists for navigation, ways to cut through things; manipulation space, interface for batch manipulation of first-order concepts. Manipulation is a really interesting territory. Problem with changing information on the web is that the interface widgets we have aren’t that powerful, so AJAX is trying to fix that problem to give more application-y style widgets to help us manipulate data. Core aspects are not to break the web, so if you’re doing to use AJAX, or Flash, on the destination page is should be to only change that concept on that destination page alone.

8. Parallel data representations. All types of pages can be queried by APIs; destination and list-view pages can be represented by microformats; and then there’s also parallel XML for destination pages and RSS for list-view pages. RSS in this environment is great, the ability to send data from one place to another in the easiest fashion. Delicious does that incredible ways – every page as a parallel RSS representation. So any app can go and get the RSS and use it. How you distribute the data is another matter and there aren’t many ways of doing that, so look at microformats.

9 Make your data as discoverable as possible. If you do 1 – 8 you’ll be doing 9 too.

FoWA: From web site to web application – Cal Henderson

Works at Flickr. Ten things that have been learnt from Flickr than can be applied to other web apps. Ways to move from a web site to an app. Not very geeky. Not going to talk about MySQL and nerd stuff.

Flickr

Is awesome. Photo sharing app. Has tags. Has an API. All these Web 2.0 buzzwords. Obviously there’s been the Web 2.0 conference for a couple of years. Has taken off as a term. Lots of people will be talking about what web 2.0 means whether they say it or not, and trying to explain some of those principles. It’s nothing really, just a name for a tech that’s been around for a while.

Flickr is going to be 2 on Friday. Big party in San Francisco with cake. All welcome to go.

Flickr has 2m users, who are very passionate. Passionate users are important to Flickr, but the developers who built it who are also passionate. People don’t start a business to make money. you do it because you care about what you’re doing. BNy having passionate developers you get passionate users.

have to start from a pov where you care about what you’re building.

With Flickr, there’s a difference between what users want and what they need. People didn’t necessarily want the things that Flickr built, but maybe they needed it. We thought about feature that people needed first and foremost. You don’t listen to what your users say because they will tell you what they think they want. You have to watch what users do and look at their behaviour to understand what they need not what they say they want. If you give them what they need you’ll make them passionate.

Collaboration

First of the ten things is collaboration. At the beginning of Flickr, before Flickr, there was Game Neverending, which was a realtime MMP game, based around a realtime engine. Flickr was built on the same tech – massively multiplayer photosharing.

Very much like a game – social network, friends list, etc. Already a load of photo site, not a new tech. That’s not a big deal. What is a big deal is sharing your photos with your friend. Very bottom up, is social network.

Create network effect, by create incentive to add people to the network, so people add value by encouraging others to sign up.

Collaborative metadata also core. Tags, notes, great, but if i can add tags to others, then that’s collaborative metadata. More uses than just pointing out things you missed.

Aggregation

A lot of web apps where you upload your own data would all be siloed into users, a ghettoised space. So huge collection by data, but instead of showing by user, show by other ways, e.g. latest photos. Loads of interesting slices of the data. So think about the whole bunch of photos as one big blob, can slice by tag, by geolocation, relation to other tags, interestingness etc.

Open APIs

Many apps have open API, but what’s the point? We needed it for our own development, for building in throttling and abuse protection etc., so why not let other people use it? Had a bit impact early on with it.

Whole tonne of value in just pushing out read data, without exposing your system.

Started build sites, then web apps, now web services. We can allow other peopel to build our interfaces, and allows othe rpeople to do really interesting things, stuff we hadn’t thought of, wouldn’t have had time fork, or which sounded insane.

Something really cool is that people built a game called Fastr, multiplayer game with a series of photos and you have to figure out as fast as poss which take the photos came from. It’s really cool, but there’s no reason for usto have built it.

If you don’t provide an API, people will just scrape data, so it makes it easier for you and them.

Clean URLs

Getting very popular in Web 2.0. Core reason is that you don’t need to expose the core workings of your system in your URL. Use mod_rewrite under Apache. So the URL doesn’t have to point to a file name, so just translate from file name to URL using mod_rewrite. Core to Flickr URL.

The URLs don’t reflect the folder structure at all, so none of the URLs map to physical part of the disk. But that’s good because they are human readable and they are guessable. If you can guess the URL to a page, then people can get around fast.

Sometimes we’ll find we’ve removed a link from the nav, though, and never realise because the developers always type in the URL.

URLs can’t change. If they point to one resource now, they have to point to that resource forever. When we started Flickr we picked some really bad URLs, but we can’t change them because people have used those URLs and you can’t break those links.

This was a problem when scaling. When the URL scheme was changed, have to support the old URLs forever, because you can’t have them break for people.

AJAX, been around for a while. Worst name ever. Asynchronous Javascript And XML. Not always about XML or Javascript, because you could write an AJAX-like script in VB if you wanted. But the Asynchronous bit is important.

After you’ve loaded a Flickr page and you want to add a tag, the box appears without the page reloading, and when you submit it the page doesn’t need to reload either.

Need strong API so that it can be accessed via AJAX. It’s used primarily to streamline interactions that you already have. E.g. keeping the tags on the same page instead of having to go to a new page to add them in. AJAX saves a bit of bandwidth, stops page reloading, don’t lose context etc.

Also use it for creating new experiences. Not seeing this so much on the web yet, but we build self-contained apps in AJAX which have no way to address in to them.

Unicode

Internationalisation comes first, and localisation later. Important thing to consider is whether you want basic international support from the beginning. E.g. do you store all your data in unicode?

Usual use is to store, present and receive data in UTF-8. Some apps use UTF-16.

Desktop integration

Or platform integration, because your platform isn’t necessarily a desktop. Need to move interactions out of the browser. Most of this is grounded in APIs.

Bunch of desktop apps which work with Flickr, e.g. the upload app. Some tasks are crappy on the web, especially uploading photos, and especially more than one photo. Desktop app allows people to perform tasks that would be difficult, suck or just be stupid, in a browser.

Not just desktop apps. Also possible to do browser apps, such as bookmarklets.

More complex platform integration, e.g. toolbars or dialogues into the browsers.

Also integrating the app with email. Everyone has email, and email is integral to the way people interact online, but we don’t often think about building it in to a web app, e.g. to send email to our system as well as receive it. Difficult to get a photo off a mobile phone, but most can email, so as soon as Flickr could accept photos via email then that’s an uploader for every phone.

Also, simple notification emails to draw people back to the system.

Mobile

Will become the most important platform. Heard this for the last five years. You probably remember WAP and how influential that was… but there are some standards so you can build stuff, and so some people will use them. But they usually all support XHTML Mobile, quite a sensible standard.

Build small apps for mobile devices using XHTML Mobile will work on most phones.

People used to think that it was about creating a smaller logo, but that just doesn’t work. Can’t have more than a couple of lines of text and expect anyone to interact with it. It’s not just a case of re-presenting the same content using a different format. Need to think about what you’re presenting. Needs to be snappy. Think about stuff you wouldn’t want to do on the website, maybe. But not all data is suitable for mobile devices. Need to think in small chunks.

Open data

Import and export of data from systems we build. Export through RSS feeds, but can only get 10, 15 or 20 most recent photos. But doesn’t give a mechanism to get a few hundred out of Flickr. Provide a method to get all their data out, or to import everything in from another app.

If you give people the feeling that they can leave at any time, they are more likely to stay.

It’s not just primary data that’s important. People may already have a photo back up because they upload it from somewhere. But they don’t have the metadata, so what’s important are the notes, tags, comments etc. and getting some sort of local back up. That’s done through the API.

There are 3rd party services that back up your photos to a DVD and then sell you the DVD. Makes people feel safer.

Open Content

Previous to Flickr, a lot of web apps that stored data, once you uploaded your data, they own it. Still the case with Google Video, and other photos sites – they can sell it, use it, do whatever with it because they own it. Flickr does not own their user’s data. Allow users to use Creative Commons licences if they wish.

Through the open APIs, open data, open content, and allowing people to reuse and remix and use the data for interesting purposes. There’s a video someone did with CC licensed photos, blog.flickr.com, and wrote a song about it.

All photos used in presentation were CC’d, and attributed in presentation, which is online at http://iamcal.com/talks

FoWA: Things We’ve Learned – Joshua Schachter

Browsers

Always people who some weird browser. CSS will drive you nuts. Header issues. Firefox extensions. Every broken interaction will hit your database and slow it down. It’ll drive you nuts. Need to get your browser shit fixed up straight off.

Scaling

Don’t do it. Whatever you think you’re doing to need to deal with is not going to be the problem. Read Cal Henderson and Brad Fitzpatricks’ presentations on how to make things go faster. These things talk about, not scaling but how to deal with databases. How to deal with SQL not being great. Think about splitting data over multiple machines. Things like one webserver talking to 8 dbs will blow up in your face. Test every single SQL query.

Set up a monitoring system because you’ll get paged at 2 am. No good being asleep if your db goes down because it’ll stay down til you wake up.

Understand your db. Lots of apps have tags, but that doesn’t map to SQL at all, so understand tricks and tips. Have different tables so you don’t have partial indexing.

Cache.

Understand where latency is ok. Figure out where you can be sloppy, and be sloppy, e.g. RSS feeds don’t need to be instant.

Idiots will break stuff in ways you can’t yet imagine. You can’t predict how what they will do.

Apache

Do stuff with Apache. Images off a different server, RSS. Throttle.

APIs

Were created early on in Del.icio.us. Helps the ‘priesthood’ get a measure of comfort in your system. They want to take their data and go home if you go offline. Make the API easier to get into and out of, the more it will be used. Make it simple. Del.icio.us is just XML.

Identifiers

Unique IDs in your db is a mistake for scaling. But do not expose that ID to the outside world, because some idiot is going to use that to try to scrape your database. That’s a hint that people want the data, but they aren’t going to wait for you to give it to them, so they will hammer your systems to get it.

Features

Significantly influence behaviours. Also, which ones do you leave out? If no tags, Del.icio.us wouldn’t have worked. Try not to add features that exist elsewhere. No need to come up with a new way to do things that are already being done, so don’t use messaging, because we have email already.

When people ask for stuff, that’s important info. Try to understand why they are asking for that. Extract the use case. E.g. people want boolean search on the tags, but hardly anyone searches on more than one tag at at time.

RSS

Important in Del.icio.us. It’s another set of tools to get in and out of the system. Everything that could have a feed should have a feed. This was more important a year or two ago, when RSS was the big thing. Now it’s not quite so exciting, and maybe there’s another feature that will cause people to show up.

Need to understand the headers, caching, etc. If you can stash the timestamp then this can save an enormous amount of effort.

URLs

Primary vehicle for people to to find your stuff. Don’t do session keys. Leave underlying framework out. It doesn’t help the user.

Also allows you to expose some functionality to the priesthood who care.

Surprises

Watch for interesting behaviour in the system. If people do things you didn’t expect or intend, that’s neat.

Passion

Choose a problem that you yourself have. Had a text file with 20,000 links in it. Couldn’t find stuff anymore. Had a URL, space, then descriptive text, e.g. wifi. That was the first tagging system.

Build a db system which was like Del.icio.us which was single user, for him. Used that for a few years, then built for other users. Del.icio.us came out end 2003.

Don’t look for problems you don’t have, because someone else who is passionate about it will solve it better than you can.

Release

See teasers. Limited betas. All horrible. Every day you don’t have something out in the world you are losing information, feedback, users, reputations. Get it out there, get the release done asap.

Attention

Anything like RSS has some element of attention. This is useful, interesting behaviour. We do the ‘what’s been popular in the last 24 hours’, and people pay attention to that. That works reasonably well when the population is small and all biased in the same direction, so that things the group finds interesting the whole group will find interesting.

As the group gets larger, the bias will drift. But you can aggregate with a given tag and then can compensate for the decrease in bias.

Figure out how to keep things on topic or fragment into different piles of attention.

Spam

Is attention theft. Spag – tag spam. But people will try to get into any aggregation of attention.

We don’t do a top ten, because it’s biased in favour of stuff that’s been around forever. And, as soon as you do, people will try to get to the top of that list.

Understand what you’re’ building as you build it to avoid these things.

Don’t provide feedback when someone spams, and you figure out the fix. Don’t give them an error message because then they know they’re doing it wrong and will try again.

Tagging

Is not really about classification or organisation, it’s user interface. It’s a way to store your working state or context. Useful for recall. Ok for discovery because someone might tag similarly to you. Bad for distribution.

Not all metadata is tags. People ask for automatic metadata, but that’s not the value – the value is attention, that you saw it and decided that it was important enough to tag. Auto-tagging doesn’t help you do what you’re trying to do.

If you make it too easy… because there’s a small transaction cost then that adds value. But don’t make them do too much work.

Beware librarians who want an official list of tags.

Motivation

Why are people there? Have to expect the user to be selfish. But build a system people like and this breeds evangelism.

Effort

Watch what you find your time doing. If you spend a lot of time building a feature that no one uses then that’s a waste. Be careful.

Measurement

Guesswork backed by numbers. If someone is using a feature in week one, do they also use it in week five. Measure the system itself. But also understand that in the data that the system collects, measure behaviour rather than claims. In Del.icio.us there’s no stars, because why would you bookmark something that’s bad? So rely on what people are doing not what they are claiming to the system.

Testing

User acceptance testing. Very important that everyone on the team sees the user testing. Don’t give people a list of To Dos and then watch them do that because they have a vastly different behaviour than what they do in real life. People don’t read, don’t follow instructions… people know this intuitively, but it’s far worse than you realise.

Language

Speak the user’s language. Del.icio.us is about bookmarking. Bookmarking was a Netscape or Firefox thing. In IE it’s favourites. So make sure that you speak your user’s language or they’ll leave.

Registration

Don’t make users register before they get into your site. Give them as much functionality as possible, even give them anonymous access to start with. Logging in is a big barrier. People want a good idea of what they are going to get if they register because it’s a lot of work. People are afraid of giving out email addresses, spyware, etc. They want to know what they are going to get out of it. You can’t tell them, you have to show them. Let them wander round and get a feel for it. That’s the only way to get them into the system.

Present the appropriate gestures and verbs, save, copy, bookmark, etc.

Then when you have to register, make it as short as possible, and take them back to where they were. Don’t dump them on the front page again.

Design Grammar

If you’re doing something different, then understand where you’re breaking away from normal behaviour. Design should be standard – tabs on top, nav, logo top left. What is the structure of the world? Emulate that as far as possible, but be careful of what your design implies, what it makes people expect.

Morals

It’s the user’s data, not your data. You get to use it, but it’s still their data. They have to be able to add, modify, delete their data. Up to and including removing themselves and their account from their system.

When Del.icio.us deletes a bookmark, the data’s purged from the system. Once it’s gone it’s gone.

Infection

Spent nothing on promoting Del.icio.us, ever. To do this, enable evangelism, so people want other in your system, want to tell people. So enable every communication method possible.

Email is tricky because you don’t want to be a spammer. But RSS. Think of every RSS read as a client app. Viral vectors. Desktop apps can eat a lot of data through http, so figure every app out, and see if you can get in there.

Communities

There’s not ‘quite’ a Del.icio.us community, because Del.icio.us is a tool, and the communities are elsewhere. The communities can use the tool, but no need for Del.icio.us to have a community. There are flame wars and all that, interactions that you don’t want people to have.

The Future of Web Apps

So I’m at the Future of Web Apps conference today, sat in Kensington Town Hall with 800 real proper geeks, hearing about how to develop, er, well, web apps. I’m sure at some point during the day someone will give a deeply techie presentation which will go right over my head, so notes may be patchy. Note are also going to be up on the wiki, but I’ll live blog too.