What blocks news organisations from innovating: The Innovator’s Dilemma and internal boundaries that become barriers

In preparing for a talk to master’s students at the University of Central Lancashire this week, I will discuss my motivations for the research I did during my master’s in innovation management and leadership. I wanted to understand why, despite perceiving the disruptive possibilities in digital media, traditional news organisations failed to adapt. For years, I had subscribed to Clay Christensen’s disruption theory that he outlined in The Innovator’s Dilemma. He pointed out that incumbent leaders in many industries have failed not because they didn’t recognise new technologies but because they “failed to value them correctly”. The ROI seems too low. It was the classic trading print dollars for digital pennies argument. However, disruptive innovators continue to experiment and move up the value chain.

The common argument has been that the “original sin” of digital journalism was giving away our content for free, something we never did before. However, this ignores the economics of newspapers, especially in the US, where I come from. When I was studying journalism at university, our professors told us that 80% of newspaper revenue came from advertising. The 20% of revenue from subscriptions, didn’t pay our salaries, it only paid for paper, printing and distribution. Advertising was our main revenue stream, and we failed to understand how digital media would disrupt those value networks. Digital news organisations did pivot to deal with the threat from digital classifieds with vertical-focused businesses around cars - Cars.com in the US and The Guardian’s AutoTrader - or real estate. However, the development of entirely new ad models, ads targeted by search intent or based on activity on social networks, have radically remade ad markets. It was a classic example of disruptive innovation in which upstarts found the right application and then entered the higher-value markets. For years, Google and Meta have been growing by cannibalising ad revenue from non-digital markets, print and broadcast.

For publishers operating at a certain scale, their response was to grow even bigger. This drove M&A activity as publishers tried to scale their audience to compete against Google and Facebook It got to such a point that a major UK publisher, Trinity-Mirror, even rebranded itself as Reach. This strategy has not succeeded for several reasons:

  1. For publishers like Gannett, their acquisitions left them with unsustainable levels of debt.

  2. Reporting is an expensive business, and especially for publishers with local news properties, scaling is much more expensive than for a digital business like Google.

  3. Scaling the business didn’t fundamentally answer the issue of non-competitive ad products.

When media folks speak about the duopoly of Google and Meta, it is a rather superficial analysis of how these companies changed the value networks of advertising. As Clay Christensen pointed out, upstarts experiment by trial and error to find a model that eventually scales. John Battell covers the journey that Google took to find its business model in his seminal book on the history of the search giant. He called Google the “database of intentions”, and they have used those intentions to remake marketing and advertising. It is so clear how Google and Meta disrupted the value networks of traditional media when you hear about how Temu and Shein spent billions of dollars to advertise with the two companies last year.

After developing disruption theory, Clay Christensen went on to create a process in which companies can uncover the unmet needs of their customers to respond to disruptive innovation: jobs to be done. In a Harvard Business Review article describing how focusing on jobs to be done could achieve this, he wrote:

  • Most managers have based decisions on quantitative-data-based correlations

  • We need to know what the customer is trying to accomplish. The customer’s jobs to be done.

  • “When we buy a product, we essentially “hire” it to help us do a job.”

  • “(D)isruption theory doesn’t tell you how to create products and services that customers want to buy. Jobs-to-be-done theory does.“

As I wrote recently, Seth Lewis, Alf Hermida and Samantha Lorenzo wrote about how the jobs-to-be-done (JTBD) framework could be applied to local journalism. It is a paper well worth reading. This is the kind of audience research that is so desperately needed (and frankly, I’d like to do).

Internal boundaries, barriers to innovation and burn out

However, in my master’s research, I focused on another barrier to innovation: Internal boundaries and the boundary-spanning work of product managers. As David Skok said in the report with Clay Christensen about innovation in journalism, Be the Disruptor:

Our traditional newsroom culture taken in aggregate has blinded us from moving beyond our walls of editorial independence to recognize that without sales and marketing, strategy, leadership and, first and foremost, revenues, there is no editorial independence left to root for.

Product managers operate across these internal boundaries of editorial, commercial and technical operations, but that boundary-spanning work has its own set of challenges. Of the 17 product managers and product-oriented managers I interviewed, five had left positions or the industry entirely. My research looked at what had affected their sense of professional well-being so profoundly. Also, of the five, four were women.

This is a scene setter, and I’ll be writing more about this in the coming weeks and months.

Now onto the links from the past week. INMA has a huge range of pieces from their recent subscription summit in New York. This piece has good details about how Hearst is using data to optimise its pricing. The data has allowed them to identify and target in-market audiences to drive subs, and they are also creating some interesting products that allow them to connect with subscribers at multiple price points.

And the piece has details on how the Irish Times has leaned into newsletters, push notifications and events to keep their connections with readers.

Following the non-profit news organisation trend in the US, a local news publisher in Guildford has just become the first outlet to attain charitable status.

Here is another data point of how the platforms remade the value networks of media. Facebook Marketplace has become the go-to place for young users to buy things. This would have been the newspaper classifieds years ago. Gen Z may not use Facebook to post social updates - they have TikTok and Instagram for that - but they still use the venerable network to find cheap stuff.

AI revolution continues to roll through the media

As one commentator put it, last year media experimented with AI, and now they are starting to deploy it. And they can’t move fast enough as developments in the field are announced at a furious pace.

LMA is partnering with AI companies to help its members adopt these technologies quickly, with applications across editorial and commercial operations.

A thoughtful piece in the Press Gazette explaining to publishers what hill they should choose to die on when it comes to LLMs scraping their content. Archival content isn’t of as much value as their current news content. It is one of those harsh truths that news, by definition, has a short shelf life. However, AI can’t do reporting. Most parts of that will still require journalists, and that is what news organisations should be defending.

Google’s Search Generative Experience promises to deliver information that people want without the need to go off to a site to find that out. It is one of the things that is keeping publishers up at night, and now AdWeek has put a dollar figure on exactly how much Google’s SGE may cost media. Ouch.

Gen Z is using TikTok to search, not Google, but the search giant and others are fighting to remain relevant by introducing AI into their products. The disruptors are being disrupted.

Thomson Reuters shows that some of the major media players are looking to invest in AI. It is one way that well-heeled media players can try to cash in on the technology.

Lessons from succumbing to the perverse incentives of a rented audience

This was a week in which all of the harsh lessons and dangers for media of relying on rented audiences were on display. For one, Facebook completed its divorce from news media and told Australian news outlets that it would stop making payments to them. Facebook also announced that it was retiring its news tab in Australia and the US and would soon be doing so in other countries as well.

Alan Soon of Splice Media and Adam Tinworth have both said that it was time for media to move on, and Social Media Today gave a reason for media to do so with confidence. Research from “media insights platform Memo” found “no direct link between how much engagement a post gets in social apps and how many people then read it”. People aren’t reading the article but merely reacting to the headline, Social Media Today goes on to say. No journalist or journalism business wants that result.

And all the data shows that referrals have plummeted from social media over the last few years. Through my work over the past decade, I have seen the data of hundreds of publishers, and while Facebook used to drive significant amounts of traffic, it has been declining for years. Social Media Today says that there is still value in brand awareness, but research has shown that correct attribution is much lower for visitors from search and social than it is from direct traffic.

More than that, the research in Social Media Today found that negative content received more engagement than positive or neutral content. Outrage on social media drives more engagement than positive or neutral content. While it will surprise no one, it still underscores the perverse incentives that have operated on social media that have damaged not only media businesses but our societies and democracies.

Of course, social media still has a place in audience development. As Adam Tinworth says, we are moving to a platform+ era in which platforms and rented audiences play a role, but the focus must be on the KPIs that favour converting the relationships developed on social media into direct relationships. In my previous role, we used organic as well as paid social media to drive newsletter subscriptions. I’ll be honest, the newsletter we got from Facebook weren’t as engaged as those we got directly from our marketing or on our properties. And membership and subscription services like Poool are creating opportunities to convert social media users into known, registered users.

And on Bluesky, this was shared from Josh Marshall of Talking Points Memo. It showed the collapse of programmatic ad revenue for TPM over the past eight years. “As I think is pretty clear, if this is your business, you’re dead. You don’t have a business,” he wrote. The scale model of digital media was all about using social media to build huge audiences that could then be monetized through programmatic. I even remember hearing media executives talk about how they would make newsrooms pay for themselves through traffic and programmatic ads. It didn’t play out.

And that brings us to the current sad state of affairs for the scale players. This is how it played out. For groups in the US, many of them took on unsustainable amounts of debt in their pursuit of scale as they bought up more and more properties. They were forced to make cuts, in both the newsrooms and in their ad sales staffs, which meant that they struggled with direct ad sales. The newsroom cuts ran deep, and they made tepid efforts at reader revenue experiments with little conviction. The experiments usually failed because often they came at a point when the product was so gutted that it didn’t attract enough takers to offset the loss of advertising. The paywalls came down and the ad loads went up. (My wife used to say that she knew when she was on a news website because the fans on her laptop would spin up so high that it sounded like it was ready to take off.) Invasive ads made the user experience horrible, driving down traffic and yields even further. In a slow-motion car crash that played out over years, social traffic collapsed, pushing ad revenue down even further. It is sad. Poor user experience and poor products have turned off users, and after endless rounds of cuts, the content isn’t local enough to serve communities or good enough to convince people to pay.

That brings us to where we’re at now. And now, as we see from the latest print circulation figures in the UK, newspapers will have to develop digital revenue streams. There is no other option available.

Here are some steps to start that journey:

  • The first step should be a range of tactics to convert unknown audiences to known audiences, and they need to do this with all urgency. It has shown such promise for so many publishers and has so many benefits.

  • They also need to get closer to audiences. In the past few years, I have developed such an appreciation for qualitative research. I have been a ‘numbers guy’ for most of my life, and quantitative data is a great way to measure what your audiences are doing, but qualitative data tells you why they are doing it.

  • Use all of that rich first-party data to improve all of your operations - product, revenue and marketing.

We have models of how to make this work at almost any scale, and it’s sad to see how much damage is being done to journalism and media brands by doubling down on a strategy that has not worked for years - chasing scale via rented audiences.

The chains in the US have turned to try to capture some of the philanthropy cash that is now flowing to communities, but look at this story. Report for America says that they won’t put their reporters in hedge fund-owned publications.

Google is paying some publishers to test an AI product on their content. As publishers develop their strategic guidance on AI, they will need to define their terms of engagement with AI players just as they should have with social platforms.

I am sad for all of the journalists, photographers, ad staff and others who have lost their jobs over the last 20 years in the US and UK, where I have predominantly worked. But I do see green shoots of growth now as small start-ups launch with the MVP of the day, a newsletter, and then build out from there. They are far from replacing the reporting capacity that once existed, but I do hope that it is clear that it is time to close this disastrous chapter in journalism and move on.

Now to the round-up for this week. Like other major responsible news publishers, the BBC has announced their well thought through plan on how to use generative AI. The announcement looked at experiments in three areas:

  1. Maximising the value of existing content

  2. Reaching new audiences

  3. Improving processes to make them more efficient

Meanwhile, Mattie Peretti who started out during an ICFJ Knight Fellowship to find out how AI could be used to help news organisations serve their communities better. After four weeks, he found the problem statement was wrong, and he says: “we can’t make our industry more sustainable without radical new solutions and creating products that users actually want. The role AI might play in creating them is somewhat irrelevant.” Amen and read on.

A blockbuster piece from the New York Times looking at how Google and Meta have benefited from a Chinese e-commerce spending spree to crack the US market.

How the media lost the future, and how we might regain it

My first glimpse of the future of media came in a student computer lab at my dorm at the University of Illinois at Champaign-Urbana in August of 1993. (Yeah, I’m that old.) My friends were buzzing about a new app that was in beta, something called a web browser, Mosaic, which had been developed by Eric Bina and Marc Andreessen at the National Center for Supercomputing Applications on campus. Before Mosaic, I couldn’t imagine my parents ever using the internet. It was just too technically complicated, but Mosaic made the internet visual and accessible. As a journalism student close to graduation, I know that it would change my career, and it did in so many ways I never anticipated.

In 1996, I had my first proper digital journalism as a digital news editor at WWMT - a local TV station in Kalamazoo Michigan. The next year, I moved across the state to work as a special projects producer at Advance Local’s MLive. The next year - 1998 - I became the first digital journalist for the BBC outside of the UK, working in their flagship bureau in Washington DC.

But I was not the first wave of digital innovators by a long chalk. My friend Steve Yelvington was the founding editor of Star Tribune Online and has written about pre-internet online efforts by newspapers. He was building that service as I was exploring the web with Mosaic. And there was Roger Fidler, who I knew about by reputation but never had the honour of meeting. He envisioned a future of “tablets and e-readers” in 1981 and spent the early 90s trying to build that tablet at the Knight-Ridder Information Design Lab! This is to say that plenty of visionaries were already working towards a future of media before I had even left university.

Now 30 years later, the New York Times interviewed Fidler in “How the Media Industry Keeps Losing the Future”. "After decades of decline, their collapse seems to be accelerating,” writes technology reporter David Streitfeld, adding how Fidler “helped develop technology for lightweight tablets that would use flat-panel displays that were low cost but clear and bright with a relatively long battery life”.

What went wrong?

I was too narrowly focused. I didn’t consider all the possible cross impacts of emerging technologies that would lead to Craigslist, alternative news sites, social media and other products that would greatly diminish newspaper circulation and advertising revenue.

I was too narrowly focused as well. I thought the cost savings of digital distribution would open up a new era for journalism. However, if you save money but can’t earn it, it doesn’t matter. Your business will still fail. I started thinking about digital revenue at MLive, but then I went to the BBC. We had the luxury as a public media outlet to produce incredible, ground-breaking digital journalism without having to think about a business model. When I worked at The Guardian (2006-2010), it had an almost anti-commercial culture. I did not turn my attention back to thinking about revenue until I joined Gannett in 2014, and by that time, it was too late.

How we might regain the future

During the pandemic, I finally fulfilled a promise to myself and got my master’s degree in innovation, management and leadership. I try to apply what I learned to help media companies as the consulting director at Pugpig so that publishers, their reporters, product managers and the technical and commercial staff have a brighter future.

One way publishers need to adapt is to consider their marketing orientation, which is the process that a business engages in to identify and satisfy the needs of its customers. First, let’s discuss who the customer is. In As a journalist, we always thought our customers were our audiences, but when I started working in the industry in the US, 80% of our revenue came from advertisers. People seem to think that delivering eyeballs to advertisers is a recent development of the digital age that sullied the noble profession. When I was in journalism school, my professors were honest and said that subscription revenue paid for the cost of paper, ink, presses, and distribution but not our salaries.

I’m going to reference this incredible graph from 2016 by Thomas Baekdal. It tells the story quite clearly about what happened in the US.

Search and social media advertising became a much more effective way to reach audiences than newspaper ads. What would have happened had we poured as much innovation effort into the commercial side of media as we did into the editorial side? It was done in fits and starts. Gannett owns a digital media marketing company. The Dallas Morning News bought up several local digital marketing companies in the middle of the last decade. So much more should be done in terms of digital publishing commercial innovation.

However, with the major focus now on reader revenue, our readers are our customers, which brings me back to the concept of marketing orientation, I am going to focus on three: sales, product and market orientation.

  • A sales orientation focuses its energy on selling its product to its target audience. “In a way, it does prioritise its customers but not in a sense of listening to their needs and wants – it simply wants to sell to them,” according to Orientation Marketing.

  • A product orientation focuses on continually improving its products to deliver the highest quality product possible. “Premium products fall into this category, but the approach does not always offer what its target audience actually wants or considers the factors that the audience uses to form its purchasing decision,” Orientation Marketing says. The benchmark is competitors.

  • A market orientation considers the target audience before any product is created. Audience needs are taken into account. “Market orientation, in marketing strategy terms, commonly revolves around culture, values and other internal behaviours focused on satisfying customer needs that are usually well-researched prior,” Orientation Marketing adds.

The marketing orientations all have their pros and cons. A sales orientation can be effective when you have proven the product-market fit and have a relatively stable market environment. However, that isn’t the environment that newspapers have been operating in for decades. How long did we cling to a sales orientation? How long did we simply focus on selling what we had always done without listening to our audiences? Too long.

And how many times have I heard a product orientation from news leaders who thought that quality would always cut through? Reflexively saying that “content is king” has too often been used simply as a thought-terminating cliche. It rallies the troops. But what content? In which format? Delivered in what way? Recently, there was a discussion in an industry Slack in which an exasperated product manager asked if a podcast could gain an audience simply on quality alone. It was an assertion made by a producer where she worked. What arrogance! Such self-importance! It’s as if the audience is an afterthought and the only thing that is required is to produce something that passes exacting quality control based on journalism’s own professional standards.

For product managers who use tools like design thinking or jobs to be done, we have a market orientation. We ask who is the audience for this news product. We think of the audience in granular terms and consider their needs. We actively seek out quantitative and qualitative data, and we understand the variety of news and information needs that exist in our audiences. A good example is Schibsted, which sent three qualitative researchers on a road trip for a week to find out if people outside of the two main cities in Norway - Oslo and Bergen - had similar media habits. They have balanced editorial and algorithmic curation of their homepages based on a range of criteria. They understand that providing a homepage for the “average user” would present news that appeals to a white man in their 50s, and they want their homepage to meet the needs of the range of users and engagement levels that they know they have.

And we need to break down our internal silos so that editorial, commercial and technical can think broadly about how to solve the existential crisis facing journalism. We need this kind of collaboration to create products and revenue models that will pay living wages for the journalists, editors, sales staff, developers and other staff. ader revenue, a

And now onto things that caught my eye this week.

I start with a fascinating case study from Romania. It highlights how newsletters have become the MVP for media, and it is a rare examination of the revenue sources involved in local media start-ups, a mix of reader revenue, ads and grants. The case study also highlights how scrappy journalists have had to be to make a go of it over the last decade or so.

Three young leaders shared advice for meeting challenges including imposter syndrome and having to remake a media brand to help it move upmarket. I particularly enjoyed Aliya Itzkowitz of FT Strategies view on the value of voicing uncertainty. When you’re dealing with innovation, you have to deal with uncertainty and find a way to systematically work your way through it.

My former BBC colleague Alf Hermida has just released a paper with Seth Lewis and Samantha Lorenzo on Clay Christensen’s jobs-to-be-done framework and how it can be applied to improving the products of local journalism. This was a popular framework in the first decade of this century, and there were a lot of advocates of it, including Steve Yelvington who I mentioned before.

The FT seems to be going from strength-to-strength, and now it is prospecting for new opportunities with its own venture fund.

In AI news this week, OpenAI alleged that the New York Times hired someone to manipulate its systems to make it appear that it frequently plagiarised the newspaper’s material. It was a forceful response to the Times’ lawsuit.

Having worked in US public media for four years, this makes me sad. It was one of the early efforts by public media to move beyond its traditional audio and video content to provide local news in a digital-first way. It speaks to the challenges facing US public media in this soft ad market.