In the Beginning

When we started supporting partners doing narrative strategy work, almost 10 years ago, we tested so many social listening tools. All the tools we tested provided characterizations of the total supply of information: all the news articles, all the tweets, all the social media posts.

We couldn’t find a single tool that gave us what we needed: an understanding of the actual media individual people were choosing to interact with. We wanted to get a sense of the demand side of media, not just its total supply. We wanted to know not just that there was a lot of “disinformation” or “low-quality content” circulating, which was a worry at the time, but how much of that content people actually interacted with. Who precisely was exposed? Who was engaging with what, when? Who was down the rabbit hole? How was it affecting them?

Only by answering questions like these could we deliver partners an accurate diagnosis of the problem. Only with an accurate diagnosis could partners develop targeted intervention pathways that worked at the level of audiences and people, in other words, that didn’t require altering the total pH of the ocean of content that is the internet.

For this purpose we built the Narrative Observatory, data infrastructure that knits together respondent-level behavioral media data with media content data. For the last six years, we’ve used it to support dozens of partners across dozens of issues in audience and narrative strategy development and execution. We’ve learned a lot about what people need, what’s useful, what works. And in that time, audience-first narrative strategy has become the only possible way to contend with a more and more fragmented, contentious media landscape.

The Need Now

We’ve accumulated so much data from the responses and behaviors of real people that we struggle with how to read it all—how to put it to use to create new knowledge in ways that avoid duplicative research studies, how to bring it to life to enable communications professionals to inhabit audiences’ realities. Many partners struggle with analogous problems: how to discover and make sense of all the research out there, all the findings and frameworks, especially when six-figure research budgets and months to explore and learn are out of reach . . . when that rapid-response campaign needs to have gone live yesterday.

A First Experiment: The AI Focus Group

One way to answer this need is by using AI to empanel a focus group drawn from the terabytes of data we’ve accumulated. The focus group is a format of interaction and intelligence gathering most communications professionals are already familiar with. And because all our quantitative and qualitative data are associated with values profiles and demographics, it’s possible to use them to synthesize personas, which can be queried individually or in groups.

Focus groups offer a lot of depth on not much breadth, and surveys offer a lot of breadth without the opportunity for interaction and going deep. Might a focus group synthesized from real survey data better balance breadth, depth, and interactivity?

We were eager to find out, so we started experimenting. Here are the results of one experiment using data from the five national surveys Harmony Labs conducted last year with our Deep Story Survey System, with more than 10,000 respondents, 300 quantitative questions, and 11 open-ended questions.

We crafted questions to give us both big-picture information (e.g., “who benefits from the paid and unpaid work you do in life”) and more specific information (e.g., “what should the government do to solve big problems?”). We processed the data to balance specific, actionable examples of responses with more general patterns extracted across surveys to create audience profiles. Each persona had access to their own real responses, and also to pools of responses provided by people like them.

Our goal was to use AI to give voice to the responses of real people. We were looking for responses with reliably high fidelity to the real data collected from real humans, and that were useful to communications professionals in forming hypotheses about how to reach different audiences. So using real data and identity protection statistics, we empanelled 24 personas, 6 per values quadrant in the Narrative Observatory values-based audience segmentation:

The Focus Group in Action

One of the reasons a focus group is such a powerful format for communications professionals is that it allows them to get a sense of who people are before asking about specific ideas. We asked the personas to tell us about themselves, and their answers tell us a lot about differences between audiences:

“Tell me something you're proud of accomplishing in your life.”

“Tell me about a setback you've experienced.”

Every response from the focus group is delivered with the specific evidence that drives it from Harmony’s real data. The constellation of evidence for Don’t Tread on Me’s explanation of poverty centers on individual drive, while Tough Cookies’ is based on underinvestment in family support.

“Poverty in the United States is much higher than in many other industrialized nations. Why do you think that's true? What would you say is the #1 cause of people falling into poverty?”

In our focus group “conversation,” we could see more specificity and divergence within audience groups than a traditional audience segmentation affords. In these two responses below, from personas in the If You Say So audience, two important themes emerge: Carmen talks about wanting both personal agency and protection from catastrophe; and Arthur talks about a rigged system founded in wealth inequality.

“We all know people want the freedom to work and make their way as they choose. But it sounds like everyone here agrees that sometimes bad things happen. It's hard to find a job. You get disabled. You need to care for kids and parents. What should change so that setbacks aren't the end of the road?”

The focus group also affords pressure-testing messaging across audiences. We asked a series of questions to try to create some message guidance for a universal public healthcare option. While our data don’t include this policy specifically, we have extensive information about whom audiences trust, how they want government programs to feel, and what they want from life—so we were able to extrapolate to provide within-audience messaging and cross-audience guardrails, like:

What We’ll Build Next

Our partners detect a lot of promise in this experiment. They are excited to use this tool, and others like it, for example, message and story generators derived from the media diets of different audiences. So we would like to turn our team’s focus and creativity toward building a new set of AI-powered audience intelligence tools on top of all the Narrative Observatory’s accumulated data. Tools that:

• Participate in the rigor and accuracy of traditional research studies, but are available to many more partners at a fraction of the cost
• Bring audiences to life in ways traditional research reporting cannot
• Respond to changes in media culture and scale internationally
• Support iteration and learning across our partners, helping them develop the new comms muscles needed to navigate and succeed in a fragmented, contested media environment

Just to be clear. We see these tools as access points to, rather than replacements for, the kind of deep, data-driven audience and narrative strategy research we pioneered with our partners. Understanding the total ecosystem of audience and narrative around an issue, making strategic choices about how to intervene in it, and validating intervention hypotheses as rigorously as possible will continue to be essential, especially for funders and field-level intermediaries, who can commission large-scale research projects and serve the public by making outputs widely available.

These tools are about serving the thousands of smaller players, with tiny, overburdened communications teams and budgets to match: frontline Indigenous rights defenders in Peru, democracy advocates in rural Montana, Houston Area community organizers, to name just a few of the different groups our work has brought us into contact with this year.

What We Need to Build It

Our research and strategy teams have already started experimenting with approaches that can support AI tooling in the context of existing projects and workflows. And we have received a portion of the funding we need to begin prototyping an AI-powered persona and focus group tool.

Additional funding is needed to productize the persona and focus group tool, as well as to experiment with and build out a complete audience intelligence pipeline, a suite of tools that help communications professionals develop, embody, and test hypotheses for reaching different audiences. We also need funding to learn how to scale these tools outside the U.S.

In addition, we need partners to help us form and test hypotheses about what these tools should look like. We’re committed to developing them in the same way we developed the Narrative Observatory, in community, out in the open, where others can learn from our successes and failures.

If you’d like to start a conversation and learn more, please get in touch. As always, we’re interested in your thoughts and feedback on our work.