In Year 1, the Narrative R&D Lab set out to create shared, practitioner-tested infrastructure for narrative change, complete with audience-mapping tools, a content-testing platform, creator-briefing systems, and measurement tools to assess what stories are working.

We tested what we built through live campaign experiments with media partners including NowThis Impact, BuzzFeed, ParentsTogether, and Plus Más. We then measured the impact of the content with real audiences, covering everything from skills-based hiring to surveillance pricing.

Overall, we found that the content moved people. Just not always the people we were aiming for. That gap between aggregate movement and target movement is the central lesson from the Lab’s first two campaign experiments. We’re sharing some of those field notes below.

A repeatable campaign system

The questions we sought to answer were of a fundamental nature: Can we translate audience research into briefs, deliver briefs to media partners and creators who can successfully translate them into impactful content, get that content in front of the right people, and measure whether any of it moved them? What does each link in that chain teach us along the way?

Two organizations, Opportunity@Work and Groundwork Collaborative, each acted as our campaign partner to identify a campaign topic along with a campaign goal and a longer-term narrative goal. From there, we worked with media partners to find the overlap between their existing media property audiences and the key stakeholders we most wanted to reach. We mapped this overlap using our values-based audiences (more on that here), and the overlap became the campaign’s target audience that we developed a creative brief for. Every brief produced a piece of content that we published and tested in a randomized controlled trial (RCT) against placebo content.

We map out the whole process below.

Before we walk through what happened in each campaign, here’s what we took from the experience as a whole.

Key takeaways

Going in, we treated “audience mapping” as the front-end work that sets up a campaign for success. What these initial experiments showed us is that “mapping” in narrative change work is a chain of interlocking questions, and each campaign surfaced a different link.

AUDIENCE FIT

Does the creator’s audience overlap with the people we want to move? One of the most important decisions in a campaign is who you partner with, because their audience is your ceiling. Do as much audience mapping as you can before you pick a creator or channel.

We also learned the importance of building measurement that tells you whether you moved your actual target. Overall lift is easy to report, but also an easy thing to mistake for success. Positive aggregate numbers can hide the fact that the people you most wanted to reach weren’t moved at all.

CREATOR FIT

Can the creator’s voice authentically carry the story/message? Before you commit to a creator, ask whether their typical format, vibe, and voice can deliver your story. Audience overlap with a creator isn’t the same as fit between what they organically make and what the campaign goal asks for. If the creator’s natural register is satire, gut-check whether satire can plausibly land your campaign goal. If their natural register is intimate first-person, gut-check whether that form can carry an institutional argument.

STORY FIT

Can the creator translate the brief into a story that resonates with the target audience’s values? What the brief asks for and what gets shipped don’t always match up, because sometimes, audience mapping doesn’t survive story translation. The translation layer—using audience values to construct a specific story, then collaborating with the creator to deliver it—is something we’ll continue to explore in future experiments.

Beyond story fit, we also learned that brief iteration is worth investing in, but there’s a point of diminishing returns. We experimented with varying levels of audience specificity in multiple rounds of briefs. More refinement didn’t always amount to more lift. While we can’t isolate the level of detail in a brief as the variable that mattered for these initial campaigns, we did find that the “ideal brief” should be treated much more like a collaborative process between campaign strategy, audience research, and creative production—and that looks less like a golden template and more like a responsive back-and-forth between the strategists, researchers, and creators.

…

Over the next year, with AI as our topic focus, we’ll continue to share field notes from Lab experiments we’re conducting alongside our working cohort of 18 narrative change practitioners, who’ll each be running their narrative campaigns through the Lab’s testing infrastructure. And if you’re running campaigns of your own and want to compare notes, write to Lab director Rob Avruch.

…

Watch this space for more learnings, and read on for a closer look at both campaigns.

Hire for Skills

• Campaign topic: Economic mobility through the lens of shifting skill-based hiring practices
• Campaign partner: Opportunity@Work
• Campaign goal: Move hiring managers to hire workers Skilled Through Alternative Routes (STARs) instead of a college degree
• Narrative goal: Move everyone toward seeking an end to poverty
• Media partners: NowThis Impact, Warp Zone, Plus Más.

Hire for Skills: Audience-Mapping

The Hire for Skills campaign’s key stakeholders were hiring managers, which we defined broadly as “people aged 18+ who reported their occupations as executive/managerial,” and then mapped onto specific values-based audience segments based on existing media property audiences.

For example, the target audience within NowThis Impact as a media property lives in the overlap between the overall YouTube audience associated with hiring managers (our key stakeholders) and NowThis Impact’s existing audience on YouTube—mapped against Harmony Labs’ values-based segmentation. This target audience makes up roughly 22% of “reachable” employers on YouTube, and puts them between the People Power and If You Say So audience segments, a skeptical-but-idealist group that distrusts institutions but believes change is possible.

Hire for Skills: Campaign Findings

We found that, in aggregate across all English content, the content moved viewers significantly on the campaign goal and on the poverty narrative goal, but the audience-level story is where the real learning is.

Hiring Based On Talent Not College

This NowThis Impact video moved the People Power audience significantly on both the campaign and narrative goals. However, the campaign’s key stakeholders (hiring managers), clustered between If You Say So and People Power, didn’t move significantly on the hiring ask. They did move toward the idea of ending poverty. It’s not surprising that the audience moved most on ideas that were solutions-oriented. They’re open to the goal; they just don’t yet see themselves as the people who act on it.

When the Justice League hires a new hero

Another video from Warp Zone reached the right audience neighborhood of People Power but it didn’t move them significantly on either goal. The content satirically raised the issue of skills-based hiring, but it stopped short of modeling anyone doing the right thing. This was in line with the kind of content Warp Zone produces, but not showing an alternative future might have reduced the impact of the media. At our request, Warp Zone created an alternative ending to the media that did show characters making different hiring choices. Warp Zone did not publish this piece on YouTube, but we did test its persuasive power, and it moved audiences on the narrative goal overall. The alt cut shows that satirical content, when it has a solution built into the message, can produce positive movement on both hiring behavior and beliefs about poverty.

More than 50% of Latino workers have the talent, but not the title

Plus Más’ Spanish-language video reached Don’t Tread On Me especially in a way the English-language piece didn’t, moving them significantly on the hiring ask and specifically the “If no one lived in poverty, everyone would be able to choose their own path in life” statement. The campaign’s target audience of People Power-If You Say So, however, moved in the right direction but didn’t reach statistical significance. The brief named the right audience, but the story maps more cleanly onto values held by the Don’t Tread On Me audience, rather than to values distinct for the People Power-If You Say So audience. The video seems to frame the hiring problem as artificial gatekeeping which likely resonates more with Don’t Tread On Me’s belief that hard work and earned experience should speak for themselves.

Nickel & Dimed

• Campaign topic: Economic mobility, with a focus on unfair pricing strategies
• Campaign partner: Groundwork Collaborative
• Campaign goal: Build awareness and action on unfair corporate pricing practices
• Narrative goal: Move everyone toward seeking an end to poverty
• Media partners: NowThis Impact, Warp Zone, Plus Más, Buzzfeed, ParentsTogether, Cocoa Butter

This campaign was designed to isolate a specific question from Hire for Skills: does a more detailed brief iteration improve outcomes? We ran three rounds of content creation, publishing, and testing with the same set of media partners, tightening the creative brief each time to be progressively richer and more specific—starting with industry-standard demographic information, then adding values-based audience insights and detailed content strategy—to see what best equips a creator to reach and move their target audience.

Nickel & Dimed: Brief Creation

We tested three versions of the same brief with each creator partner, each one more information-rich than the last.

The first set of briefs was a conventional brief that provided information about the campaign topic but didn’t contain any information about the target audience.

The second set of briefs served as a conventional audience brief that provided information about the campaign topic and basic demographic information about the target audience, but didn’t go into further detail.

The third set of briefs provided information about the campaign topic and detailed information about the target audience, including values and perspectives on the campaign topic, as well as content strategy as a starting point based on what our values-based research shows about the audience.

Nickel & Dimed: campaign findings

We found that eleven of the thirteen pieces of content hit statistical significance in terms of moving audiences on the campaign goal, which are strong overall numbers. But again, the brief-level insight complicates the takeaway.

How is it legal for companies to use surveillance to charge different people different prices for the same product?

Everything is already so expensive — but did you know we're not all even being charged the same amount?

The English content from the second round of briefs, which included both demographic and values-based audience info, produced consistently strong effects. Examples include NowThis Impact’s video, and ParentsTogether’s video.

Price tags used to be a promise that we all paid the same amount for the same product at the same store.

More refinement in the third round of briefs, which also included more information about the audience’s media consumption behavior and content strategy, didn’t keep adding lift, and in some cases the movement declined, like Parents Together’s video, their third piece on the same topic. It’s worth noting that by the third round, multiple factors could have been driving the testing results, including the brief’s quality and the creator working to iterate on the same topic a third time. Both factors likely contribute to the impact, and future campaign experiments should continue to try to dig into these elements.

This Is Why Everything Is So Expensive

Can You Guess The Final Price After All The Fees?

Several pieces of content produced strong, significant movement on the narrative goal among Tough Cookies and If You Say So audiences, like this Buzzfeed video and this video from Cocoa Butter respectively. But the People Power-If You Say So audience, the campaign’s target audience, didn’t reach significance on the narrative goal—the same shape of lesson as Hire for Skills.

One explanation for why the target audience didn’t move significantly is that the People Power-If You Say So audience leans toward systemic solutions like structural or regulatory action, but the questions we used to measure impact during content testing focused on individual action and consumer self-protection (“I can take action to protect myself from unfair pricing”). This means that the content may have moved them in some way, but not on the questions we were measuring.

…

We’d love to hear from you. If you’re running your own campaigns and want to compare notes, write to Narrative R&D Lab director Rob Avruch.