Should Meta Creative Testing Run to Cold Audiences Only?

Short answer: Yes. Creative testing should run to cold audiences, meaning people who have not bought from you or recently visited your site, because warm audiences buy from almost any ad and make weak creative look like a winner. Exclude past customers and recent site visitors from every test, use the exact same exclusions across every testing Ad Set, and check daily frequency to confirm the tests are really reaching new people.

A creative test is supposed to answer one question: can this ad turn a stranger into a buyer? If the test also shows the ad to people who were already about to buy, you cannot answer that question anymore. The ad gets credit for sales it did not create. This article explains why that matters and how we set up testing so the results can be trusted.

What is a cold audience in Meta ads?

A cold audience is people who have not bought from your brand and have not recently engaged with it, so they are seeing your product for the first time or close to it.

A warm audience is the opposite: past customers, recent website visitors, people who added to cart, people who engaged with your social pages. They already know you. Many of them were already leaning toward buying.

This difference matters most in creative testing, because cold and warm audiences respond to ads in completely different ways.

Why does creative testing need cold audiences?

Because warm audiences buy from almost anything, and that hides whether the creative itself is doing the work.

Meta creates demand in two ways. It introduces your product to people who never heard of you. And it helps people who already have the problem realize your product is the solution. Both are the job of a new ad.

Testing to warm audiences pollutes both signals.

  • A weak ad shown to someone who was already about to buy gets credit for that sale.
  • A strong ad shown to the same person gets the same credit. You cannot tell the two apart.
  • Ads that happen to get more warm delivery look better than ads that get less, for reasons that have nothing to do with the creative.

The result is a test that crowns the wrong winners. Then you promote those winners to Scaling, where they have to find new buyers, and they fall flat.

Which audiences should you exclude from creative testing?

Exclude all past customers, always. Exclude recent website visitors. Add social engagers when your tests start drifting warm.

Here is how we think about each one.

  • All past customers. When: Always, from day one. Why: They already bought. They are the warmest audience you have.
  • Recent website visitors. When: Always, with a window that fits your traffic. Why: They already know you and may be about to buy.
  • Social engagers. When: When tests start reaching warm people. Why: Engagers can be very warm and quietly inflate results.

The visitor window depends on the account. A brand with a lot of traffic and a short buying cycle can use a shorter window. A brand with less traffic or a longer buying cycle needs a longer one to keep warm people out.

Social engager exclusions do not have to be on from launch. We add them when we see signs the tests are going warm: net new reach falling month over month, frequency rising, or Meta over serving people who already know the brand. You can read about measuring that in net new reach.

Why must every testing Ad Set use the same exclusions?

Because if one Ad Set runs colder than another, you are comparing audiences, not creatives.

This is the rule people break most often without noticing. One Ad Set gets built with the full exclusion list. Another gets duplicated from an older version and is missing the visitor exclusion. Now one creative is running to cold traffic and the other is running to a mix of cold and warm.

When you compare those two and declare a winner, the conclusion is invalid. The creative that looks better may simply have been shown to warmer people.

So every creative testing Ad Set should carry identical exclusions. Same customer list, same visitor window, same engager rules. When anything changes, change it everywhere at once.

How do you know if your creative tests are really reaching cold audiences?

Check daily frequency. On single days, a frequency very close to one means mostly new people. Higher daily frequency means you are reaching people who have seen your ads before.

Frequency is the average number of times each person saw an ad. In a cold test, most people should see it once on a given day.

The key word is daily. Over a week, frequency always drifts up, because the same people stack across days. A weekly frequency will make almost every test look warm. Judge on individual days, and look at the range across days, not a single blended number.

We read daily frequency in rough bands:

  • Low: mostly new people. This is what a cold test should look like.
  • Middle: reaching people who have seen other ads from you.
  • High: behaving like remarketing, reaching people who have been on your site.

Frequency is a flag, not a verdict. A test that converts on high frequency days can still be a real winner. But it tells you the ad sold to warmer people, which changes where it is likely to succeed once promoted. We explain why that matters in inside the Scaling Campaign.

Do exclusions actually block everyone they should?

No. Exclusions catch a large share of the people they target, but never all of them. Privacy changes make some warm users invisible at auction time.

Some people decline tracking. Some browse on a device Meta cannot connect to your customer list. Some simply do not match. Those people are still warm, but Meta cannot see that they are, so they slip through.

What this means in practice:

  1. Some warm delivery is structural. No setting gets it to zero.
  2. Size that leftover warm delivery in dollars, not just as a percentage. A small share is noise on a small account and real money on a large one.
  3. Reported results will be slightly inflated by the warm buyers who slip through.
  4. On large accounts, reach data, not exclusion settings, is the real test of whether spend is finding new people.

Never tell a client, or yourself, that exclusions guarantee zero warm waste.

Should the Scaling Campaign also exclude warm audiences?

Only past customers. Scaling is different by design, and it should leave visitors and engagers in.

This trips people up. If cold matters so much, why not exclude everyone warm everywhere?

Because the two campaigns have different jobs.

  • Creative testing answers a question: can this ad sell to strangers? It needs a clean, cold audience to answer that honestly.
  • Scaling is where proven winners spend real money. It serves cold and warm non buyers together, and it doubles as your warm recovery campaign. It works best with minimal restrictions. The only exclusion is past customers, so you are not paying to reach people who already bought.

Most accounts built this way do not need a separate remarketing campaign at all, because Scaling already covers warm non buyers. You can read more about whether dedicated remarketing earns its budget in is your remarketing spend doing anything, and how the whole structure fits together in the CTSG system.

What are the red flags in a creative testing setup?

The common ones are mismatched exclusions between Ad Sets, missing exclusions, and retargeting ads mislabeled as tests.

Here is what we look for in an account audit:

  • Testing Ad Sets with different exclusions. Results cannot be compared.
  • Testing Ad Sets missing customer or visitor exclusions. Every result is warm and inflated.
  • Retargeting ads labeled as prospecting tests. They will always look like winners and will pollute your creative learnings.
  • A Scaling Campaign with no purchaser exclusion. You are paying to reach people who already bought.
  • Promotional campaigns with no purchaser exclusion. They can take credit for customers who would have bought anyway.

Campaign names are a claim, not a fact. Check the actual audiences and exclusions inside each Ad Set, and use daily frequency as a tiebreaker when the setup is unclear.

How does cold testing change how you judge an ad?

It makes the test result mean something. When an ad wins on cold traffic, it has shown it can create a buyer, not just collect one.

That is the whole point of creative testing. A cold winner is much more likely to keep working when it moves to Scaling and has to find new buyers at volume. A warm winner often stalls there. You can see how we judge each ad in how a creative testing ad gets judged.

Key takeaways

  • Creative testing should run to cold audiences, because warm audiences buy from almost any ad and hide whether the creative is working.
  • Always exclude past customers and recent website visitors from tests. Add social engagers when tests start drifting warm.
  • Every testing Ad Set must use identical exclusions, or you are comparing audiences instead of creatives.
  • Judge frequency by single days, not weekly blends, to confirm tests are really reaching new people.
  • Exclusions never catch everyone. Some warm delivery is structural, so measure it in dollars.
  • Scaling is different by design: exclude past customers only, and let it serve warm non buyers too.

FAQ

Can I test creative on warm audiences if I have a small account?You can run ads to warm audiences, but the results will not tell you much about the creative. Warm buyers convert on almost anything. If you want to learn which ads create new buyers, test to cold, even with a smaller budget.

How long should my website visitor exclusion window be?It depends on your traffic and buying cycle. Higher traffic and faster purchases can use a shorter window. Lower traffic or longer consideration needs a longer one. The key is to use the same window on every testing Ad Set.

Why does my weekly frequency look high even on a cold test?Weekly frequency always drifts up, because the same people stack across days. Look at single day frequency instead. On a cold test, most days should be very close to one.

Should I exclude social engagers from the start?Not necessarily. Many accounts start with customers and site visitors excluded, then add engagers when reach starts falling or frequency starts rising. Watch the data and add them when tests begin to drift warm.

Will exclusions stop all warm people from seeing my tests?No. Privacy changes hide some warm users from Meta at auction time, so a share always slips through. That is normal. Track how much spend reaches warm people and size it in dollars.

Want to know if your tests are really cold?

If your creative winners keep stalling once they hit Scaling, warm contamination in testing is a common cause. Our free Meta account audit includes a full account audit plus net new reach and cumulative reach reports, so you can see who your tests are actually reaching. Request your free audit here.

Keep reading

More on the CTSG system:

See every article on the CTSG system

Ready for your free Meta audit?

We will review your account and run net new reach and cumulative reach reports, free.

Get your free audit

Check what you learned

Tap an answer to see if you got it right.

1. Why should creative tests run to cold audiences?

People who already know you buy anyway, so you can't tell if the ad did the work.

2. Which audiences should always be excluded from tests?

Leaving them in turns your test warm. Add social engagers too when tests drift warm.

3. How should Scaling exclusions differ from testing?

Scaling is built to also reach warm non buyers, so it only excludes past customers.

Ready for the full test? Take the Media Buying exam and get certified. Want this run on your account? Get my free audit.

About the author

Jake Mehani, founder of Creative Analytics

Jake has spent 15 years running Meta and paid social ads for consumer brands. Meta published his RBX Active Advantage+ case study, he taught the DMAnc Meta and TikTok ads masterclass, and he spoke at Motion's 2024 Creative Strategy Summit. His senior team runs media buying, creative and measurement like an in house team.

Jake on LinkedIn  |  Case studies  |  Free Meta audit

Previous
Previous

How to Run a Daily Meta Ads Optimization (Without Overreacting)

Next
Next

How Do You Judge a Creative Testing Ad on Meta?