How to Optimize Your Email Program Using Strategic A/B Testing

Mashkoor Alam
ByMashkoor Alam

Updated:

8 mins read

Updated:

8 mins read

Summarize with AI

Insights from Jean Jennings, 20-Year Email Veteran & Founder of Email Optimization Shop

Email continues to be one of the most profitable marketing channels — yet many companies barely scratch the surface of what’s possible. During our session with Jean Jennings, she shared how two decades of optimizing email programs for brands like PayPal, Capital One, Hasbro, and Verizon have shaped her approach to testing, strategy, and performance improvement.

What follows is a practical, insight-heavy guide built entirely from Jean’s real experiences — how she thinks, how she tests, and how she helps brands turn “good enough” email programs into high-performing revenue engines.

Watch the full webinar:

Why many email programs stay stuck — and what needs to change

Jean sees a consistent pattern across companies: email is profitable, so teams assume it doesn’t need investment.

In her words, email is “so inexpensive to do and not really that hard to make profitable,” which often leads brands to pour resources into newer channels while ignoring huge upside inside email. Instead, she believes every single send should be an opportunity to optimize — and when teams commit to that mindset, they often “double or triple their profits quite easily.”

This shift begins with using the right metrics and structuring email tests the right way.

What metrics actually matter

Jean calls open rate and click-through rate diagnostic metrics — useful to observe, but not useful for decision-making.

For real performance improvement, she relies on:

1. Revenue per email (RPE)

Jean’s favorite metric, because it measures what truly matters:
how much revenue a single email actually generates.

It’s simple:
RPE = revenue generated ÷ non-bounce sends
(and Jean always removes bounces because “bounces aren’t going to generate you any revenue.”)

2. Conversion rate

For non-revenue goals like lead submissions, Jean uses:
conversion ÷ non-bounce sends

She stresses that using opens or clicks as KPIs is a huge mistake.

In her own research:

  • Click-through rate predicts the true winner only 7% of the time

  • Open rate predicts the winner only 20% of the time

Her takeaway:

If you’re not measuring the action that keeps the lights on for your business, you’re optimizing the wrong thing.

How Jean approaches A/B testing — the scientific way

Jean has seen too many marketers run a/b tests “just because,” without a strategy.
Like testing red vs. green buttons without understanding why.

She pushes teams to test the way scientists do — starting with a hypothesis that has a real reason behind it.

Her iconic example: the color psychology test

Years ago, while reading a book on color psychology at Barnes & Noble, Jean realized:

  • Red means stop, danger, or “in the red”

  • Green means go, and also represents money
    (which fit the financial products she was promoting)

Despite pushback from the brand team (because green wasn’t a brand color), she tested it — and the green button drove a higher response.

This wasn’t random guessing. It was thoughtful hypothesis-driven testing rooted in consumer psychology.

That’s the core of her method.

What to test: 15+ elements, but start where you're weak

Jean shared that subject lines are the most commonly tested element — but rarely generate big wins.

Instead, she looks for the weakest spots in the email flow and tests the elements that matter most, like:

1. Image placement

Hero images often get blocked. She frequently sees lifts when:

  • the hero image is halved and paired with rich text

  • or a headline is placed on top of it

2. Landing pages

Landing pages are often “where it’s leaky.”
If that’s where drop-off happens, no subject-line test will save you.

3. Body copy approaches

She recommends building a message map of features, benefits, and advantages — and testing different angles of body copy.

4. How subject lines, preheaders, and headlines work together

Instead of testing them in isolation, she writes these three together so they build on each other, not repeat each other.

Her philosophy:

Go deeper than surface-level tests. Test the things that shape the decision, not just the click.

How young marketers and founders should build their email calendar

Jean sees new marketers get stuck at step one:

“Should we send promos or newsletters?”

Her answer: Yes — both.

Successful programs are relationship-driven, which means:

  • Send regular promotions (even once a month is fine to start)

  • Layer in newsletters to add value

She warns against building massive automation flows too early.

Instead:

Start with small automations

A simple 3-email sequence is often enough to begin testing.
Then:

  • If the 3rd email still performs → add a 4th

  • If the 2nd email underperforms → remove it

This iterative approach prevents teams from wasting months building complex flows that don’t work.

Jean’s mantra:

Start small, learn from the data, then build.

How to choose your first automations

When advising clients, Jean evaluates automation opportunities using two criteria:

1. Potential revenue or conversion impact

Example:
Cart abandonment emails usually generate strong returns.

Jean shared a surprising story:

One client manually ran a cart abandonment process once a week — and even that imperfect system generated $16 per email.

That signaled huge automation potential.

2. Resources needed to activate it

If a sophisticated version requires deep integrations, she recommends:

  • Launch the simple version first

  • Put the advanced version on the roadmap

This prevents teams from getting stuck in planning while revenue sits on the table.

Why some emails generate 1200% lifts — and what those tests revealed

Jean described one of her most dramatic wins:

  • A holiday email generated a 1221% increase in revenue per email

  • Some test emails for that client routinely performed 188% better than control

The reason?
The brand believed a certain product needed long, detailed copy.
Jean removed all the text and let the product image and basics stand alone — and conversions skyrocketed.

Her takeaway:

You can’t predict what will resonate. That’s why we test.

Why so many teams think email “doesn’t work”

Jean has heard every claim that email is dead — replaced by RSS, social media, or whatever the current “shiny new thing” is.

In her experience, the real issue is:

  • Teams don’t invest in improving email

  • They don’t learn from the industry

  • They don’t test enough

  • They don’t expose themselves to examples outside their niche

She encourages marketers to get inspiration from:

  • their own inbox

  • brands in other verticals

  • conferences

  • well-written newsletters

  • communities where marketers share results and learnings

Her belief:

You can’t come up with great ideas if you never expose yourself to them.

Segmentation that actually works (and why basic behavior is enough to start)

Jean sees many marketers overthinking segmentation and ignoring the easiest, most valuable data they already have:

Behavioral segmentation

The clicks inside their emails.

For example:

  • People who click often are very different from those who rarely engage

  • People who click on specific categories reveal their interests

  • Behavioral signals can fuel powerful personalization without extra data

She’s used link-level click behavior to target content effectively — like sending automotive content only to users who previously clicked automotive links.

Jean’s rule:

Start with the behavioral data you already have. It can keep you busy for a year.

She also warns against collecting unnecessary data because it creates privacy risk without adding value.

Open rate after Apple MPP: why it’s unreliable and what signals still matter

Jean explained why open rate was never accurate, even before Apple’s Mail Privacy Protection:

  • Some opens are false (image preview triggers)

  • Some opens are untracked (images disabled)

MPP magnifies the inaccuracy because Apple now triggers opens automatically.

But Jean shared two important nuances:

  1. Open rate is still directionally useful
    If you send to the same list regularly, rises and drops still tell a story.

  2. A drop in MPP opens can signal a deliverability problem
    If your email lands in junk, MPP doesn’t trigger an open — making falling MPP opens an early warning sign.

B2B vs. B2C email: the differences are smaller than people think

Jean sees both as relationship-driven channels.
The core strategies stay the same.

Differences tend to be:

  • tone and voice

  • sales cycle length

  • account-based nuances in B2B

But the testing strategy, content principles, and optimization methods are largely identical.

Her warning:

Ignore one-size-fits-all rules like “use emojis for B2C but never for B2B.”
Test everything. Every audience is different.

How she sees AI shaping email marketing

Jean has seen everything from panic to excitement about AI tools like ChatGPT.

Her view is grounded:

  • AI helps newer marketers write better subject lines and copy

  • AI helps experienced marketers with research and brainstorming

  • AI won’t replace high-level strategists or strong copywriters

  • Prompt mastery will become a key skill

In her words:

AI won’t replace you, but someone who knows how to use AI better might.

Key takeaways

Jean’s approach to email is anchored in scientific testing, smart metrics, and continuous improvement. She believes every send is an opportunity to optimize, every audience behavior is a clue, and every idea is worth testing if it has a strong hypothesis behind it. Revenue per email and conversion rate should guide decisions, not superficial metrics like opens, links.

And whether you're B2B or B2C, email is fundamentally about relationships — strengthened by segmentation, thoughtful content, and insights from testing. When teams embrace curiosity, experiment intentionally, and invest in learning, email becomes one of the most powerful and consistently profitable channels in their marketing mix.

What should you do next?

You made it till the end! Here's what you can do next to grow your business:

2_1_27027d2b7d
Get smarter with email resources

Free guides, ebooks, and other resources to master email marketing.

1_2_69505430ad
Do interactive email marketing with Mailmodo

Send forms, carts, calendars, games and more within your emails to boost ROI.

3_1_3e1f82b05a
Consult an email expert

30-min free email consultation with an expert to fix your email marketing.

Table of contents

chevron-down
Why many email programs stay stuck — and what needs to change
What metrics actually matter
How Jean approaches A/B testing — the scientific way
What to test: 15+ elements, but start where you're weak
How young marketers and founders should build their email calendar
How to choose your first automations
Why some emails generate 1200% lifts — and what those tests revealed
Why so many teams think email “doesn’t work”
Segmentation that actually works (and why basic behavior is enough to start)
Open rate after Apple MPP: why it’s unreliable and what signals still matter
Key takeaways

Meet the only AI
email automation
platform

Mailmodo Logo

Enter with an idea.
Exit with a winning email campaign.

Check.svg

Brainstorm email campaign ideas with Mailmodo’s AI assistant

Check.svg

Build precise segments in seconds with AI segmentation

Check.svg

Generate ready-to-use email templates with AI

Experience true AI email marketing automation with Mailmodo

Trusted by 10000+ brands

Group_1110166020_1_6fb9f2bd9a
Group_1110165532_1_bf39ce18b3
Ellipse_Gradientbottom_Bg
Ellipse_GradientLeft_Top