CRO & experimentation / 2022–2025
By Anshul
Updated
50+ conversion experiments across D2C brands, with six published examples showing the changes, results and lessons.
- Client
- Seven D2C brands
- Period
- 2022 to 2025
- Examples
- Six published experiments
- Testing tools
- Convert, Shogun, Google Optimize
- Brands tested
- KettlebellKings, Flygrubs, Silvon, RDN, Baby Merlin, TribeWod, HappyPo
Philosophy
Not every test wins
Across the work summarized here, roughly 40% of our tests failed. These six examples include wins, mixed results and failures across collection pages, product pages and carts.
Each example describes what changed and what the team measured. The Flygrubs collection page example compares a control and a variant separately on mobile and desktop. The KettlebellKings example compares periods before and after a theme upgrade.
The figures reflect the reported project results. The published examples do not include traffic counts or confidence intervals for every comparison.
Process
How I run a test
- Identify the opportunity via analytics (high traffic, low conversion pages)
- Form a hypothesis based on competitive research or user behaviour data
- Design a variant with the specific change isolated
- Set a minimum sample size and a stopping rule before interpreting results
- Document results regardless of outcome
- Implement winners, learn from losers
Wins, partials and failures
Six experiments
01 / Flygrubs (chicken feed, US market) / Collection page
Flygrubs collection page: subscription highlighting
Hypothesis
Highlighting subscription benefits and bestseller products on the collection page will increase conversion and subscription sign-ups.
Why the collection page? It had the most organic and paid traffic. More traffic means bigger impact.
What changed
- Added subscription benefit callouts above the product grid
- Redesigned the UI to highlight bestseller products (recommended size)
- Clearer subscription pricing comparison
Flygrubs collection page: subscription highlighting: Mobile results
| Metric | Control | Variant | Change |
|---|---|---|---|
| Conversion rate | 6.53% | 7.68% | +17.5% |
| Add to cart rate | 13.80% | 15.03% | +9% |
| Avg session duration | 4:32 | 4:41 | +3% |
| Odds to win | – | 95% | – |
Flygrubs collection page: subscription highlighting: Desktop results
| Metric | Control | Variant | Change |
|---|---|---|---|
| Conversion rate | 10.75% | 12.38% | +15.2% |
| AOV | $43.94 | $46.58 | +6% |
| Odds to win | – | 97% | – |
Outcome
Winner. Implemented site-wide.
What I learned
Customers needed clearer subscription value props at the browsing stage, not just on the product page.
02 / KettlebellKings (fitness equipment, US market) / Site-wide theme and product detail pages
KettlebellKings theme and product page upgrade
Hypothesis
Upgrading to the latest theme version and adding A+ content on product pages will improve engagement and conversion.
What changed
- Upgraded the entire theme codebase to the latest Shopify version
- Added A+ content sections on all product pages
- Improved image galleries
- Better mobile responsiveness
KettlebellKings theme and product page upgrade: Dec 2022 to Apr 2023 results
| Metric | Before | After | Change |
|---|---|---|---|
| Overall CVR | 1.97% | 2.48% | +25.8% |
| Sessions with PDP views | 42.4% | 53.8% | +26.8% |
Outcome
The reported conversion rate rose from 1.97% to 2.48% across the before and after periods. This comparison cannot isolate the effect of the redesign.
What I learned
Sometimes you need a bigger swing (a full theme upgrade) rather than incremental tweaks.
03 / RDN (home decor, US market) / Homepage hero section
RDN hero section redesign
Hypothesis
A custom hero section based on competitive research will increase product discovery and engagement.
What changed
- Redesigned the hero with category-focused navigation
- Added lifestyle imagery
- Clearer value proposition
- Improved visual hierarchy
RDN hero section redesign: Mobile results
| Metric | Change |
|---|---|
| Product views | +30.88% |
| Engagement | +20.36% |
| Bounce rate | +8.13% (worse) |
RDN hero section redesign: Desktop results
| Metric | Change |
|---|---|
| Product views | +0.67% |
| Engagement | +1.49% |
| Bounce rate | −0.80% (better) |
Outcome
Partial win. Mobile showed a significant product view lift; desktop was flat. We implemented it for mobile and iterated on desktop.
What I learned
Mobile and desktop users behave differently. Sometimes you need device-specific solutions.
04 / Silvon (bedding, US market) / Homepage
Silvon homepage improvements
Hypothesis
A new homepage template with better information hierarchy and competitive positioning will improve revenue.
What changed
- Redesigned the homepage layout based on GA scroll data
- Added social proof sections
- Improved the product discovery flow
- Better trust signals right in the first scroll
Silvon homepage improvements: Desktop results
| Metric | Change |
|---|---|
| Conversion rate | +34.40% |
| Scroll depth | +6.78% |
| Bounce rate | +1.31% (slightly worse) |
Silvon homepage improvements: Mobile results
| Metric | Change |
|---|---|
| Scroll depth | +23.92% |
| Avg session duration | +4.84% |
| Conversion rate | −13.64% |
Outcome
Split decision. Desktop revenue significantly up; mobile engagement up but revenue down. We implemented the desktop version and continued iterating on mobile.
What I learned
Longer sessions do not always mean more purchases. Mobile users may have been overwhelmed by the additional content.
05 / KettlebellKings (fitness equipment, US market) / Homepage and product pages
KettlebellKings social proof: video on homepage and product pages
Hypothesis
Adding customer videos will increase social proof, engagement and conversion.
What changed
- Added customer video testimonials on the homepage
- Added product-in-use videos on product pages
- Embedded video player widgets
KettlebellKings social proof: video on homepage and product pages: Mobile results
| Metric | Change |
|---|---|
| Engagement | +7.19% |
| Avg session duration | +7.19% |
| Bounce rate | −0.08% (slightly better) |
| Revenue goal | −1.81% |
KettlebellKings social proof: video on homepage and product pages: Desktop results
| Metric | Change |
|---|---|
| Engagement | −5.07% |
| Avg session duration | −0.53% |
| Revenue goal | −1.92% |
Outcome
Failed. Engagement improved on mobile but did not translate to revenue. Desktop was worse across the board.
What I learned
Video content can increase time on site without improving conversion. Users may have been distracted rather than persuaded. We reverted and focused on static social proof instead.
06 / Baby Merlin (baby products, US market) / Product detail page
Baby Merlin product page A+ content
Hypothesis
Adding UGC images, testimonials and detailed benefits below the fold will improve engagement and conversion.
What changed
- Added an A+ content section with UGC
- Included detailed product benefits
- Added customer testimonials
- More lifestyle imagery
Baby Merlin product page A+ content: Desktop vs mobile results
| Metric | Desktop | Mobile |
|---|---|---|
| CVR | −10.5% | −2.47% |
| ATC rate | −14.7% | −14.4% |
| Session duration | −10.3% | −13.4% |
Outcome
Failed. All metrics declined.
What I learned
More content is not always better. For the Baby Merlin audience of new parents, the original concise product page was more effective: they wanted quick answers, not more information to process.
Learnings
What the tests taught me
- Mobile and desktop need separate strategies
- Results often diverged significantly between devices.
- More content can hurt conversion
- Especially for audiences seeking quick decisions.
- Engagement is not revenue
- Longer sessions and more scroll depth do not automatically mean more purchases.