A/B testing in C#: How to stop guessing and start earning

Why your C# code is working in vain (and how to fix it)

You spent weeks refactoring the recommendation algorithm. The new version of RecommendationService looks flawless, the code is cleaner, the LINQ queries are faster. You roll out the update to production, and… conversion drops by 15%. Sound familiar? If not — you're lucky. If yes — you just lost money and time because you relied on intuition, not data.

In the world of C# development, there is a golden rule: "Never trust your opinion, trust statistics". This is exactly what A/B testing (or split testing) is for. It's not a marketing gimmick — it's an engineering approach to decision-making. In this article, we'll break down how to implement full-fledged A/B testing in your C# application: from architecture to calculating statistical significance. And yes, there will be a lot of code.

What is A/B testing in the context of C#?

Imagine you have two variants of the same logic — control group A (old version) and experimental group B (new version). You randomly distribute users between them, collect metrics, and use math to determine which variant is actually better.

In C#, this can apply to anything:

\n
• Changing the search algorithm
\n
• New payment form design
\n
• Different product sorting options
\n
• Experiments with caching time
\n
• Testing ML models
\n

The key point: A/B testing in C# is not about "setting a flag in the config". It's about clean architecture, randomization, and bug-free metric collection.

A/B testing architecture: Layer Cake in C#

Let's get straight to practice. I'll show you a minimal but working architecture. It consists of three layers:

\n
1. Assigner — decides which group the user falls into.
\n
2. Feature Decision — which code variant to execute.
\n
3. Tracker — records the event and the result.
\n

Step 1: Assigner — who are you, user?

The most common bug is incorrect randomization. Using Random is a bad idea because groups will "drift" when the application restarts. The correct solution is deterministic distribution based on UserId.

public interface IExperimentAssigner\n{\n    string GetVariant(string userId, string experimentName);\n}\n\npublic class ConsistentExperimentAssigner : IExperimentAssigner\n{\n    public string GetVariant(string userId, string experimentName)\n    {\n        // Create a stable hash from userId + experimentName\n        var hash = HashCode.Combine(userId, experimentName);\n        // Take modulo 100 — get a percentage\n        var bucket = Math.Abs(hash) % 100;\n        \n        // Let's say 50% in A, 50% in B\n        return bucket < 50 ? "A" : "B";\n    }\n}\n

Now the same user will always be in the same group for a specific experiment. This is critical for data correctness.

Step 2: Feature Decision — battle of algorithms

Now we need the code to automatically choose which logic to execute. We use the Strategy pattern with a factory.

public interface IPricingStrategy\n{\n    decimal CalculatePrice(decimal basePrice);\n}\n\npublic class OldPricingStrategy : IPricingStrategy\n{\n    public decimal CalculatePrice(decimal basePrice) \n        => basePrice; // No discount\n}\n\npublic class NewPricingStrategy : IPricingStrategy\n{\n    public decimal CalculatePrice(decimal basePrice) \n        => basePrice * 0.9m; // 10% discount\n}\n\npublic class PricingStrategyFactory\n{\n    private readonly IExperimentAssigner _assigner;\n\n    public PricingStrategyFactory(IExperimentAssigner assi