Skip to content
StatsKit Developers

Recipe: A/B test a headline

A headline test is the classic first experiment: swap the words in your hero and see which version drives more signups. This uses an element experiment, which the tracker applies before paint, so there is no flicker.

Before you start

Section titled “Before you start”
  • The tracker is installed on the page.
  • You know the CSS selector of the headline (for example #hero h1).
  • You have a vsk_live_… secret key for the API, or you use the dashboard.

  1. Create the experiment on the headline selector

    Section titled “Create the experiment on the headline selector”

    An element experiment swaps content on a selector. Add a control and one variant.

    Create the headline test
    curl -X POST https://statskit.ai/api/v1/experiments \
      -H "Authorization: Bearer vsk_live_…" \
      -H "Content-Type: application/json" \
      -d '{
        "type": "element",
        "name": "Hero headline test",
        "url": "/",
        "selector": "#hero h1",
        "goal": "signup",
        "statsEngine": "bayesian",
        "variants": [
          { "key": "control" },
          { "key": "b", "content": "Ship 2x faster" }
        ],
        "activate": true
      }'

  2. Define the goal event

    Section titled “Define the goal event”

    The goal above is signup: the event that counts as a conversion. Make sure that event actually fires when a visitor signs up. In the browser:

    Fire the goal on signup
    window.statskit?.track("signup", { plan: "free" });

    Or declaratively on the signup button:

    Declarative goal
    <button data-statskit-event="signup">Create account</button>

    See track events for both approaches. Name the event stably, the experiment reads conversions by this exact name.

  3. Publish

    Section titled “Publish”

    Creating with "activate": true starts the experiment immediately. Visitors are deterministically bucketed into control or b on their visitor id, and the tracker applies variant b’s copy before paint. If you created it as a draft, set it running from the dashboard or with a status update.

  4. Read the results

    Section titled “Read the results”

    As exposures and signup conversions accumulate, results compare each variant against control. With the default bayesian engine you get the probability b beats control and a credible uplift range.

    Fetch results
    curl https://statskit.ai/api/v1/experiments/exp_123/results \
      -H "Authorization: Bearer vsk_live_…"

    See reading results for how significance and direction are reported.

Section titled “Related”