If you would like to evaluate Guides and Surveys, please reach out to a member of your account team or fill out this form to be connected with a Fullstory team member.
Who can use this feature?
- Part of Guides and Surveys, an add-on for Business, Advanced, and Enterprise plans.
- Requires an Admin, Architect, or Standard role to configure.
Guides and Surveys is in active development. Product functionality and documentation may change without notice.
A/B experiments in Guides and Surveys let you compare two or more versions of content to determine which performs better. By splitting users into groups and showing each group a different variant, you can make data-driven decisions about which tours, checklists, banners, or other content drives the best results.
This article covers the following:
- Creating an A/B experiment
- Using experiment conditions in content targeting
- Frequently Asked Questions
Creating an A/B experiment
To create an A/B experiment, navigate to Guides and Surveys and select A/B Experiments from the sidebar.
- Click Add Experiment.
- Give your experiment a name and define the variants along with the distribution percentage for each audience segment.
- Click Install to retrieve the tag values for each variant (for example, "Champion" and "Challenger").
When you create the experiment, Guides and Surveys begins assigning users to variants based on the distribution percentages you configured. Assignment is an ongoing process: each user is assigned independently the first time Guides and Surveys encounters them, using a weighted random draw against the configured percentages. Users who appear after the experiment is created are assigned in the same way as users who existed at creation time.
Using experiment conditions in content targeting
After creating an experiment, you can use it to control which content each variant group sees. In your content's targeting settings (for example, when configuring a tour or banner), add a condition using AB experiment is.
- Open the targeting settings for the content you want to test (for example, a tour).
- Add a new condition and select AB experiment is.
- Choose the active experiment and the variant (for example, "Champion" or "Challenger") that should see this content.
Frequently Asked Questions
Why aren't users being assigned to experiment variants?
Users must have profiles in Guides and Surveys before they can be assigned to an experiment variant. Profiles are created when user IDs propagate from Fullstory to Guides and Surveys. Users who do not yet have a Guides and Surveys profile will be assigned to a variant once their profile is created.
How are users assigned to experiment variants?
Each user is assigned independently the first time Guides and Surveys encounters them after the experiment is created. Assignment uses a weighted random draw based on the distribution percentages you configured—for example, a 50/50 split gives each user an equal chance of landing in either variant. This is an ongoing process: users who appear after the experiment is created are assigned the same way as those who existed at creation time.
Why don't my variant groups match the exact percentages I configured?
Because each user is assigned independently via a random draw, the actual distribution in your account may not perfectly match the configured percentages— especially with smaller audiences. This is normal random variation, not a bug. With large audiences, the observed split converges close to the configured percentages. With small audiences, the difference can look more pronounced.