StoryAI Opportunities - Funnel Drops

Overview of Funnel Drops

Funnel drop monitoring proactively watches key conversion funnels. The goal is to identify significant decreases in conversion rates at any stage of the funnel. When a drop occurs, the system aims to provide insights into the potential reasons behind it. Even if a definitive root cause isn't immediately apparent, the analysis process itself offers valuable context to accelerate resolution.

This article will cover the following:

• Overview of Funnel Drops
  • Quick start
  • Overview of the Funnel Drops interface
  • Understanding funnel baselines
  • Investigating a funnel drop
• Recommendations for using funnels in Opportunities
  • Example: Recommended funnel drop setup
• Frequently asked questions

Quick start

Customers with StoryAI Premium can quickly get started with Funnel Drops by doing the following:

1. Within Fullstory, select Opportunities from the left-hand menu.
2. Click Funnel Drops at the top of the Opportunities page.
3. Click the 3 dots (⋮) next to Opportunities and choose Connect Funnel.

4. Select an existing funnel to monitor for drops.

   Note: We recommend starting with a two-step funnel. See recommendations below to learn more.
5. After selecting a funnel, you'll see a message that data is being baselined analyzed and to report back soon.
6. ⏰ Once the data is ready, you'll see one of two things:
   • An indication that no drops occurred between the specified date range or
   • A series of drops within the specified date range, with a button to Investigate.

Continue reading to learn more about the funnel drop page.

Overview of the Funnel Drops interface

When viewing Funnel Drops within the Fullstory interface, you'll notice the following:

• Funnel Drops are always grouped by the funnels that have been added to Opportunities.
• A banner at the top will call out drops that you should consider looking into, with a button to Investigate (see below). If there are no drops that occurred within the specified date range, the banner will be updated accordingly.
• A bar chart on the left shows you at which step in the funnel the drop occurred.
• A trend chart on the right reflects at which point in the funnel the step dropped below baseline.

Understanding funnel baselines

To identify a "drop," Fullstory needs to assess what typical performance looks like. We begin by establishing an hourly baseline for each monitored funnel, which involves examining the conversion rates from the previous three weeks, specifically looking at data from the same day of the week and the average of hours right before the current time window. This approach helps to automatically account for factors like seasonality and holidays, creating a dynamic expected range for funnel performance.

A significant deviation below this expected range triggers the "funnel drop" investigation and initiates the root cause analysis process. This definition applies to both the overall conversion rate baseline and the baselines of the step-to-step conversion rates.

While funnel drop analysis is done on an hour-by-hour basis, Funnel Drops that persist across multiple hours are counted as a single "continuous funnel drop". Rather than creating separate reports for contiguous hours of Funnel Drops, if you have a continuous drop that's persistent—for example, for three straight hours—it will be treated as a single funnel drop report.

Investigating a funnel drop

When you click the Investigate button on a funnel drop, it will take you to a detailed page where you can start triaging and dive deeper into likely factors that contributed to the drop.

• Change the Status and Assigned To to begin the triage process. For example, setting the status to "Investigating" will move the issue forward to "In Progress" on the Opportunities kanban board.
• The same bar chart and trend chart from the Funnel Drops page will appear at the top of the investigation.
• An Investigation Summary and corresponding metrics provides a quick overview.
• Investigation Details help you to dive into Likely factors across Properties, Errors, and Traffic.
  • Likely factors and Traffic can be expanded to show a session playlist.
  • Unlikely factors show you which dimensions were analyzed but saw no significant changes within the specified time window.
• Fullstory users can leave Comments at the the bottom of the investigation.

Recommendations for using funnels in Opportunities

Funnel Drop monitoring in Opportunities provides detailed analysis on your key funnels. To get the most out of it, here are a few guidelines:

• Time: The more time that we can have to analyze a funnel, the more we can learn about it.
• "Canonical" funnels: Funnel Drop monitoring works best with canonical funnels within your website experience—the step-by-step processes that define your core user pathways—and not temporary ones that need ad-hoc analysis or short term alerting.
• Zero overlap: You should aim to have zero overlap (or as little as possible) in the funnels that you choose to monitor with Opportunities. Overlap leads to noisier alerts.
• Two-step funnels: We generally believe that users get the best results with two step funnels, where each step represents a significant milestone in the user journey. This can be combined with another two step funnel that encapsulates the entire journey. The more users and distance in the customer journey that are encapsulated within the funnel, the more volatile the conversion rate is, making it more difficult to get accurate baselines. 

Example: Recommended funnel drop setup

Applying the guidance from above, here is our recommendation of how to track funnels for an eCommerce website. While this example is tailored for eCommerce, the same general principles will apply to any industry.

• Overall Conversion Rate
  1. Any Activity
  2. Revenue event
• Home Page to Product Listing Page (PLP)
  1. Visited Page is Home Page
  2. Visited page is Product Listing Page
• Search to Product Detail Page (PDP)
  1. Custom event is Search
  2. Visited Page is Product Detail Page
• PLP to PDP
  1. Visited Page is Product Listing Page
  2. Visited Page is Product Detail Page
• PDP to Checkout
  1. Visited Page is Product Detail Page
  2. Visited Page is Checkout
• Checkout to Revenue Event
  1. Visited Page is Checkout
  2. Custom event is Revenue Event

If you have a website and a mobile app, the same structure of funnels should be created twice, one for each property.

Frequently asked questions

How does the timing of the funnel drop analysis work?

Funnel drop analysis happens on an hourly basis. The time of the hour that the analysis occurs is optimized for each funnel.

When you first connect a funnel to Funnel Drops, it can take a few hours for both charts to show up. Once a funnel is added, Fullstory backfills the last three weeks of data to allow for baseline calculations. From that point forward, the funnel will get checked hourly. Note that if a drop occurs within the first hour of monitoring, it will be shown in the bar chart, but the trend chart will not have sufficient data until the subsequent hour.

Why don't you recommend one large funnel?

While Opportunities supports larger multi-step funnels, two-step funnels provide the benefit of not requiring all of the previous steps in the journey, which gives us the ability to track users who skip portions of the early journey, but ultimately still may convert. As an example, take the following funnel steps:

1. Visited Page is Home Page
2. Visited Page is Product Listing Page
3. Visited Page is Product Detail Page
4. Visited Page is Checkout
5. Custom event is Revenue Event

If you were to define your funnel like above, your overall conversion rate will only include people who followed every single step in this funnel, rather than those who may have found a way to a page further in the checkout journey (say, from an email marketing campaign). 

What funnels are restricted?

At this time, there are a few funnel definitions that are not supported in Opportunities. To make the funnel compatible for use, ensure that funnels do not have the following:

• Multi-segment funnels
• Funnels set to “across any number of sessions”
• Funnels with applied segments that have event filters (user filters are OK)
• Funnels with applied custom or since time ranges
• Funnels with applied “group by” properties
• Median time to complete is over 2 hours