Available for the following Plan types:
FullStory Enterprise*
FullStory Advanced*
FullStory Business*
*with the following add-on:
Data Destinations
Available to the following User roles:
Admin
Architect
Standard
Data Destinations is available as an add-on feature. If you are interested in adding this to your account, please reach out to your account representative for more details.
When FullStory is deployed on your site or app, we ingest a massive amount of raw data that we then clean and structure into analytics-ready event data. All this data is used to power FullStory's search and digital experience analytics features when you want instant, out-of-the-box analysis. However, there are times when you need to dive deeper into the data to truly understand custom use cases. FullStory has already captured the most complete product experience dataset, and now we are making this data available to our customers via Data Destinations.
What is Data Destinations?
Data Destinations enables FullStory customers to sync their digital experience data directly into their cloud data warehouse, such as Snowflake or Google BigQuery. FullStory will manage the structuring, delivery, and deduplication of captured events without customers needing to engage their engineering teams to build custom ETL pipelines.
With Data Destinations, you can merge FullStory's digital experience with data from your other essential business platforms to get a complete picture of your business's health. Now, every analyst at your company can explore FullStory data directly through their preferred SQL interface or an existing Business Intelligence tool.
When should I use it?
Data Destinations is the answer for customers who want to integrate FullStory's digital experience data into their internal reporting pipelines. With this technology, FullStory will sync all the events captured from your product into your data warehouse in an analytics-ready format so that you can monitor the metrics that matter most to your business. Perform robust analyses of your product and model KPIs relevant to your business.
How does it work?
We will manage a regular sync process that directly pushes all captured user events into your warehouse. We'll also handle all of the logic around retries and deduplication to ensure that you always have the most accurate data at your fingertips.
Integrations
There are two supported integrations. Please review the additional documentation that supports installing these integrations:
Note: FullStory only supports one active destination per org
How to enable
To get started, navigate to Settings > Integrations > Data Destinations and select Snowflake BigQuery or Redshift, depending on which warehouse you use to store your data. Once you've saved your credentials, the integration will push data to your warehouse for as long as the integration remains active. Keep in mind only one destination is supported at a time. If you want to change destinations, remove any existing Data Destinations integrations first.
FullStory's approach to data
Built to Support Planned Workflows
The Data Destinations feature is designed for customers with dedicated analytics support who want to automate their data ingestion and reporting process. This feature will only sync data captured when the integration is active and will not sync data retroactively. To ingest all the data FullStory processes, you should activate a destination when installing the FullStory JS tag or SDK.
Future Proof Schemas
FullStory has designed our data model around an expectation for two main types of data: required fields and conditional fields. For fields that are required on every event, we have extracted those fields into a dedicated top-level column. Our goal is to avoid sparse data in the raw table, so any field that is conditional to the event type is packed into the source_properties
or event_properties
columns as JSON objects. This data model allows FullStory to add new event_types
and properties without having to run costly data migrations every time a new event is defined. FullStory will maintain a consistent and future-proof data model while enabling customers to handle denormalization and aggregation downstream.
Sync Interval vs. Update Window
Sync interval is a concept that tracks the cadence with which FullStory syncs new events to the warehouse. Captured events flow into FullStory constantly, then get bundled into a package each hour and pushed to the warehouse destination. All events ingested into FullStory's servers within that hour are included in the package.
Due to the nature of web traffic, distributed systems, and worldwide users, not all events will hit FullStory's servers chronologically. There are some scenarios where events come out of order, like swan song events that appear hours after they occur. FullStory's reporting pipeline treats pages and sessions as top-level entities. All events that flow into the system will be associated with a page, a session, or both. This relationship is then used to derive metrics like "session length" and "active time on page." When a late event is captured, FullStory will reprocess the page or associated session to "put the events back in order." When this reprocessing happens, FullStory will also push the updated events out to the warehouse, sometimes overwriting the data previously associated with that event. This is called the Update Window, or the rolling lookback period where FullStory may update events in the warehouse. For a small set of edge cases, events may be reprocessed after the update window, causing minor discrepancies between the FullStory UI and the data warehouse data. A very long update window reduces discrepancies but can make it harder to "close the books" from an accounting perspective.
Event Time vs. Processed Time vs. Updated time
Event time is an immutable field that records the timestamp of each event according to the user's device.
Processed time is a timestamp field indicating when the event was captured and processed by FullStory's servers. On average, 95% of events are captured and processed within 20 minutes of the original event time. Several events, including server side events, may reach FullStory's servers much later than the original event time. Depending on the contents of these late events, FullStory may need to reprocess them to report on the most accurate metrics, including session length, page active time, etc. In these scenarios, the Processed Time for all events in the session or on the page will be updated and the events will be re-synced to the warehouse.
Updated time is the equivalent of running current_timestamp
in your warehouse every time a record gets inserted or updated. This field tracks the last time a record in your warehouse was changed and can be used as a filter to determine which new records to pull into your query.
Moving Analysis to the View layer
Our approach to support a future-proof data model means we are pushing all of the data FullStory captures into a pre-defined set of columns. This proves efficient from a data processing standpoint and sets clear expectations around the data contract. With that said, it has some implications when approaching queries. Specifically, FullStory's underlying events table needs to be unpacked and denormalized into analytics-ready models to make querying easier, especially for BI tools.
To help with this process, FullStory has published a set of DBT SQL transformations that you can deploy in your warehouse. These views will not solve every use case possible, but they aim to provide context around how the data can be transformed and explain how many of the metrics within FullStory can be calculated.
Event Source Info
Every event now contains contextual information about where that event came from and how the event was passed into the FullStory system. To keep your custom event payload from overloading with these extra parameters, we've implemented support for Source Properties. This will make it transparent where the event came from (web, app, or server integration) and which FullStory API endpoints were invoked to pass the event. You can easily QA your instrumentation by performing data quality checks against your other systems.
Source Properties Fields
Category | Field Name | Warehouse Schema Field |
Browser | Initial Referrer | source_properties.initial_referrer |
User Agent | source_properties.user_agent | |
Visited URL | source_properties.url | |
Location | City | source_properties.location.city |
Country | source_properties.location.country | |
IP Address | source_properties.location.ip_address | |
Latitude | source_properties.location.lat_long | |
Longitude | source_properties.location.lat_long | |
Region | source_properties.location.region | |
Mobile | App Screen Name | source_properties.app_screen_name |
Source | Entrypoint | source_properties.entrypoint |
Integration | source_properties.integration | |
Origin | source_properties.origin |
User Info
User data, such as the user ID or custom user properties you pass through FS.identify, is available in the Data Destinations event data. All user data can be found within `identify` event types. This user information can be easily joined back onto all events using the device_id column. While this approach does require an additional join, it provides better support for sessions that are identified out of chronological order. If a session is identified after being synced to the warehouse, we emit one new identify event rather than updating existing records for that session.
Schemas and Event Definitions
Events Table Data model
Field name |
Type |
Description |
event_id |
STRING |
unique key to reference each individual event |
event_time |
TIMESTAMP |
timestamp associated with the event |
processed_time |
TIMESTAMP |
timestamp the event was processed by FullStory's servers |
updated_time |
TIMESTAMP |
timestamp the event was last inserted or updated in the warehouse table |
device_id |
INTEGER |
persistent storage identifier for the user's device |
session_id |
INTEGER |
unique id associating multiple events to the same session. this id is unique for any given device_id. |
view_id |
INTEGER |
unique id associating multiple events to a single page, screen or tab. this id is unique for any given device_id and session_id. |
event_type |
ENUM |
classifier for the type of event |
event_properties |
JSON |
json column containing all captured properties associated with each event |
source_type |
ENUM |
classifier for the originating source of the event |
source_properties |
JSON |
json column containing all the captured properties associated with the originating source |
Event Types and their Properties
Event Type |
Description |
Event Properties |
background |
When a mobile app is running it is not in focus on the screen.
Triggers when: Native Mobile lifecycle event when the app, or an activity within the app, moves from the foreground to the background |
|
change |
Finds activities where a user changed a specific element on your website, such as an input field.
Triggers when: Change to a text field |
fs_suspicious_kind:string target.masked:boolean target.raw_selector:string target.text:string |
click |
Finds activities where a user clicked on an element.
Triggers when: a pointing device button (such as a mouse's primary mouse button) is both pressed and released |
fs_dead_count:number fs_error_kind:string fs_rage_count:number is_click_long:boolean (*mobile only) is_click_unhandled:boolean target.masked:boolean target.raw_selector:string target.text:string |
consent |
A call to FS.consent |
consent_action_type:string consent_scope:string is_consent_given:boolean |
console_message |
Finds activities where a console error occurred. Surfaced by its console error message.
Triggers when: Web: https://developer.mozilla.org/en-US/docs/Web/API/console/error() and https://developer.mozilla.org/en-US/docs/Web/API/console/assert (false, ...)
Android: https://developer.fullstory.com/logging?lang=Java (’FS.LogLevel.ERROR’, ...)
iOS: https://developer.fullstory.com/logging?lang=Swift (’FSLOG_ERROR’, ...) or https://developer.fullstory.com/logging?lang=Swift (’FSLOG_ASSERT’, ...) |
console_message_level:string |
copy |
Finds cases where a user highlights text on the screen and copies it.
Triggers when: User copies highlighted text via CTRL+C or right-click |
target.masked:boolean target.raw_selector:string target.text:string |
crash |
Finds activities where a crash occurred on a mobile application.
Triggers when: detect an Unhandled Exception in Java, which causes the app to force quit |
|
cumulative_layout_shift |
Finds activities where content shifts webpage when it's not in response to a user's action. : The largest layout shift that occurs within a 5-second window on the page, as reported by https://developer.mozilla.org/en-US/docs/Web/API/LayoutShift , when hadRecentInput is false. |
cumulative_layout_shift:number |
custom |
Finds activities where an FS.event API or Defined Event occurs.
Triggers when: A call to FS.event |
event_name:string event_properties:json fs_api_errors.field:string fs_api_errors.reason:string fs_api_errors.value:string |
element_seen |
Finds activities where a watched element was rendered on the page and if it was visible in the user's viewport.
Triggers when: at least 25% of the watched element enters the user's viewport. |
element_render_duration_millis:number element_start_time:number element_start_type:string element_type:string element_visible_duration_millis:number target.masked:boolean target.raw_selector:string target.text:string |
exception |
Finds activities where a code exception occurred. Surfaced by its exception message. These are only uncaught exceptions for now.
Triggers when: Javascript exception occurs in the browser. Code exceptions on Android / iOS. |
exception_count:number exception_source_file:string is_exception_handled:boolean |
first_input_delay |
Finds the user's first activity on a web page and reports the delay between the user's input and the time it takes for the browser to respond. : https://developer.mozilla.org/en-US/docs/Web/API/PerformanceEventTiming is reported with type 'first-input' for the first time on a web page. |
first_input_delay_millis:number |
form_abandon |
Finds cases where a user started filling out a form but left before submitting their data.
Triggers when: a user navigates away from a page with a form filled out, but not submitted. |
target.masked:boolean target.raw_selector:string target.text:string |
highlight |
Find activities where a user highlights content on a page.
Triggers when: content is highlighted on a page |
target.masked:boolean target.raw_selector:string target.text:string |
identify |
A call to FS.identify or FS.setUserVars |
fs_api_errors.field:string fs_api_errors.reason:string fs_api_errors.value:string user_display_name:string user_email:string user_id:string user_properties:json |
keyboard_close |
The user's keyboard was closed
Triggers when: The keyboard is no longer visible after capturing keyboard visibility changed event (FSEventEventKeyboardVisibility) |
|
keyboard_open |
The user's keyboard was opened.
Triggers when: The keyboard is visible after capturing keyboard visibility changed event (FSEventEventKeyboardVisibility) |
|
load |
Finds activities where a user refreshes the URL using their browser refresh button or keyboard shortcut.
Triggers when: A user-initiated refresh of the current page |
dom_content_time_millis:number first_paint_time_millis:number load_time_millis:number largest_paint_time_millis:number |
low_memory |
Finds activities where data capture stopped due to low device memory.
Triggers when: On Android, this event is triggered when there is 10% of RAM available. On iOS, we listen for this as the trigger for this event. |
memory_available:number memory_maximum:number |
mouse_thrash |
Find activities where a user rapidly moves their cursor back and forth in a short period of time.
Triggers when: Rapid mouse movement within a short timeframe |
target.masked:boolean target.raw_selector:string target.text:string |
navigate |
Finds activities where a user navigates to a specific app screen.
Triggers when: User navigates to mobile app screen |
fs_suspicious_kind:array navigate_reason:string |
page_properties |
Define page names and additional properties related to the page (ie. Category, Cart Size).
Triggers when: A call to FS.setVars API |
fs_api_errors.field:string fs_api_errors.reason:string fs_api_errors.value:string page_name:string page_properties:json |
paste |
Finds cases where users are pasting text into a field.
Triggers when: User pastes text into a field, either via CTRL+V or right click |
target.masked:boolean target.raw_selector:string target.text:string |
pinch_gesture |
Finds activities where a user zoomed in or zoomed out using a touch device.
Triggers when: Multi-touch pinch event on a touch screen or multi-touch trackpad |
pinch_direction:number pinch_gesture_type:string pinch_scale:number pinch_visual_scale_change:number target.masked:boolean target.raw_selector:string target.text:string |
request |
Surfaces network request errors when any captured network request has a status code ≥ 400.
Triggers when: A captured network request has a status code ≥ 400 |
request_duration_millis:number request_method:string request_status:number request_url.full_url:string request_url.hash_path:string request_url.hash_query:string request_url.host:string request_url.path:string request_url.query:string |
These definitions are provided for informational purposes only. Please remember that you can only use this information in accordance with FullStory's terms.
Defined Events, Pages & Elements
FullStory now includes lookup tables in the warehouse for the defined events, pages, and elements that you create in the FullStory app so that you do not need to recreate these definitions in SQL logic.
The definition tables will be updated on each sync to reflect the current name and description set in the UI. As events are captured and processed by FullStory's event stream, we will evaluate the definitions of all active data objects at that point in time and enrich the event data with all matching object IDs. This object ID can be used to join back to the corresponding lookup tables for names and descriptions. The object ID can also be used to group and filter event data for analysis. For example, you could query for unique visits to your checkout page or for all click interactions with your "Sign Up" element.
Schema for Object Definitions
Object | Lookup Table | Object ID Schema in Events |
Events | event_definitions |
Most specific matching Event (used in FullStory UI analysis): event_properties.event_definition_id Additional IDs are available in the data to query other matching objects: event_properties.additional_event_definition_ids |
Pages |
page_definitions |
FullStory supports one and only one active page definition per URL source_properties.page_definition_id |
Elements | element_definitions |
Most specific matching Element (used in FullStory UI analysis). Element matching is available for all event_types that support the `target` object: event_properties.target.element_definition_id Additional IDs are available in the data to query other matching objects: event_properties.target.additional_element_definition_ids |
Due to the nature of the event stream, object IDs are only enriched from when they were created forward, similar to how FullStory reports activity in Journeys. This means that changing the definition of an existing object will only match applicable events from the time of that change forward. It will not apply retroactively to the historical data in your warehouse. See below for a diagram explaining the flow of how data is processed.
Example Data Flow Diagram for Defined Events
These lookup tables may introduce a many-to-1 relationship between events and object definitions. To account for this we have separated the most specific match, the one that will be used for analysis in the FullStory UI, from all other matching object definitions.
- For defined events and defined elements you will also see fields in the data for
additional_event_definition_ids
andadditional_element_definition_ids
respectively. Since multiple defined events can point to the same underlying event and multiple defined elements can point to the same underlying selector, we expose these so you are able to run different types of analyses in the warehouse from the behavior we apply in the FullStory UI. - For the FullStory UI we choose the object with the most specific definition that matches the data, then we aggregate the data based on that object alone (ignoring the additional matches to avoid duplicate counts). The ID that you see in
event_definition_id
andelement_definition_id
will reflect that most-specific ID that will be used for reporting in the FullStory UI - Multiple defined pages cannot point to the same URL
Note: This feature is available by default for all customers who activate Data Destinations after 9/11/23. If you activated Data Destinations prior to that date, please reach out to support to have this feature enabled.
FAQ
How often will data be synced between FullStory and my Data Destination of choice?
The FullStory pipeline syncs to your integrated warehouse on an hourly basis when your integration is active and correctly configured. Each sync contains the events processed by FullStory in the prior hour. Due to the distributed nature of the Internet and FullStory's capture pipeline, a complete set of events are not guaranteed to be available every hour. Any missing events will be reconciled automatically during subsequent syncs. We expose both event_time
and updated_time
columns in the data that can be used to tune your queries as needed.
Are there any plans to add other Data Destinations in the future?
Yes, we plan to add more destinations in the future based on customer demand. If there's a destination you'd like added, make sure to let us know.
I see more event types in the event filter list in the UI than I see in the data. Is something missing from the data?
No, All captured data is included in the Data Destination Sync. Some of the "events" that are shown in the UI are derived from the same underlying event_type
. For example, "Refreshed URL" in the UI is a navigate event_type
with event_properties:navigate_reason= 'reload'
in the warehouse.
There are a number of events that have a NULL event_type
. What are those records representing?
FullStory's data capture pipeline is heavily distributed, so there is no guarantee that events arrive (through our ingestion pipeline) in chronological order. As our pipeline processes these events, it associates them with a view_id
and orders them chronologically. If an event is synced to the warehouse and then FullStory receives a new event for the same view_id
with an older timestamp, both events will be updated so they are in order. This will create two new events in the warehouse and null out the previously synced event.
Is it possible to configure event types that we do not want to sync? For example, if we decide that we do not want "change" events, is it possible to not have those sent?
Currently, FullStory only supports syncing all captured events to your warehouse. Events that you are not interested in can easily be filtered out in your SQL analysis. We are exploring the possibility of adding event filters prior to the sync. If this is something you would be interested in, please reach out for more information.
I see two tables, one called events and another called raw_events
but raw events is empty. Is that expected?
Yes, the raw_events
table is used in the Snowflake data load process as a temporary staging table. This table may include some data while the hourly load process is running but should be empty at all other times. Consumers of the data can ignore this table and should only use the events table for operational queries.
Does Data Destinations support a Bi-directional sync?
Not yet, but we would love to hear more about any use cases you've got in mind or any workflows that you'd like to enable.
I see a slight discrepancy between FullStory UI metrics and warehouse data.
All timestamps in the warehouse are in UTC whereas the FullStory UI localizes metrics based on your browser's local timezone. To align these timezones, apply a timezone conversion from UTC to your local timezone in your SQL query.
Data Security Warning: If FullStory receives a data deletion request from a customer, and that customer's data has already been exported to their warehouse, FullStory will not be responsible for deleting the previously exported data, once it is out of FullStory's possession. Once the data has been synced to a customer's system, it is the responsibility of the customer to ensure the data is secure and that they adhere to all privacy, security, and other applicable regulations.