HTTP API - Using Segment Export

This API is only available for FullStory Enterprise plans. Please contact your Account Executive or reach out to Support to learn more.

FullStory’s new Segment Export API provides an asynchronous workflow via a set of API endpoints that can be used together to download recorded event data from FullStory’s servers. Learn more about the Segment Export API in our API docs.

FullStory offers an open-source tool, Hauser, that uses the Segment Export API to create export files, download them, and load them into a data warehouse. If you use Amazon's Redshift or Google's BigQuery as your data warehouse, we recommend you start with Hauser to see if it meets your requirements.

If you need to create your own Extract, Transform, & Load (ETL) pipeline for Segment Export data, you can follow the steps outlined below.

Workflow

The Segment Export API endpoints can be coordinated to perform the following steps:

  1. Initiate an export job to aggregate data from FullStory’s database.
  2. Query for the status of this job (including % completion).
  3. Once the job is done, get a URL to a data file stored in cloud storage.
  4. Download the file.

1. Initiate an export job

Using Create Segment Export, you can create a new segment export file that includes data from a designated segment. 

  • The segmentId can be located in the URL of the FullStory app when viewing that segment:

    image.png

    The segmentId in this example is: “johndoe@fullstory.com:5636470266134528"
    (You may also use List Segments to find the segmentId).
  • Indicate if you want data about Individuals (TYPE_INDIVIDUAL) or Events (TYPE_EVENT). 
  • You can also set preferred format (FORMAT_CSV, FORMAT_JSON, or FORMAT_NDJSON) and time range of the data.
  • Create an ‘Admin’ API key for Segment Export.
export FS_API_KEY=<your api key>

curl -X POST -H "Authorization: Basic ${FS_API_KEY}" "https://api.fullstory.com/segments/v1/exports" -d '{
 "segmentId": "everyone",
 "format": "FORMAT_CSV",
 "type": "TYPE_EVENT",
 "timeRange": {
   "start": "2020-10-13T00:00:00Z",
   "end": "2020-10-13T01:00:00Z"
 }
}'

Note: timeRange start and end values are in ISO 8601 format.

A successful response will appear as follows :

{"operationId": "abcdefg123"}

2. Query for the status of this job (including % completion)

Next, you will need to check the status of the operation created in the step above before continuing to Step 3.

Use the operationId from the prior response with the following command:

curl -H "Authorization: Basic ${FS_API_KEY}" 
"https://api.fullstory.com/operations/v1/{operationId}"

When an operation is still running, you will see a response such as this:

{
"type": "SEARCH_EXPORT",
"details": null,
"results": null,
"state": "PENDING",
"errorDetails": "",
"createdAt": "<timestamp>",
"finishedAt": null,
"estimatePctComplete": 10,
"step": ""
}

Note: The raw responses will not contain new lines. If pretty printed json is desired, jq is a popular command line tool that can be used to parse and print json.

When the operation has successfully completed, you will see a response such as this:

{
"type": "SEARCH_EXPORT",
"details": null,
"results": {
"expires": "<timestamp>",
"searchExportId": "efghijk456"
},
"state": "COMPLETED",
"errorDetails": "",
"createdAt": "<timestamp>",
"finishedAt": "<timestamp>",
"estimatePctComplete": 100,
"step": ""
}

If you see a state of “ERROR”, check the errorDetails provided, resolve the conditions, and restart the export with Step 1 above.

3. Get a url to a data file stored in cloud storage

Once the operation has successfully completed, you will then be able to retrieve a URL to the export file that is stored in cloud storage.

Use the `searchExportId` from the prior step in the following command:

curl -H "Authorization: Basic ${FS_API_KEY}" 
"https://api.fullstory.com/search/v1/exports/{searchExportId}/results" | jq

A successful response will appear as follows:

{
"location": "<a url>",
"expires": "<timestamp>"
}

Note: The URL will only be available until the expiration timestamp provided in the response. 

4. Download the file

You will then download the data export file from the URL provided in the previous step in a regular curl GET command (no auth header needed):

curl -JO <location>

Note: If you did not use jq in Step 3, the URL in the raw JSON output may contain encoded characters (for example, "&" will be encoded as "\u0026"). In this case, use the echo command within the curl command as follows:

curl -JO $(echo "<location>")

Optional:

With jq and xargs, you can use the following command to initiate the download automatically and save to name in the `Content-Disposition` header. This combines Step 3 and Step 4:

curl -H "Authorization: Basic ${FS_API_KEY}" 
"https://api.fullstory.com/search/v1/exports/{searchExportId}/results"
| jq '.location' | xargs curl -JO

Additional Resources

A Segment Export CLI (which uses the API methods in this document) can be used to test out Segment Export capabilities: https://github.com/patrick-fs/segment-export-cli.

Need to get in touch with us?

The FullStory Team awaits your every question.

Contact us