Importing Offline Data (Omnichannel)

Importing Offline Data (Omnichannel)

by on ‎08-27-2015 08:19 PM - edited on ‎10-30-2018 01:44 PM by (11,426 Views)

Omnichannel is a feature for importing offline data into the Universal Data Hub (UDH) where it enriches visitor profile data and becomes actionable within connectors.

Before you begin:

This guide has the following sections:

  • How It Works 
    Learn about the process: from the basic components and import process to the final results within the UDH.
  • Using Omnichannel 
    Step by step instructions on configuring your omnichannel imports.

Here's a quick overview of the omnichannel feature: 

This guide covers the following topics:

Table of Contents Placeholder

How It Works

The primary goal of an omnichannel import is to supplement online visitor profile data with valuable offline data. This is accomplished using two types of attributes: visitor and omnichannel. Visitor attributes define your visitor profiles and represent the unified view of your customer. Imported omnichannel attributes help complete this view of the customer via attribute enrichments.

An omnichannel attribute must be used in an enrichment of a visitor attribute in order for its data to be imported.

There are two components used in setting up an omnichannel file import: a file transfer service and a file definition.

File Transfer Service

The file transfer service is a secure location where you upload your files for Tealium to retrieve them. Tealium supports the following file transfer services:

  • Amazon S3 (Tealium bucket or your own bucket)
  • Microsoft Azure File/Blob Storage
  • FTP/SFTP

If using your own file transfer service, be sure to have the connection details ready before proceeding.

File Definition

A file definition is a detailed description of the CSV files to import. This includes the names of the files and the name of each CSV column (corresponds to a UDH attribute). Each file definition must also specify a visitor identifier that will appear in each row of the file. This identifier will link the imported data to visitor profiles in AudienceStream.

Visitor Enrichment

When a file is processed, the omnichannel attributes are used to create and update visitor attributes via enrichments. For this reason, it's important that your visitor attributes are configured to use the new omnichannel attributes in their enrichments.

For example, if you are importing offline purchase data and want to update a visitor lifetime order value, you will create an enrichment in the visitor attribute to leverage the omnichannel attribute.

  • Omnichannel Attribute: order_total_amount
  • Visitor Attribute: Lifetime Order Value

Resulting enrichment:

visitor-attribute-omnichannel-enrichment.png

Import Process

Once a file transfer service is configured with a file definition, you will upload a file to the service. The omnichannel system then follows this order of operations:

  1. Check for New Files
    The system checks the file transfer service for new files every 10 minutes.

    When multiple files are uploaded at the same time using SFTP or S3, the files are processed in the order of the upload timestamp.

  2. Copy New Files
    When a new file is detected it is copied from the file transfer location and processed in the UDH.
  3. Match Filename Prefix to File Definition
    The prefix of the filename is used to identify which file definition to use when importing the data in the file. The file definition is used to identify which visitor ID attribute to use and which data attributes to import.
  4. Process Files
    The header line is read to identify the attributes being ingested. From there the following processing is performed:
    • Visitor Lookup: the visitor ID is used for a lookup of the visitor record in AudienceStream. If an existing visitor record is not found a new one is created.

      Grouping rows with the same visitor ID will increase the speed of the import.

    • Attribute Enrichment: the visitor record will be enriched according to the attributes imported and the existing enrichments in your account.
    • EventStream/DataAccess Processing: If configured to do so, the imported data will be made available to EventStream connectors and stored in DataAccess.

Using Omnichannel

Setting up an omnichannel enrichment requires you to configure a file transfer service and a file definition. To begin setting up an omnichannel enrichment, go to Sources > Omnichannel and click + Add Enrichment. The omnichannel dialog will appear with a two-step process to configure a file transfer service and create a file definition.

Adding a File Transfer Service

To configure your file transfer service:

  1. Select a Service.
    The remaining fields will change depending on the service selected.
    Select Service

    uploading file to s3.png

  2. Enter a Title to identify this enrichment, preferably by its channel e.g. "In-Store Transactions"
  3. Enter the credentials necessary to access the service you selected.
    If you selected Tealium S3 Bucket, then the necessary credentials are generated for you automatically.
  4. (Optional) Enter notes to describe the enrichment.
  5. Click Next to proceed to file definitions.

Adding a File Definition

A file definition identifies the names of the columns in the CSV file. These column names automatically generate omnichannel attributes with the same names. In the next section these attributes are used to enrich visitor attributes.

To set up a file definition and define omnichannel attributes:

  1. Click + Add File Definition.
    The file definition form appears.
  2. Send to DataAccess and EventStream Connectors - Check this box if the imported data should be made available to EventStream connectors and DataAccess storage.
  3. Unique File Prefix - Enter a prefix to identify these files.
    This prefix identifies which files on the file transfer service to retrieve.
  4. Visitor ID - Select a Visitor ID attribute from the Map drop-down.
  5. In the From text field, enter the name of the corresponding column from the CSV file.
  6. Omnichannel Attributes (CSV Column Names) - Enter the name of each column to import from the file.
    Each column creates a new omnichannel attribute with the same name, which is made available for use in rules, enrichments, and streams.

    Column names are case sensitive and may not contain "#", "^", or whitespace characters.

    • Date Format - Columns with date/time values must have a matching date format setting. Click the calendar icon and select a data format e.g. "yyyy-MM-dd"
    • Restricted Data - Check this box to mark a column as having restricted data and to exclude it from other Tealium services (for more information, see What is Restricted Data?).
  7. Click Save.
    omnichannel-file-definition.png

Download/Copy Sample CSV 

To see an example of what your CSV file should look like based on the file definition, click Download CSV or Copy CSV. The resulting file will contain two rows of data, the header line with the column names and one row of placeholder data.

Example sample CSV:

"Email","OrderID","OrderDate","OrderTotal","TotalProductSpend","TotalShippingCharges"
"example@email.com","yyyyMMdd","value_1","value_2","value_3","value_4"

Using Omnichannel Attributes

The next step is to configure your visitor attributes with enrichments based on the omnichannel attributes. Omnichannel attributes appear in the attribute drop-down menus for rules, enrichments, and streams.

To view the list of existing omnichannel attributes:

  1. Navigate to EventStream > Attributes.
  2. In the side bar, select the Omnichannel scope.
    omnichannel-attributes-list.png

To enrich a visitor attribute with an omnichannel attribute:

  1. Navigate to AudienceStream > Attributes.
  2. In the side panel, under Scope, select Visitor.
  3. Select a visitor attribute to enrich.
  4. Add an enrichment.
    The omnichannel attributes are available from the attribute drop-down menus for use within the enrichment.
    omnichannel-attributes-drop-down.png

Uploading Omnichannel Files

The final step is to upload your CSV files. You will require a third-party application to initiate the upload. We recommend Cyberduck because it’s free and supports FTP and Amazon S3, but you may use any client for this purpose.

To upload a file via FTP or Amazon S3 using Cyberduck:

  1. Install (or launch) Cyberduck.
  2. Create a new connection and give it a title.
  3. From the first drop-down list, select the file transfer service used in your omnichannel configuration.
  4. Enter the credentials (username and password) for your service.
    • My FTP Connection and My SFTP Connection
    • Tealium S3 Bucket - the credentials are pre-populated as Access Key (username) and Secret Key (password).
  5. Provide other details, such as the server, path, port, etc, required by your service.
    Read more about accessing third-party S3 buckets using Cyberduck.
  6. Save the connection.

You are now ready to upload files by dragging and dropping your CSV files into Cyberduck.

When using S/FTP, the file must be located in the root folder. Omnichannel cannot locate files in the sub-folders of an S/FTP connection.

Additional Resources

Frequently Asked Questions

What if the file transfer service fails?

If AudienceStream fails to copy a file via the file transfer service it will simply wait 10 minutes and try again.

What if a file failed to be processed?

When a process failure occurs, the file will be ignored and not attempted to be processed again. Please submit a Support Desk request to get assistance diagnosing a problem with a failed import.

File processing can fail for one of the following reasons:

  • the CSV file is improperly formatted eg. not a valid CSV file
  • the column names used in the File Definition do not exist in the file