Manual event calls for Tealium_Collect / Datahub / Audience Stream

[Michael_Kim_shc]

Similar to how Google Analytics has it's measurement protocol for sending events, and allows custom/manual/automated events to be sent via that - is there a similar option for DataHub?  

I'm hoping a solution may be available that allows for faster processing of data than the omnichannel import, where instead of loading a file with 100k rows, I can run a script that makes 100k server calls ( with maybe a few milliseconds between each one?)

Otherwise the 1M omnichannel upload will be somewhat limiting for larger data sets and profile updates.

[dan_george · Accepted solution]

Hey @Michael_Kim_shc, check out the visitor Data Enrichment API.

[Michael_Kim_shc]

Perfect!  Thanks Dan.

[Michael_Kim_shc]

@dan_george

I've tried using the code provided, and fixed a bug so the python code running is now the below - and i'm getting a 200 OK response - but i'm looking at the trace in DataHub and not seeing anything happening.

Also - i'm not seeing any authentication tokens anywhere - is this an open endpoint that doesn't require any kind of authentication other than account and profile name?

import httplib, urllib
params = urllib.urlencode({'data':{'event_name':'myEvent','cp.trace_id':'07244'},'event':'view'})
headers = {"Content-type": "application/json","Accept": "image/gif"}
conn = httplib.HTTPSConnection("collect.tealiumiq.com")
conn.request("POST", "/{myaccount}/{myprofiel}2/i.gif", params, headers)
response = conn.getresponse()
print response.status, response.reason
data = response.read()
conn.close()

[dan_george]

@Michael_Kim_shc

Can you try going to Discover > Live Events and checking if you see the event feed in there? I'm curious if the UDH isn't receiving the data at all, or if passing the Trace ID is incorrect.

I know for a standard GET call it works by passing in the query string so you may think about going that route (following examples at the top of the previously linked article) instead of a POST if you don't have to pass much data.

Regarding authentication, this is an open endpoint API.

[Michael_Kim_shc]

Going through the live Events view is going to be rather unwiedly because of the amount of live traffic.  Unless there's a way to filter that view?

[Michael_Kim_shc]

Arg.  nevermind I see the add stream by trace now..

[dan_george]

@Michael_Kim_shc you can also take an existing data variable and pass through a unique value like "page_name":"sundayfunday"

[Michael_Kim_shc]

Still seeing anything via trace ID, nor a couple different event Attributes I've tried.

Also - in the help doc code, i had to change the help doc code to:

conn = httplib.HTTPSConnection("collect.tealiumiq.com")

(instead of HTTP) and removed the https:// from the URL in order for python to parse.

[dan_george]

@Michael_Kim_shc I suggest you reach out to your Deployment/Account manage who can align you with an Engineer to help look into this further.

[kathleen_jo]

Hello @Michael_Kim_shc. I am trying to find your account manager now. Stay tuned...

[aniket_mane]

I am also trying to do a exact same thing. Were you able to solve this by contacting account manager and have an engineer look at it.

Also I had one questions about trace_id. Can the trace_id be any number or should I start one trace in UDH and for that trace I would POST data to the endpoint ?

[joshua_mesick]

Hey @aniket_mane, I believe you need to create a trace ID from the UDH and then pass that within the vdata call. From there you would then create a stream filtering for the trace ID and you can see the events flow in that way. If you are having difficulties please let me know what specifically is happening so I can investigate further or by all means use our support portal via https://support.tealiumiq.com.

[craig_rouse]

@aniket_mane Josh is correct - you must create the trace ID within the UDH console - it cannot be an arbitrary number. Additionally, the data source name that you pass on your vdata pixel must be "tealium_trace_id", not just "trace_id" e.g.

collect.tealiumiq.com/vdata/i.gif?tealium_account=myaccount&tealium_profile=myprofile&tealium_vid=myemail%40tealium.com&page_name=homepage&tealium_trace_id=53124

[aniket_mane]

Thanks Josh.

[aniket_mane]

Thanks Craig. I will keep that in mind

[aniket_mane]

Hello All,

I tried doing this with the below steps

1) Started a new trace in AS UI
2) Used the POST Python code on site and did some modification based on this feedback on this post to get a 200 OK response. 
import httplib, urllib
params = urllib.urlencode({'data':{'event_name':'abc_score','tealium_trace_id':'09098','abc_score':'0.56','customer_id':'9999999'},'event':'view'})
headers = {"Content-type": "application/json","Accept": "image/gif"}
conn = httplib.HTTPSConnection("collect.tealiumiq.com")
conn.request("POST", "/<account>/<profile>/2/i.gif", params, headers)
response = conn.getresponse()
print response.status, response.reason
data = response.read()
conn.close()

​3) We did DISCOVER --> TRACE . Nothing is there . It just says LISTENING events. We did DISCOVER --> LIVE STREAM --> 0 events etc.

Could anyone help out here.

Thanks,
Aniket

[Michael_Kim_shc]

Have you tried starting a trace in chrome, doig someo web activity that gets included, (1 pageload), copy the network request sent to tealium-collect and replicate that request in your python?
If that works, then you can narrow down the difference in network calls (python vs web) until you find the offender.

[craig_rouse]

@aniket_mane Apologies - I wasn't aware you were using the "POST" api - I thought you were using GET. Try changing "tealium_trace_id" to "cp.trace_id" and it should work (I just tested it myself). tealium_trace_id is the required variable name for the GET API (vdata), but it's different for POST.

Here's an example in JavaScript&colon;

var xhr = new XMLHttpRequest();
xhr.open('POST', "https://collect.tealiumiq.com/account/profile/2/i.gif", true);
var json_string = JSON.stringify({
"data": {
"cp.trace_id" : "01559",
"test" : "test"
},
"event" : "view"
});
xhr.send(json_string);

[aniket_mane]

Hi @craig_rouse and @Michael_Kim_shc,

We tried using "cp.trace_id" in Python but no success. The code gave 200 OK response but nothing in AudienceStream UI . In the DISCOVER --> LIVE EVENTS nothing was coming(it was saying LISTENING EVENTS). Then I started a trace using chrome extension and put the trace id from website . I visited website and did activities. We started seeing the visit events but not the event fired from our python code using vdata enrichment.

Is there any other way you tested it to make sure it worked even after getting 200 OK response ?

Thanks,
Aniket

[craig_rouse]

@aniket_mane I have tested, and can only get the JavaScript snippet working - the python code sample does not seem to be working correctly. I suspect this is due to the url encoding of the JSON data, but need to do some more testing to verify. If you are able to test the JavaScript snippet above, and replicate that with Python, then you will start to see requests. If I figure out what's broken in that Python snippet, I'll post back here.

[craig_rouse]

Hi @aniket_mane,

I have got this working, but I'm using the Python "Requests" library. Please see snippet below:

import requests, json
body = {'data':{'event_name':'abc_score','cp.trace_id':'04759','abc_score':'0.56','customer_id':'9999999'},'event':'view'}
headers = {"Content-Type": "text/plain","Accept": "image/gif"}
r = requests.post('https://collect.tealiumiq.com/<account>/<profile>/2/i.gif', data = json.dumps(body), headers=headers);

Please note, I'm not a Python expert (far from it!), so please don't inspect the code too closely :-)

I have seen data in Live Events and also in Trace.

If you don't have the requests library installed, you can install through pip:
Python2:
pip2 install requests

Python3:
pip3 install requests

Let me know how you get on.

Craig.

[aniket_mane]

Awesome Craig. let me check it out and see if it works or not. Will post back soon

[aniket_mane]

Hi Craig,

The python code you gave worked like a charm. Thank you. I had few other questions related to it.

1) Is there a RATE LIMIT to the open endpoint . We are planning to send millions of events every 3 hours to it in BULK
2) The trace showed "Visitor stitched" - Found a visitor with matching id. But in the data object we send there was only customer_id and other fields and we did not specify any matching rules. On what basis AS stitchted it as we are firing it from the code and not from our website by visiting different pages.
3) If you were to use JAVASCRIPT way to send 3 million fire to this pixel , how would you do it ?

Thanks again for your help on this

[craig_rouse]

@aniket_mane I'm glad the code worked for you. I'm not completely sure what was wrong with the old code, but this seems more compact in any case.

There is no rate limit as such, but I would recommend speaking to your account manager before you begin so that we can keep an eye on things at the backend, and also to check that you are covered for all the events you intend to use.

Regarding stitching, if you have an attribute defined, of type "Visitor ID", and you populate this field in the request, as soon as a matching visitor ID is found on a subsequent request, the data will be stitched to the existing visitor profile. Thus, if "customer_id" is specified in the request, as soon as you make a second request with the same customer ID, the visitor stitching will kick in. Because you are making the request from Python, we are not able to use cookies to match the visitor, so you must have a consistent visitor ID on every request for stitching to function correctly.

If you really wanted to use JavaScript instead of Python, it would be best to set up a Node.js server or similar, and run the code from there. I would not recommend running this number of requests through a web browser.