UTF-8 Encoding in Publish Settings - Help Page not available (dead link) and what is the effect?

[loldenburg]

In the Publish Settings under UTF-8 Encoding, I am sent to 

https://community.tealiumiq.com/t5/iQ-Tag-Management/Publish-Configuration/ta-p/13632#implementation

However I get the error that I cannot view this page. Can someone update the link in Tealium iQ please?

I have 2 questions regarding this:

1. Does this mean that publishes will encode "special" characters correctly in the JavaScript code (äöüßéè etc.) without having to use the Unicode signs for them anymore?

2. What happens to the code parts where Unicode signs are in use today? Do we have to change the Unicode characters to the UTF-8 characters or will they continue to work?

Thanks!

[kathleen_jo]

Hi @loldenburg. The link is now fixed. Thank you for bringing this to my attention. 

[loldenburg]

thanks, but the page does not answer my two questions.

[kathleen_jo]

Okay let me see if I can find you some assistance @loldenburg. 

[adrian_browning · Accepted solution]

@loldenburg

In answer to your questions:

---

1. Does this mean that publishes will encode "special" characters correctly in the JavaScript code (äöüßéè etc.) without having to use the Unicode signs for them anymore?  

---

 

Correct, you can have "special" characters in the JS now with no issues. Without having to use the workaround of unicode characters.

---

2. What happens to the code parts where Unicode signs are in use today? Do we have to change the Unicode characters to the UTF-8 characters or will they continue to work?

---

 

These will continue to act like they have done. You can either switch them over to their retrospective charater or quite happily leave them as they are.

Adrian

[loldenburg]

The great Adrian again, thank you!!!

[CesarFlorero]

@adrian_browning, will turning the UTF-8 encoding in Publish Configuration fix the issue I have with my UDO variables having Unicode special characters?

Here is an example:

section: türkmenistan --> needs to read --> section: türkmenistan

This issue is driving my insane, Can this also be fixed with extensions?

I appreciate all the help I can get.

Thanks,
Cesar

[UnknownJ]

Hi @CesarFlorero, you could do this in an extension. So for example, you might create an After Load Rules extension thus:

var decodeEntities = function(input){ return $('<textarea />').html(input).val(); };
var recodeEntities = function(input){ return encodeURIComponent(decodeEntities(input)); };
if(b.website_section){
  b.website_section = recodeEntities(b.website_section);
}

For the sake of being quick and easy, it needs jQuery to work, but it'll convert any HTML entities first to their unicode equivalents, and then it'll URLEncode them. You can skip the URL encoding if the tags you're mapping that data to will do that for themselves as part of building their respective querystrings, but it's left for the sake of completeness..

[loldenburg]

@UnknownJ's approach may work, but it is imho not the most browser-resource-friendly approach (moreover getting values by scraping them from form fields is something I would not recommend unless for a temporary "hack"). The UTF-8 support in the publish settings did the job for us, so try that.

[adrian_browning]

@CesarFlorero

If the text  section: türkmenistan is inside the utag.js or utag.#.js file then yes this will solve your issue, as the publish engine will keep the encoding the same. It will not however convert section: türkmenistan -->  section: türkmenistan.

If you are pulling from the page then, again this is determined from the pages encoding type. If the page is set to UTF-8 then again you should see the correct value.

Adrian

[CesarFlorero]

@adrian_browning @loldenburg @UnknownJ Thank you guys for the quick response. I am new to Tealium and I appreciate all the help I can get. 

To follow up on your response, I am attaching a couple of screen shots to better illustrate my dilemma. I have not made any changes to my Publish settings yet.

This is what my UDO looks like currently. Notice the unicode characters.

Strangely the UTAG Debugger tools shows this, notice how this "appears correct", but...

If I look at the Chartbeat pixel and try to validate the correct variable, it shows the special characters.

I hoping all of this can be resolve if I just update my Publish settings or is this more complecated than I thougth?

Again, Thanks for your help on this.

Cesar 

[adrian_browning]

@CesarFlorero

So looking at that I think your best option is to run the values through a function to convert, something like:

function convertHTMLCodeToString(htmlString){
  htmlString = htmlString || "";
  return htmlString.replace(/&#(\d+);/g, function(group, htmlNum){ return String.fromCharCode(htmlNum); }); 
}

Adrian 

[CesarFlorero]

@adrian_browning is your code Universal? meaning if I use a JS extension it will decode the characters that passed to all my tags or do I need to scope it. Does it need to be set to 'Pre loader" or "After Load Rules". Not really sure what step to take here.

Cesar

[adrian_browning]

@CesarFlorero

The code indeed is generic. You can paste it into a JS extension scoped to All Tags, then loop through the whole b object.

function convertHTMLCodeToString(htmlString){
    htmlString = htmlString || "";
    return htmlString.replace(/&#(\d+);/g, function(group, htmlNum){ return String.fromCharCode(htmlNum); }); 
}
for (var prop in utag.loader.GV(b)) {
    if (utag.ut.typeOf(b[prop]) !== "string") continue;
    b[prop] = convertHTMLCodeToString(b[prop]);
}

Adrian

[CesarFlorero]

Again, Thank you for all your help with this. :) What do you mean loop through the whole 'b' object, @adrian_browning? Could you share a screenshot of what this is exactly?

Cesar

[adrian_browning]

@CesarFlorero

Here is an example of how I would use this in iQ.

Making sure to select Execution: Before Load Rules, to make sure you can use these decoded strings in Load Rule evaluation.

Adrian

[loldenburg]

And why is the link in the publish settings STILL leading to a non-existing page 1 1/2 years later?
https://community.tealiumiq.com/t5/Archived-Posts/OLD-Publish-Configurations/m-p/2010#implementation
"ACCESS DENIED"

[loldenburg]

correction: 3 years later!

[KristenMeren]

Hi @loldenburg, 

Do you mind giving it a shot now? I just confirmed the URL redirect with our community platform provider, and tested it with success. Feel free to send me a private message if you see anything else amiss. 

Kristen