Automating Omnichannel files upload with R or Pentaho Data Integration

[Ingo]

Hello all,

We are having problems to automate the upload of Omnichannel files to the S3 bucket with the tools we are usually using and I was wondering if anyone has experience with it.

First we tried to create the files directly with Pentaho Data Integration and then uploading pre-created files using the aws.s3 library in R.

Both seem to have problems, as they try to get some metadata from the bucket itself and only then they try to put the files into the folder we are supposed to use. Since our user has no access right to the bucket root we get errors and data/files are not uploaded.

Any help or hints would be greatly appretiated.

Many thanks in advance,

Ingo

[Ingo · Accepted solution]

We finally got it to work using AWS CLI called from a Pentaho Data Integration Job via Shell Script step. At first we struggled With AWS CLI, too, when trying to use the ENV-variable authentication method. But with the pointers from this document and using the profile-file approach for authentication all works fine now: 

https://community.tealiumiq.com/t5/Universal-Data-Hub/Working-with-AudienceStore-and-EventStore/ta-p...

Best regards,

Ingo