how to Put json data using Amazon API gateway to Kinesis stream

Question

I setup an API using amazon api gateway and want to put data into kinesis streams. Amazon api gateway has inbuilt support for it. But when i trying to put json data it gives "Serialization exception".

var data = {"ua_platform":"iPhone","ua_browsercodename":"Mozilla","ua_browserlanguage":"en-us","ua_header":"Mozilla\/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit\/601.1.46 (KHTML, like Gecko) Version\/9.0 Mobile\/13B143 Safari\/601.1","ua_browsername":"Netscape","key":"livestream_hindi",,"datetime_ut":"1458711871","datetime_dt":"2016-03-23","value":"15","source":"0","browser":"Mobile Safari-9.0.","os":"iOS-9.1.","device_detail":"iPhone Apple iPhone"};
    var json = JSON.stringify(data);    
var params = { 'ContentType' : 'application/json','Access-Control-Allow-Headers' : 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'};
    var body =  {"Data":json,
    "StreamName":"XXXXXX",
    "PartitionKey":"XXXX"
    };

After this i make a put request

apigClient.functionPut(params, body, additionalParams)
    .then(function(result){
        //This is where you would put a success callback
        console.log("success");
    }).catch( function(result){
        //This is where you would put an error callback
        console.log("catch");
    });

The api gives 200 ok along with the serialization exception. It is stated in put request the "Data" key in body variable will accept only "blob" type. Now i also tried converting JSON data to BLOB but no luck at all.

I am not able to figure out what i am doing wrong. Please help.

Solution 1

There's a walkthrough of how to set up API Gateway in front of Kinesis in the official AWS docs at http://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-kinesis.html

Solution 2

You must use the $utils variable to convert to base64 on your integration request template, here's my integration request template:

#set($msgBody = $util.parseJson($input.body))
#set($msgId = $msgBody.messageId)
{
    "Data": "$util.base64Encode($input.body)",
    "PartitionKey": "$msgId",
    "StreamName": "arena-hub-dev-ks"
}

See the line with the "Data" I'm using $util.base64Encode. In this case my APIGW endpoint accepts text/plain.

Comments

[nikhil khare]

Thanks for the Help. However i rectified the issue. I need to convert the stringified data to base64encoded data in order to push it into the stream. I used JAVASCRIPT:btoa() function and it works as expected.