How to read data from s3 bucket to Kinesis Stream

Question

I am new to Kinesis. I want to read data from s3 bucket to Kinesis stream. I have created a kinesis stream called 'mystream'. In my project every 3 month a record will be uploaded to the s3. At the same time I need to read the record to kinesis stream. I have created Lamda for reading the file and save it into the RDS DB instance. I found in lambda project there is one kinesis event. Is this the way to do the thing? please help me to get this. I don't know how to start it and all.. It would be great if some one can help me....

Solution

If my understanding is right your question is about how to write s3 object data into kinesis.

You can create an event Notification in S3 for lambda. In events, you can select the event (ObjectCreate (All), put, post). After Successful creation for each object create action you get notifications in lambda. In lambda event context you get the s3 object information of each invocation.

Now you can read about the object information and write it in kinesis stream.

Links for AWS S3 and Kinesis Stream using Java

https://docs.aws.amazon.com/AmazonS3/latest/dev/RetrievingObjectUsingJava.html

https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-sdk.html

Comments

[ShaiNe Ram]

Yes you are right!!! The first part you mentioned is done. But how could I read the object information and how do i write it into the kinesis stream using java? will you please help me to get thi?