Api gateway get output results from step function?

Question

I followed tutorial on creating and invoking step functions

I'm getting output in my GET request of api as

 {
  "executionArn": "arn:aws:states:ap-northeast-1:123456789012:execution:HelloWorld:MyExecution",
  "startDate": 1.486772644911E9
}

But, instead of above response I want my step functions output, which is given by end state as below.

{
   "name":"Hellow World"
}

How to achieve this?

Solution 1

Update: You can now use Express Step Functions for synchronous requests.

AWS Step Functions are asynchronous and do not immediately return their results. API Gateway methods are synchronous and have a maximum timeout of 29 seconds.

To get the function output from a Step Function, you have to add a second method in API Gateway which will call the Step Function with the DescribeExecution action. The API Gateway client will have to call this periodically (poll) until the returned status is no longer "RUNNING".

Here's the DescribeExecution documentation

Solution 2

New Synchronous Express Workflows for AWS Step Functions is the answer: https://aws.amazon.com/blogs/compute/new-synchronous-express-workflows-for-aws-step-functions/

Amazon API Gateway now supports integration with Step Functions StartSyncExecution for HTTP APIs: https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-api-gateway-supports-integration-with-step-functions-startsyncexecution-http-apis/

Solution 3

Use Express Step Functions instead. This type of Step Functions can be called synchronously. Go to your API Gateway and in the Integration Request section make sure you have the StartSyncExecution action:

After that, go a bit lower in the same page to the Mapping Templates:

and include the following template for the application/json Content-Type:

#set($input = $input.json('$'))
{
   "input": "$util.escapeJavaScript($input)",
   "stateMachineArn": "arn:aws:states:us-east-1:your_aws_account_id:stateMachine:your_step_machine_name"
}

After that, go back to the Method Execution and go to the Integration Response and then to the Mapping Templates section:

And use the following template to have a custom response from your lambda:

#set ($parsedPayload = $util.parseJson($input.json('$.output')))
$parsedPayload

My testing Step Function is like this:

And my Lambda Function code is:

Deploy your API Gateway stage.

Now, if you go to Postman and send a POST request with any json body, now you have a response like this:

Solution 4

First of all the step functions executes asynchronously and API Gateway is only capable of invoking the step function (Starting a flow) only.

If you are waiting for the results of a step function invocation from a web application, you can use AWS IOT WebSockets for this. The steps are as follows.

• Setup AWS IOT topic with WebSockets.
• Configure the API Gateway and Step functions invocation.
• From the Web Frontend subscribe to the IOT Topic as a WebSocket listener.
• At the last step (And in error steps) in the Step Functions workflow use AWS SDK to trigger the IOT Topic which will broadcast the results to the Web App running in the browser using WebSockets.

For more details on WebSockets with AWS IOT refer the medium article Receiving AWS IoT messages in your browser using websockets.

Solution 5

Expanding on what @MikeD at AWS says, if you're certain that the Step Function won't exceed the 30 second timeout, you could create a lambda that executes the step function and then blocks as it polls for the result. Once it has the result, it can return it.

It is a better idea to have the first call return immediately with the execution id, and then pass that id into a second call to retrieve the result, once it's finished.

View more solutions

Comments

[Neo]

Darn. I guess I'll go back to embedded lambda calls. It would be nice if there was a synchronous checkbox, but I'm not sure how AWS would do that without having a parent lambda handling the initial http request and response.

[Memke]

DescribeExecution is not supported by EXPRESS state machines

[myroslav]

example lambda: gist.github.com/andrhamm/3db24c60fa320617217e5a3c84a7a2e3

[Tim Mironov]

Thanks for this info! What happens when the step function is not executed fast enough to stay inside the API requirements?

[Pooya Paridel]

Similar to when API Gateway is executing a Lambda function that has a bigger timeout. If you have something that needs more processing time, the Async workflow is more suitated. Create an endpoint to accept a request and another endpoint to get the status of process.

[Dima Lituiev]

how do you call "with the DescribeExecution action"? I can't find it in the documentation url.

[Jack Marchetti]

This should be the answer now.

[Alina Striltsova]

Sorry, how to add 'a secodn method in API Gateway'?