How to debug a aws lambda function?

Question

During this weekend i was running some node functions on aws lambda integrated with dynamodb, im using x-ray to debug it creating my own annotations, i like to know if there is a better approach to debug lambda functions something like a step-by-step?

Solution 1

Lambda local can be used to test the lambda code in local machine.

Solution 2

I think there is some misconception, pardon for switching from node.js context to python (the title is not specific to node so many folks will end up here anyway).

You CAN debug actual deployed AWS lambdas line-by-line. The approach involves running a debug server pydevd on the development machine and lambdas communicating with a debug-server over long-running TCP connection.

Is it supported by various IDEs, we use is with PyCharm Pro, so setup for it (hypothetically you should be able to setup VSCode similarly):

1. Debug configuration with Path mapping: right-top corner run/debug configuration dropdown -> Edit Configurations -> + (add new configuration) -> Python remote debug -> specify host name / port for your dev machine (which needs globally exposed: setup path forwarding on your router or use ngrok if previous option is not available). Make sure to setup path mappings, that is how your IDE can map the remote scripts to local sources.

2. pip install pydevd. (For pycharm, you will need to install IDE-version specific custom distribution like: pip install pydevd-pycharm~=193.5233.109)

3. Add the RemoteDebugSession context manager abstraction(since debug session is over long-running tcp connection, its needs to be closed explicitly in your lambda otherwise lambda will timeout) like:

# from config import config
from logging import getLogger
logger = getLogger(__name__)


class RemoteDebugSession:
    def __init__(self):
        self.active = False
        if not self.is_available():
            logger.warning(f"Remote debugging is not available")
            return

        try:
            # pydevd_pycharm exposes only settrace() from pydevd, import pydevd directly instead
            # import pydevd_pycharm
            import pydevd
            self.pydevd = pydevd
        except Exception as e:
            logger.warning(f"Remote debugging is unavailable")
            logger.warning(e)
            self.pydevd = None

    def __enter__(self):
        if not self.is_available() or self.pydevd is None:
            return

        self.pydevd.settrace(config.REMOTE_DEBUG_HOST, port=config.REMOTE_DEBUG_PORT,
                             suspend=False,
                             stdoutToServer=True,
                             stderrToServer=True)

        logger.warning("Starting remote dubugging session")
        self.active = True

    def __exit__(self, exc_type, exc_val, exc_tb):
        if not self.active:
            return

        if exc_type or exc_val or exc_tb:
            logger.warning(
                f"Remote debugging on {config.REMOTE_DEBUG_HOST}:{config.REMOTE_DEBUG_HOST} failed")
            logger.warning(exc_type)
            logger.warning(exc_val)
            logger.warning(exc_tb)
        else:
            logger.warning(f"Remote debugging on {config.REMOTE_DEBUG_HOST}:{config.REMOTE_DEBUG_HOST} closed")

        self.pydevd.stoptrace()

    @staticmethod
    def is_available():
        return hasattr(config, 'REMOTE_DEBUG_ENABLED') \
            and hasattr(config, 'REMOTE_DEBUG_HOST') \
            and hasattr(config, 'REMOTE_DEBUG_PORT') \
            and config.REMOTE_DEBUG_ENABLED \
            and config.REMOTE_DEBUG_HOST \
            and config.REMOTE_DEBUG_PORT

You need to have config object with REMOTE_DEBUG_ENABLED, REMOTE_DEBUG_HOST, REMOTE_DEBUG_PORT set. (pydevd.settrace is used with suspend=False so the executing won't stop unless we set a breakpoint)

5. Wrap your feature of interest with it like:

 with RemoteDebugSession() as session:
    # your code to examine
    pass

In practice, when building web servers, this can be inside an individual API handler, which will allow you to debug multiple simultaneous incoming requests as other lambdas will block on connection.

Please REFRAIN from using this in production. Apart from security risk, remote evaluation can allow you to do all sort of nasty things.

Solution 3

BothSAM Local and The Serverless framework provide tools to locally execute an AWS Lambda functions. In case you need to debug your function in the AWS environment, I suggest using Cloud9Cloud9 IDE for AWS Lambda debugging.

However, to debug invocation failures occurred in production, you could use something like Dashbird to detect and debug failures using logs, X-ray traces and other metadata.

Solution 4

You wont be able to do a step by step debug like you would on a regular program by attaching a debugger. As you mentioned you can use X-Ray and basically logging statements to figure out what went wrong.

As @ConfusedCoder pointed out there are ways in which you can run a lambda locally and debug it.

But also make sure you have enough logging in to ensure that you can try to figure out what went wrong, for future issues, using the logs as you would typically be looking at it after the actual execution happened.

Solution 5

Rookout offers live debugging of NodeJS on AWS Lambda. This gives you breakpoints on the production function to allow traditional step-debugging. There are also lots of hacky solutions that effectively run your Lambda locally (eg lambda.local, serverless framework) but if you want to debug with true inputs, then debugging it in its true environment using Rookout is a good option.

View more solutions