One of the most misunderstood aspects of Lambda is how throttling applies to async invocations. Or rather, how it doesn't! Every Lambda invocation has to go through its Invoke API [1], whether you're invoking the function directly or through an event source such as API Gateway or SNS. With the Invoke API, you can choose invocationType as either "RequestResponse" (i.e. synchronous) or "Event" (i.e. asynchronous). Synchronous invocationsWith synchronous invocations, throttling limits are checked to make sure you stay within:
However, this is not true for async invocations. Async invocationsWith asynchronous invocations, the Event Invoke Frontend service (see diagram below) accepts the request and passes it onto an internal queue. It does not check the concurrency limits and will succeed even if the function does not have the concurrency to process the request. But that's OK because it does not have to process the request right away, given the asynchronous nature of the invocation. Instead, concurrency limits are checked when the internal poller attempts to invoke the function synchronously. This means that you will never experience throttling when you invoke a function asynchronously. Even if you set the reserved concurrency to 0 - which will stop the function from running - the "Event" Invoke call will still succeed. But what happens when the internal poller invokes the function synchronously and the function is throttled? In that case, the invocation request is returned to the internal queue and is retried for up to 6 hours. This is described in the official documentation here [3]. Async invocations vs. Async event sourcesAnother important detail to consider is that async event sources such as SNS and EventBridge also invoke Lambda asynchronously. This means, even though they each offer a longer retry period:
But, because async invocations never fail due to throttling, so they count as successful deliveries for SNS and EventBridge. Lambda's Event Invoke Frontend service accepts the request, and any throttling errors will be retried for up to 6 hours ONLY. I asked about this on Twitter, and two of the principal engineers on the Lambda team confirmed my hypothesis above. See their responses here and here. So what?Why do these details matter? Quite a few of you have told me that you prefer SNS -> Lambda over a direct async Lambda invocation because it protects against throttling errors. Good news, given the above, you don't need the SNS topic! (unless you need it for fan-out) This is a good thing because:
You are welcome :-) This follows one of my most important architectural principles and I think you should follow it too. Aren't Lambda-to-Lambda calls an anti-pattern?Yes, synchronous Lambda-to-Lambda calls are an anti-pattern. However, there are valid use cases for asynchronous Lambda-to-Lambda calls. For example, when you offload secondary responsibilities (e.g. analytics tracking) from a user-facing API function to a second function and invoke it asynchronously. This is so that:
These benefits justify the extra cost of invoking a second function instead of doing everything in the API function. Links[1] Lambda's Invoke API [2] AWS re:Invent 2022 - A closer look at AWS Lambda (SVS404-R) [3] How Lambda handles errors and retries with asynchronous invocation |
Join 15K readers and level up you AWS game with just 5 mins a week.
Last week, we looked at 6 ways to version event schemas [1] and found the best solution is to avoid breaking changes and minimise the need for versioning. But how exactly do you do that? How can you prevent accidental breaking changes from creeping in? You can detect and stop breaking changes: At runtime, when the events are ingested; During development, when schema changes are made; Or a combination of both! Here are three approaches you should consider. 1. Consumer-Driven Contracts In...
Synchronous API integrations create temporal coupling [1] between two services based on their respective availability. This is a tighter form of coupling and often necessitates techniques such as retries, exponential delay and fallbacks to compensate. Event-driven architectures, on the other hand, encourage loose coupling. But we are still bound by lessor forms of coupling such as schema coupling. And here lies a question that many students and clients have asked me: “How do I version my...
ICYMI, Serverless Inc. recently announced the Serverless Container Framework. It allows you to switch the compute platform between Lambda and Fargate with a one-liner config change. This is a game-changer for many organizations! It'd hopefully nullify many of the "lock-in" worries about Lambda, too. As your system grows, if Lambda gets expensive, you can easily switch to Fargate without changing your application code. To be clear, this is something you can already do yourself. It's not a...