Here is one of the most misunderstood aspects of AWS Lambda


One of the most misunderstood aspects of Lambda is how throttling applies to async invocations. Or rather, how it doesn't!

Every Lambda invocation has to go through its Invoke API [1], whether you're invoking the function directly or through an event source such as API Gateway or SNS.

With the Invoke API, you can choose invocationType as either "RequestResponse" (i.e. synchronous) or "Event" (i.e. asynchronous).

Synchronous invocations

With synchronous invocations, throttling limits are checked to make sure you stay within:

  • The regional concurrency limit, and;
  • The function's reserved concurrency.

However, this is not true for async invocations.

Async invocations

With asynchronous invocations, the Event Invoke Frontend service (see diagram below) accepts the request and passes it onto an internal queue. It does not check the concurrency limits and will succeed even if the function does not have the concurrency to process the request. But that's OK because it does not have to process the request right away, given the asynchronous nature of the invocation.

Instead, concurrency limits are checked when the internal poller attempts to invoke the function synchronously.

This means that you will never experience throttling when you invoke a function asynchronously.

Even if you set the reserved concurrency to 0 - which will stop the function from running - the "Event" Invoke call will still succeed.

But what happens when the internal poller invokes the function synchronously and the function is throttled?

In that case, the invocation request is returned to the internal queue and is retried for up to 6 hours. This is described in the official documentation here [3].

Async invocations vs. Async event sources

Another important detail to consider is that async event sources such as SNS and EventBridge also invoke Lambda asynchronously.

This means, even though they each offer a longer retry period:

  • EventBridge retries failed deliveries for up to 24 hours.
  • SNS retries failed deliveries for up to 23 days.

But, because async invocations never fail due to throttling, so they count as successful deliveries for SNS and EventBridge. Lambda's Event Invoke Frontend service accepts the request, and any throttling errors will be retried for up to 6 hours ONLY.

I asked about this on Twitter, and two of the principal engineers on the Lambda team confirmed my hypothesis above. See their responses here and here.

So what?

Why do these details matter?

Quite a few of you have told me that you prefer SNS -> Lambda over a direct async Lambda invocation because it protects against throttling errors.

Good news, given the above, you don't need the SNS topic! (unless you need it for fan-out)

This is a good thing because:

  • Fewer moving parts.
  • Fewer things to pay for.
  • One less place where things can go wrong (e.g. delivery problem from SNS to Lambda).

You are welcome :-)

This follows one of my most important architectural principles and I think you should follow it too.

Aren't Lambda-to-Lambda calls an anti-pattern?

Yes, synchronous Lambda-to-Lambda calls are an anti-pattern.

However, there are valid use cases for asynchronous Lambda-to-Lambda calls.

For example, when you offload secondary responsibilities (e.g. analytics tracking) from a user-facing API function to a second function and invoke it asynchronously.

This is so that:

  • The user-facing API function can respond to the user quicker and improve user experience.
  • We get built-in retries + DLQ/failure destination support for the second, asynchronously invoked function.

These benefits justify the extra cost of invoking a second function instead of doing everything in the API function.

Links

[1] Lambda's Invoke API

[2] AWS re:Invent 2022 - A closer look at AWS Lambda (SVS404-R)

[3] How Lambda handles errors and retries with asynchronous invocation

Master Serverless

Join 15K readers and level up you AWS game with just 5 mins a week.

Read more from Master Serverless

Last week, we looked at 6 ways to version event schemas [1] and found the best solution is to avoid breaking changes and minimise the need for versioning. But how exactly do you do that? How can you prevent accidental breaking changes from creeping in? You can detect and stop breaking changes: At runtime, when the events are ingested; During development, when schema changes are made; Or a combination of both! Here are three approaches you should consider. 1. Consumer-Driven Contracts In...

Synchronous API integrations create temporal coupling [1] between two services based on their respective availability. This is a tighter form of coupling and often necessitates techniques such as retries, exponential delay and fallbacks to compensate. Event-driven architectures, on the other hand, encourage loose coupling. But we are still bound by lessor forms of coupling such as schema coupling. And here lies a question that many students and clients have asked me: “How do I version my...

ICYMI, Serverless Inc. recently announced the Serverless Container Framework. It allows you to switch the compute platform between Lambda and Fargate with a one-liner config change. This is a game-changer for many organizations! It'd hopefully nullify many of the "lock-in" worries about Lambda, too. As your system grows, if Lambda gets expensive, you can easily switch to Fargate without changing your application code. To be clear, this is something you can already do yourself. It's not a...