profile

Master Serverless

AppSync's new async Lambda resolver is great news for GenAI apps

Published over 1 year ago • 1 min read

​

A common challenge in building GenAI applications today is the slow performance of most LLMs (except ChatGPT 4-o and Groq). To minimize delays and enhance user experience, streaming the LLM response is a must.

As such, we see a common pattern emerge in AppSync:

  1. The caller makes a GraphQL request to AppSync.
  2. AppSync invokes a Lambda resolver.
  3. The Lambda function queues up a task in SQS.
  4. The Lambda resolver returns so that AppSync can respond to the caller immediately. In the meantime, a background SQS function picks up the task and calls the LLM.
  5. The caller receives an acknowledgement from the initial request.
  6. The background function receives the LLM response as a stream and forwards it in chunks (as they are received) to the caller via an AppSync subscription.

This workaround is necessary because AppSync could only invoke Lambda functions synchronously. To support response streaming, the first function has to hand off calling the LLM to something else.

AppSync now supports async Lambda invocations.

On May 30th, AppSync announced [1] support for invoking Lambda resolvers asynchronously.

This works for both VTL and JavaScript resolvers. Setting the new invocationType attribute to Event will tell AppSync to invoke the Lambda resolver asynchronously.

Here's how the VTL mapping template would look:

And here's the JavaScript resolver:

The response from an async invocation will always be null.

The new architecture

With this change, we no longer need the background function.

  1. The caller makes a GraphQL request to AppSync.
  2. AppSync invokes a Lambda resolver asynchronously.
  3. AppSync immediately receives a null response and can respond to the original request.
  4. The Lambda function receives the LLM response as a stream and forwards it in chunks (as they are received) to the caller via an AppSync subscription.

This is a simple yet significant quality-of-life improvement from the AppSync team.

It's not just for GenAI applications. The same pattern can be applied to any long-running task requiring more than AppSync's 30s limit.

Links

[1] AWS AppSync now supports long running events with asynchronous Lambda function invocations​

Master Serverless

by Yan Cui, AWS Serverless Hero

Join 17K readers and level up you AWS game with just 5 mins a week.

Read more from Master Serverless

AppSync doesn’t allow unauthenticated API calls. To allow users to call your API without first authenticating themselves, you must mimic the behaviour using one of the available authorization methods [1]. In this post, let’s look at three ways to implement unauthenticated GraphQL operations with AppSync and their pros & cons. API Keys To use API keys, you need to: Add an API Key in AppSync. Pass the API Key in the x-api-key header. That’s it! It’s the easiest and most common way to implement...

2 months ago • 2 min read

A common challenge when building APIs is supporting multiple live versions without letting the system turn into an unmaintainable mess. You need to keep older versions running for existing clients, roll out new versions safely, and avoid breaking changes that might take down production. And you need to do all that without duplicating too much infrastructure or introducing spaghetti logic inside your code. There’s no official pattern for versioning APIs in API Gateway + Lambda. API Gateway...

4 months ago • 3 min read

I recently shared six event versioning strategies for event-driven architectures [1]. In response to this, Marty Pitt reached out and showed me how Orbital [2] and Taxi [3] use semantic tags to eliminate schema coupling in event-driven architectures and simplify the schema management. It's a novel way to manage schema evolution, and I want to share what I learnt with you. Problems with Schema Coupling In an event-driven architecture, event consumers are typically coupled to the schema of the...

5 months ago • 2 min read