First impressions of the fastest JavaScript runtime for Lambda
First impressions of the fastest JavaScript runtime for Lambda Read on my blog Read time: 4 minutes I thought Lambda needed a specialised runtime. One that works well with its resource-constraint execution environment. I even floated a few ideas in the past but sadly I don’t have the chops to make them happen myself. So I was pleasantly surprised when AWS open-sourced the LLRT runtime for JavaScript [1]! What is LLRT?LLRT, or Low Latency Runtime, is a new and experimental JavaScript runtime for Lambda. It promises 10x faster startup time. Which should significantly help with the dreaded Lambda cold starts. Naturally, I had to test it out for myself and see if the hype was real. My first impressions of LLRTIn my limited tests, the Along the way, I also discovered several limitations:
LLRT is not ready for prime time yet. And given that it’s been in the works for almost two years, we should set expectations accordingly. It’s probably not going to be production-ready in the coming months. But the potential is there and I’m really excited about it! For one thing, it might finally stop people talking about cold starts. As I said on LinkedIn [5] yesterday, most people are overthinking about Lambda cold starts. If you’re not sure if Lambda cold starts is likely a problem for you, then go read the LinkedIn post and come back here. Back to LLRT. I later spoke with Richard Davison, the creator of the LLRT project. I wanted to learn more about the project and the design decisions behind it. What made it start so much faster? What trade-offs did they make? LLRT is the answer to the question “What would a purposely built JavaScript runtime look like for Lambda.” A lot of people see that “It’s built in Rust” and automatically assume that’s why it’s fast. But it’s more than that. When it comes to performance optimizations, it always boils down to what you let go. If you choose to do everything the incumbent does, then you won’t make any significant performance gains. No JIT compilationWith LLRT, they chose to not include a JIT compiler. Because it’s focused on Lambda’s resource constraint and short-lived execution environments. As a result, LLRT is likely less performant than the Node.js runtime for CPU-intensive tasks. However, most Lambda functions do not perform CPU-intensive tasks. Instead, they tend to be IO-intensive. And the Lambda execution environments are short-lived. So a JIT compiler would have been less effective at optimizing hot code paths. At the same time, a JIT compiler comes with significant startup costs. It also introduces occasional latency spikes when it needs to evict cached items. So it appears a sensible trade-off for LLRT to not include a JIT compiler. QuickJs + RustThe QuickJs engine [6] plays a crucial role in LLRT and its outstanding performance. Another reason why LLRT is fast is because they wrote all the APIs in Rust. As much as possible, the team wants to stay in native code to guarantee a strong performance. Bun [7] took the same approach and implemented all the APIs in a system language called Zig [8]. The downside to this approach is that it’s harder for contributors to get in on the action. There are a lot fewer Rust developers than Node.js developers. And even fewer Rust developers who are interested in a JavaScript runtime. LLRT vs Bun (and other JS runtimes)LLRT is different from other JavaScript runtimes. It’s not a general-purpose runtime for JavaScript. It doesn’t have to worry about running in the browser or on mobile phones. Instead, it’s solely focused on the Lambda execution environment. This allows them to make decisions that just wouldn’t make sense with Bun or Node.js. Decisions such as not including a JIT compiler. Or which of the JavaScript APIs do they implement first, or at all? The goal is to eventually become WinterCG compliant. But we don’t have to wait for that to start using LLRT. For LLRT to be useful (but not perfect!), it just needs to support the AWS SDK and a few popular libraries. SummaryTo summarize, LLRT is an exciting new runtime for JavaScript. It’s purposely built for Lambda. It’s not intended as a general-purpose runtime for JavaScript. It makes design trade-offs (such as not having a JIT compiler) to achieve an optimal cold start time. It uses the QuickJs engine and implements all the JavaScript APIs in Rust. To learn more about LLRT and how to contribute to it, please check out my conversation with Richard on YouTube [9]. I will be covering LLRT in my upcoming workshop, including how to use it with the Serverless Framework and CDK. If you wanna take your serverless game to the next level, then you should check it out! More information is available on the course page [10]. Links[1] GitHub repo for LLRT [3] LLRT’s API compatibility page [4] AWS Lambda Powertools for TypeScript [5] Most people are overthinking about Lambda cold starts [6] QuickJs engine [7] The Bun runtime for JavaScript [8] The Zip programming language [9] My interview with Richard Davison, the creator of LLRT [10] Production-Ready Serverless workshop Free workshop giveawayI'm giving away FREE seat(s) to my workshop in March. All you have to do is give me some feedback on my current landing page and leave your feedback in a comment here. Depending on the no. of comments, I will give away one or more seats. The winners will be picked at random on the 1st March. I'm good at this AWS and serverless thing, but not so much on UI and UX... So any help would be greatly appreciated! |
Master Serverless
Join 17K readers and level up you AWS game with just 5 mins a week.
Step Functions is often used to poll long-running processes, e.g. when starting a new data migration task with Amazon Database Migration. There's usually a Wait -> Poll -> Choice loop that runs until the task is complete (or failed), like the one below. Polling is inefficient and can add unnecessary cost as standard workflows are charged based on the number of state transitions. There is an event-driven alternative to this approach. Here's the high level approach: To start the data migration,...
Lambda Durable Functions comes with a handy testing SDK. It makes it easy to test durable executions both locally as well as remotely in the cloud. I find the local test runner particular useful for dealing with wait states because I can simply configure the runner to skip time! However, this does not work for callback operations such as waitForCallback. Unfortunately, the official docs didn't include any examples on how to handle this. So here's my workaround. The handler code Imagine you're...
Lambda Durable Functions is a powerful new feature, but its checkpoint + replay model has a few gotchas. Here are five to watch out for. Non-deterministic code The biggest gotcha is when the code is not deterministic. That is, it might do something different during replay. Remember, when a durable execution is replayed, the handler code is executed from the start. So the code must behave exactly the same given the same input. If you use random numbers, or timestamps to make branching...