First impressions of the fastest JavaScript runtime for Lambda
First impressions of the fastest JavaScript runtime for Lambda Read on my blog Read time: 4 minutes I thought Lambda needed a specialised runtime. One that works well with its resource-constraint execution environment. I even floated a few ideas in the past but sadly I don’t have the chops to make them happen myself. So I was pleasantly surprised when AWS open-sourced the LLRT runtime for JavaScript [1]! What is LLRT?LLRT, or Low Latency Runtime, is a new and experimental JavaScript runtime for Lambda. It promises 10x faster startup time. Which should significantly help with the dreaded Lambda cold starts. Naturally, I had to test it out for myself and see if the hype was real. My first impressions of LLRTIn my limited tests, the Along the way, I also discovered several limitations:
LLRT is not ready for prime time yet. And given that it’s been in the works for almost two years, we should set expectations accordingly. It’s probably not going to be production-ready in the coming months. But the potential is there and I’m really excited about it! For one thing, it might finally stop people talking about cold starts. As I said on LinkedIn [5] yesterday, most people are overthinking about Lambda cold starts. If you’re not sure if Lambda cold starts is likely a problem for you, then go read the LinkedIn post and come back here. Back to LLRT. I later spoke with Richard Davison, the creator of the LLRT project. I wanted to learn more about the project and the design decisions behind it. What made it start so much faster? What trade-offs did they make? LLRT is the answer to the question “What would a purposely built JavaScript runtime look like for Lambda.” A lot of people see that “It’s built in Rust” and automatically assume that’s why it’s fast. But it’s more than that. When it comes to performance optimizations, it always boils down to what you let go. If you choose to do everything the incumbent does, then you won’t make any significant performance gains. No JIT compilationWith LLRT, they chose to not include a JIT compiler. Because it’s focused on Lambda’s resource constraint and short-lived execution environments. As a result, LLRT is likely less performant than the Node.js runtime for CPU-intensive tasks. However, most Lambda functions do not perform CPU-intensive tasks. Instead, they tend to be IO-intensive. And the Lambda execution environments are short-lived. So a JIT compiler would have been less effective at optimizing hot code paths. At the same time, a JIT compiler comes with significant startup costs. It also introduces occasional latency spikes when it needs to evict cached items. So it appears a sensible trade-off for LLRT to not include a JIT compiler. QuickJs + RustThe QuickJs engine [6] plays a crucial role in LLRT and its outstanding performance. Another reason why LLRT is fast is because they wrote all the APIs in Rust. As much as possible, the team wants to stay in native code to guarantee a strong performance. Bun [7] took the same approach and implemented all the APIs in a system language called Zig [8]. The downside to this approach is that it’s harder for contributors to get in on the action. There are a lot fewer Rust developers than Node.js developers. And even fewer Rust developers who are interested in a JavaScript runtime. LLRT vs Bun (and other JS runtimes)LLRT is different from other JavaScript runtimes. It’s not a general-purpose runtime for JavaScript. It doesn’t have to worry about running in the browser or on mobile phones. Instead, it’s solely focused on the Lambda execution environment. This allows them to make decisions that just wouldn’t make sense with Bun or Node.js. Decisions such as not including a JIT compiler. Or which of the JavaScript APIs do they implement first, or at all? The goal is to eventually become WinterCG compliant. But we don’t have to wait for that to start using LLRT. For LLRT to be useful (but not perfect!), it just needs to support the AWS SDK and a few popular libraries. SummaryTo summarize, LLRT is an exciting new runtime for JavaScript. It’s purposely built for Lambda. It’s not intended as a general-purpose runtime for JavaScript. It makes design trade-offs (such as not having a JIT compiler) to achieve an optimal cold start time. It uses the QuickJs engine and implements all the JavaScript APIs in Rust. To learn more about LLRT and how to contribute to it, please check out my conversation with Richard on YouTube [9]. I will be covering LLRT in my upcoming workshop, including how to use it with the Serverless Framework and CDK. If you wanna take your serverless game to the next level, then you should check it out! More information is available on the course page [10]. Links[1] GitHub repo for LLRT [3] LLRT’s API compatibility page [4] AWS Lambda Powertools for TypeScript [5] Most people are overthinking about Lambda cold starts [6] QuickJs engine [7] The Bun runtime for JavaScript [8] The Zip programming language [9] My interview with Richard Davison, the creator of LLRT [10] Production-Ready Serverless workshop Free workshop giveawayI'm giving away FREE seat(s) to my workshop in March. All you have to do is give me some feedback on my current landing page and leave your feedback in a comment here. Depending on the no. of comments, I will give away one or more seats. The winners will be picked at random on the 1st March. I'm good at this AWS and serverless thing, but not so much on UI and UX... So any help would be greatly appreciated! |
Master Serverless
Join 15K readers and level up you AWS game with just 5 mins a week.
ICYMI, Serverless Inc. recently announced the Serverless Container Framework. It allows you to switch the compute platform between Lambda and Fargate with a one-liner config change. This is a game-changer for many organizations! It'd hopefully nullify many of the "lock-in" worries about Lambda, too. As your system grows, if Lambda gets expensive, you can easily switch to Fargate without changing your application code. To be clear, this is something you can already do yourself. It's not a...
During this week's live Q&A session, a student from the Production-Ready Serverless boot camp asked a really good question (to paraphrase): "When end-to-end testing an Event-Driven Architecture, how do you limit the scope of the tests so you don't trigger downstream event consumers?" This is a common challenge in event-driven architectures, especially when you have a shared event bus. The Problem As you exercise your system through these tests, the system can generate events that are consumed...
I recently helped a client launch an AI code reviewer called Evolua [1]. Evolua is built entirely with serverless technologies and leverages Bedrock. Through Bedrock, we can access the Claude models and take advantage of its cross-region inference support, among other things. In this post, I want to share some lessons from building Evolua and offer a high level overview of our system. But first, here’s some context on what we’ve built. Here [2] is a quick demo of Evolua: Architecture This is...