Rust for serverless applications

2020-08-11
6 minutes
Rust

Over the past few months, I finally started to understand the Rust programming language. Like other low-level languages such as C and C++, learning it is a pretty daunting task. If you ever try (unsuccessfully) to learn C/C++, you might remember being greeted with segfaults more often than you’re willing to admit. That was the case for me, at least.

With Rust, this was replaced with arcane errors about borrow, ownership, lifetimes, etc. Contrary to C/C++, Rust will throw these errors at compile time rather than at runtime, and the compiler is refreshingly helpful. I found that, once I got past that initial barrier, I was faster to add new features to my project with Rust than with languages like Python, mostly because I had to write fewer tests.

Once I started deploying and comparing workloads in Rust and other languages, I got into the third benefit of rust: performance. Rust’s slogan, “Fast, Reliable, Productive. Pick Three.” really held in my experience. I wouldn’t consider myself a Rust expert by far, but I’ve encountered little problems so far. That said, I’m also lucky to be surrounded by lots of rustaceans that help me along the way.

Working with custom runtimes

If you have only ever built Lambda functions with one of the standard runtimes, using a custom one might seem like a daunting task. That said, there have been significant improvements on the tooling side and, in the case of Rust, a set of crates that abstract that complexity away.

At the core of a custom runtime is the bootstrap file, an executable at the root of your Lambda function’s zip file that will contain a loop to poll an HTTP API (called the Lambda runtime interface) to receive an event and send then a reply back via the same API.

In languages that compile to a single file (like Rust), this means that we only need a single bootstrap file that encompasses both that loop and our business logic. This means that if our project has four Lambda functions, we’d need four executables at the end.

Hello, Serverless!

Enough theory, let’s write a simple Lambda function. The first thing we need is to add a few dependencies in Cargo.toml. The lambda crate will help us manage the main loop and runtime interface, while lambda_http will help with Amazon API Gateway events. We’re using an experimental of the Lambda runtime, which supports asynchronous calls through the tokio crate.

[dependencies]
lambda = { git = "https://github.com/awslabs/aws-lambda-rust-runtime" }
lambda_http = { git = "https://github.com/awslabs/aws-lambda-rust-runtime" }
tokio = { version = "0.2", features = ["full"] }

From there, we can write a simple function (for example at src/bin/hello.rs) that will return a personalised “Hello, {name}!” or a generic “Hello, world!”.

use lambda_http::{
    handler,
    lambda::{self, Context},
    IntoResponse, Request, RequestExt, Response,
};

type Err = Box<dyn std::error::Error + Send + Sync + 'static>;

/// Main function handler for our executable
///
/// This will take care of contacting the Lambda runtime API for us.
#[tokio::main]
async fn main() -> Result<(), Err> {
    lambda::run(handler(hello_world)).await?;

    Ok(())
}

/// Hello world Lambda function
///
/// This will return a personalised Hello if we have a first_name query
/// parameter, otherwise it will return a generic 'Hello, world!'.
async fn hello_world(event: Request, _ctx: Context) -> Result<impl IntoResponse, Err> {
    Ok(match event.query_string_parameters().get("first_name") {
        Some(first_name) => format!("Hello, {}!", first_name).into_response(),
        _ => Response::builder()
            .status(200)
            .body("Hello, world!".into())
            .expect("failed to render response"),
    })
}

Multiple executables to the rescue

A common challenge when building lots of small Lambda functions in a project is sharing logic across all of them. Code duplication feels wrong because of the Don’t Repeat Yourself principle. If you use something like Python, you could get around that with a shared library and use paths in requirements.txt.

Rust supports creating multiple executables from a single crate. All you have to do for this is to have one file per executable in the src/bin/ folder of your crate. For example, if you have a CRUD API, your project could look like this:

.
├── Cargo.toml
└── src/
    ├── lib.rs
    ├── logic/
    │   └── mod.rs
    └── bin/
        ├── create.rs
        ├── read.rs
        ├── update.rs
        └── delete.rs

When you compile your project, you will get four executables named create, read, update and delete, for each of the Lambda function and each of those can directly leverage all the shared logic in your library. Since this is a compiled language, the executables will not contain references to code that is not used, which means that the functions won’t carry unnecessary dependencies.

Building functions

I’ve used AWS SAM a lot for my Serverless projects. It comes with a CLI tool that allows you to build Lambda functions, test them locally, and deploy to AWS (amongst other things). In May this year, it added support for building provided (a.k.a. custom) runtimes. To leverage this, you need to create a Makefile with a build-{function} target for each Lambda function.

However, there is a bit of a gotcha there. At first, I just put cargo build --release in these targets, but I noticed that my build went from seconds to minutes. SAM will build each function in an isolated environment, which means that it would rebuild everything, including all dependencies, for each function.

Thus you could move the build part outside of SAM, and use a Makefile for the entire build process like such. The SAM-specific build targets would just be used to copy the executable as a bootstrap file at the root of the Lambda zip file. I then run make build instead of sam build to build the project.

build:
  # Compile once for the entire project
	cargo build --release
	sam build

# The function specific build step only copies the file in the right position,
# which is much faster.
build-CreateFunction:
	cp ./target/x86_64-unknown-linux-musl/release/create $(ARTIFACTS_DIR)/bootstrap

build-ReadFunction:
	cp ./target/x86_64-unknown-linux-musl/release/read $(ARTIFACTS_DIR)/bootstrap

build-UpdateFunction:
	cp ./target/x86_64-unknown-linux-musl/release/update $(ARTIFACTS_DIR)/bootstrap

build-DeleteFunction:
	cp ./target/x86_64-unknown-linux-musl/release/delete $(ARTIFACTS_DIR)/bootstrap