Blog -- Bradford Hovinen, IT-Freelancer

Recently the general availability of the AWS SDK for Rust was announced. I thought this might be a good occasion to write about my recent experiences writing and testing with AWS Lambda and Rust.

When I started my freelancing business, I set up a website with a contact form. I looked around at existing solutions, but couldn't find anything which quite ticks all the boxes. So, I decided to write my own backend.

The job of the backend is pretty simple: get the data from the form, pack them in an email, and send that email to a specific destination. It also needed a way to protect from bots, such as a captcha.

For the backend itself, I chose AWS Lambda. This makes sense given the low traffic volume I expect. The AWS API Gateway exposes the backend as an HTTP endpoint. The lambda sends emails via SMTP using the AWS Simple Email Service. I use a (privacy- and accessibility-friendly) captcha solution called FriendlyCaptcha. Credentials for both of these are stored in the AWS Secrets Manager.

Writing a Lambda in Rust

I decided to write the lambda in Rust to see how the AWS Rust SDK works. It runs as an ordinary binary which can process any number of lambda events. It has some modest boilerplate to call into the AWS SDK to set up a listener for lambda events and invoke a handler on each event. It runs in the Tokio runtime and uses non-blocking variants of all APIs, so each running instance could actually handle a lot of traffic if it needed to.

use lambda_http::{
    http::StatusCode, run, service_fn, Body, Error, Request, Response
};

#[tokio::main]
async fn main() -> Result<(), Error> {
    let handler = ContactFormMessageHandler::new().await;
    run(service_fn(|event| handler.handle(event))).await
}

struct ContactFormMessageHandler {...}

impl ContactFormMessageHandler {
    async fn handle(&self, event: Request) -> Result<Response<Body>, Error> {...}
}

The request and response are essentially the HTTP request and response received from, respectively sent to the user agent. The Lambda SDK provides facilities for parsing the payload and constructing the response:

#[derive(Deserialize)]
struct ContactFormMessage {
    name: String,
    email: String,
    subject: String,
    body: String,
}

async fn handle(&self, event: Request) -> Result<Response<Body>, Error> {
    let Some(message: ContactFormMessage) = event.payload()? else {
        return Ok(Response::builder()
            .status(StatusCode::BAD_REQUEST)
            .body("Could not extract message payload".into())
            .unwrap());
    };
    match self.process_message(message).await {
        Ok(_) => Ok(Response::builder()
            .status(StatusCode::OK)
            .body("Message sent".into())
            .unwrap()),
        Err(_) => Ok(Response::builder()
            .status(StatusCode::INTERNAL_SERVER_ERROR)
            .body("Could not send message".into())
            .unwrap()),
    }
}

async fn process_message(message: ContactFormMessage) -> Result<(), Error> {...}

This Rust program is compiled to a binary, packed into a Docker image, and uploaded to AWS. The Cargo Lambda tool builds and deploys the lambda automatically. AWS Lambda then launches instances of the binary to handle incoming events. Each instance typically runs for a short time, shutting down on its own if it is no longer needed to serve traffic.

Cross-compiling for arm64

AWS Lambda allows running functions on either the x86-64 architecture or arm64. The latter has somewhat lower costs, so I decided to target it to see how well that goes. This means compiling the lambda for the arm64 architecture. Nominally this requires just passing the flag --arm64 to the cargo lambda build command.

There is one twist, however. By default, the SMTP and HTTP clients my lambda uses rely on OpenSSL for TLS support. This means that to cross-compile the lambda, one must cross-compile the OpenSSL C++ library itself. This would mean going down a rabbit hole.

Fortunately, this is easily solved by configuring the SMTP and HTTP dependencies to use Rustls, a pure Rust TLS implementation. This requires just flipping a few feature flags in Cargo.toml:

[dependencies]
...
lettre = { version = "0.11.1", features = ["rustls-tls", "tokio1-rustls-tls", ...], default-features = false }
reqwest = { version = "0.11.22", features = ["rustls", "tokio-rustls", "__tls", "__rustls", "rustls-tls", ...], default-features = false }
...

Testing the lambda

I felt this project wouldn't be complete without exploring how to perform automated testing on the solution. The lambda is the kind of component which gives unit testers headaches. It has relatively little business logic but a lot of integrations with external systems. The business logic which does exist is pretty critical, though. If something goes wrong and the message can't go out, the user needs to see an alternative way to reach out to me. There's just enough in there to trip one up when refactoring. At the same time, experience shows that it's at the external integrations that things tends to go wrong.

Thus I wanted to cover as many of the external integrations as possible with tests, and in a way which is as close to a production setup as possible. I also wanted to cover the business logic without the tests running too slowly. With these requirements in mind, I created two test suites:

a single end-to-end integration test covering the happy path, and
a set of less integrated tests covering various error conditions.

The end-to-end test uses LocalStack running in Testcontainers to simulate real AWS services. The test runs the Lambda inside LocalStack and invokes it using the AWS SDK. It sets up its own fake SMTP server and fake FriendlyCaptcha verification server. The AWS Secrets Manager is provided by LocalStack. The following diagram summarises the setup. The component under test is in blue, components written as part of test are in green, and components supplied by LocalStack are in red.

The test invokes the Lambda with a valid request. It verifies that the email was sent through the simulated SMTP server and that the response is correct.

The major complication in this test setup is that the component under test runs inside a Docker container (managed by the Lambda environment), which itself runs inside another Docker container (LocalStack running with Testcontainers). This makes instrumentation of the component under test quite difficult. One must communicate with the lambda that it should connect with the fake SMTP and FriendlyCaptcha implementations and not attempt to use the real AWS services. I solved this with environment variables, which is not ideal but works well enough.

The test unfortunately leaves out the API gateway. This is because the required features of LocalStack are not available in the free version. For the same reason, it uses the Rust crate mailin-embedded for the fake SMTP server, simulating the AWS Simple Email Service.

The other tests have a much simpler architecture. They do not use LocalStack at all but instead invoke the event handler directly. They still use the fake SMTP server and FriendlyCaptcha running as in-process servers. But they replace the secrets manager with a test double.

To facilitate the test double for the secrets manager, I introduced a bit of hexagonal architecture in the lambda. The trait SecretRepository abstracts away communication with the secrets manager:

#[async_trait]
pub trait SecretRepository {
    async fn open() -> Self;
    async fn get_secret<T: DeserializeOwned>(
        &self,
        name: &'static str,
    ) -> Result<T, lambda_http::Error>;
}

The implementation for the AWS Secrets Manager is a straightforward wrapper over the AWS SDK. The unit tests use an implementation FakeSecretRepository which allows the test to provision and remove secrets as needed.

To install the correct version in the event handler, the lambda event handler use generics:

struct ContactFormMessageHandler<SecretRepositoryT: SecretRepository> {
    secrets_repository: SecretRepositoryT,
    ...
}

impl<SecretRepositoryT: SecretRepository> 
    ContactFormMessageHandler<SecretRepositoryT>
{
    async fn new() -> Self {
        let secrets_repository = SecretRepositoryT::open().await;
        Self {
            secrets_repository,
            ...
        }
    }

    ...
}

With this setup, the unit tests can access and manipulate all of the external systems the lambda backend uses. They can simulate any error condition which may come up and assert on the correct behaviour.

pub struct FakeSecretRepsitory(HashMap<&'static str, String>);

impl FakeSecretRepsitory {
    pub fn remove_secret(&mut self, name: &'static str) {
        self.0.remove(name);
    }
}

#[async_trait]
impl SecretRepository for FakeSecretRepository {...}

type ContactFormMessageHandlerForTesting = 
    ContactFormMessageHandler<FakeSecretRepsitory>;

#[googletest::test]
#[tokio::test]
async fn returns_contact_page_when_secrets_service_fails_for_smtp() {
    // ...other setup...
    let mut subject = ContactFormMessageHandlerForTesting::new().await;
    subject.secrets_repository
        .remove_secret(SMTP_CREDENTIALS_NAME);

    let response = subject.handle(event).await.unwrap();

    expect_that!(response.status().as_u16(), eq(500));
    expect_that!(
        response.body(),
        points_to(matches_pattern!(Body::Text(contains_substring(
            "Something went wrong"
        ))))
    );
}

Conclusion

This project was a worthwhile exploration of the AWS SDK for Rust, both using it and testing against it. While it was a bit more challenging than, say, a JavaScript runtime, it wasn't too hard to get everything working. The biggest challenges were in the testing setup, but I was able to achieve fairly good coverage in the end. All in all, I'm satisfied with the result.

The complete solution is available on GitHub.

See also this blog post for another exploration of AWS Lambda with Rust with more information on using Lambda with Terraform as well as manual invocation through Cargo Lambda.

Category: Walkthroughs | Tags: AWS Lambda Rust testing

Exploring the AWS Lambda SDK in Rust

Writing a Lambda in Rust

Cross-compiling for arm64

Testing the lambda

Conclusion