Dancing with the compiler: Rust and explicitness

2022-03-03
12 minutes
Rust

One thing that often surprises newcomers to Rust is how verbose it can be. A simple application can quickly have hundreds of lines of code, which can be overwhelming at first. However, the process of creating that application doesn’t start with all that complexity from the get-go but is something I build incrementally working with the compiler.

For example, let’s say I want to create a Lambda function behind an API Gateway that will take a probability between zero and one, and return either true or false based on that probability. If I send a value of 0.3, I’d expect the function to return true 30% of the time.

For building this function, I can use the lambda_http crate which integrates with the http crate, providing Request and Response structs. I then need to have a handler function that would match this signature:

async fn handler(request: Request) -> Result<impl IntoResponse, Error>;

Parsing user input

Since there’s nothing complicated in this example, we can ask the end-users to send GET requests to / with a query parameter of probability. We can then retrieve this parameter as a string from the request, and parse it into a f64 value.

1 // Use declarations for the crates we'll be using
2 use lambda_http::{service_fn, Error, IntoResponse, Request};
3 
4 // Entrypoint for the Lambda function
5 #[tokio::main]
6 async fn main() -> Result<(), Error> {
7     lambda_http::run(service_fn(handler)).await?;
8     Ok(())
9 }
10 
11 async fn handler(request: Request) -> Result<impl IntoResponse, Error> {
12     // Get the probability from the request
13     let probability = request
14         .query_string_parameters()
15         .first("probability") // Returns an Option<&str>
16         .unwrap()
17         .parse::<f64>() // Returns a Result<f64, ParseFloatError>
18         .unwrap();
19 
20     // TODO - this is just a placeholder for now
21     Ok("hello")
22 }

If you’ve been playing with Rust for a while, this code should make you feel uneasy - and for good reason! Those unwraps make big assumptions regarding the request sent to the Lambda function. In this case, we assume it contains a query parameter named probability with a valid f64 value. Rust makes those assumptions visible through the Option and Result types. Because we are just unwrapping them here, the function would panic if the user sends a malformed input. We’ll come back to that later.

The most astute observers amongst you will also have noticed that this doesn’t compile. If I run cargo check, which allows me to check for errors in my code, I get the following message:

error[E0599]: no method named `query_string_parameters` found for struct `Request` in the current scope
   --> lambda-explicitness/src/main.rs:14:10
    |
14  |         .query_string_parameters()
    |          ^^^^^^^^^^^^^^^^^^^^^^^ method not found in `Request<Body>`
    |
   ::: /home/XXX/.cargo/registry/src/github.com-1ecc6299db9ec823/lambda_http-0.5.1/src/ext.rs:117:8
    |
117 |     fn query_string_parameters(&self) -> QueryMap;
    |        ----------------------- the method is available for `Request<Body>` here
    |
    = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope; perhaps add a `use` for it:
    |
1   | use lambda_http::RequestExt;
    |

http::Request doesn’t have a method called query_string_parameters()! That method is only available through an extension trait provided in the lambda_http crate. An extension trait is a trait that provides a default implementation for foreign types, and thus add new methods to those types. Thankfully, the compiler tells us what we’re missing, and we can just add the use statement to the top of the file.

Generating a boolean

Rust doesn’t provide a way to generate a random number in its standard library. We can use the quite popular rand crate to do this. If you’re unsure if a crate is well-maintained, popular, or what its license is, you can always check it on crates.io or lib.rs. The latter website provides an easy way to see popular crates for different categories, such as cryptography, data structures, networking, etc.

If you have cargo-edit installed, you can just type the following command to add rand:

cargo add rand

From there, we can generate a random boolean based on the given probability. rand provides a helpful gen_bool() method for our use case, which returns a boolean given a probability.

21     // Retrieve a thread-local random number generator
22     let mut rng = rand::thread_rng();
23     // Generate the boolean
24     let value = rng.gen_bool(probability);

Once again, the gen_bool() function is not specific to this random number generator. rand provides different generators, and you could even make your own if you so desire (but you probably shouldn’t). That means the gen_bool() function is tied to a trait that we didn’t import, and we get the same error as before:

error[E0599]: no method named `gen_bool` found for struct `ThreadRng` in the current scope
   --> lambda-explicitness/src/main.rs:22:21
    |
22  |     let value = rng.gen_bool(probability);
    |                     ^^^^^^^^ method not found in `ThreadRng`
    |
   ::: /home/XXX/.cargo/registry/src/github.com-1ecc6299db9ec823/rand-0.8.5/src/rng.rs:270:8
    |
270 |     fn gen_bool(&mut self, p: f64) -> bool {
    |        -------- the method is available for `ThreadRng` here
    |
    = help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope; perhaps add a `use` for it:
    |
1   | use rand::Rng;
    |

At this point, you might wonder why do we need to use the trait at all. After all, the Rust compiler already knows which trait has the method we’re looking for! However, you could have different traits that have methods with the same name, and the compiler can’t be sure which one to use, or if this is what you wanted to do at all.

Sending the response

Now that we have our boolean, we can finally send it back to the user. As lambda_http expects some value that implements IntoResponse, we can look in the documentation what types are implemented out of the box: String, &str, and serde_json::Value.

Since this is an API, it would be best to return a JSON response. We can use the serde_json crate to do this. It provides a serde_json::json! macro that will create a JSON Value. Under the hood, lambda_http will add a 200 status code and the right Content-Type header. Putting it all together, our function currently looks like this:

1 use lambda_http::{service_fn, Error, IntoResponse, Request, RequestExt};
2 use rand::Rng;
3 
4 #[tokio::main]
5 async fn main() -> Result<(), Error> {
6     lambda_http::run(service_fn(handler)).await?;
7     Ok(())
8 }
9 
10 async fn handler(request: Request) -> Result<impl IntoResponse, Error> {
11     // Get the probability from the request
12     let probability = request
13         .query_string_parameters()
14         .first("probability") // Returns an Option<&str>
15         .unwrap()
16         .parse::<f64>() // Returns a Result<f64, ParseFloatError>
17         .unwrap();
18 
19     // Retrieve a thread-local random number generator
20     let mut rng = rand::thread_rng();
21     // Generate the boolean
22     let value = rng.gen_bool(probability);
23 
24     // Send the boolean back
25     Ok(serde_json::json!({
26         "result": value,
27     }))
28 }

Apart from fetching the probability from the request, the complexity of this function is still pretty low at the moment. And as mentioned earlier, this is Rust explicitly asking us how we want to handle values that could be invalid.

Handling invalid user inputs

Speaking of which, it’s time to handle the cases when users either don’t provide a probability, or provide an invalid one. There are three possible cases here:

The user doesn’t provide a probability at all.
The user provides a probability that isn’t a number.
The user provides a probability that is a number, but not between 0 and 1.

That leaves us with deciding what to do with invalid inputs. We could return an Error, but that would mark the Lambda execution as failed and return a 5xx error code - indicating a server error. Instead, it’s best to handle this gracefully with a 400 error explaining to the users what they did wrong. That means telling them that the probability is missing, that it’s not a number, or that its value is not between 0 and 1.

Handling missing probabilities

Since first() returns an option, we can use match to check if the probability is missing. If it is missing, we can short-circuit the rest of the function and return an error message directly.

    // Get the probability from the request
    let parameters = request.query_string_parameters();
    let probability_str = match parameters.first("probability") {
        // If the parameter is present
        Some(probability) => probability,
        // If it's missing, return an error
        None => {
            return Ok(serde_json::json!({
                "message": "missing 'probability' in query string",
            }))
        }
    };

    // Parse the probability into a float
    let probability = probability_str.parse::<f64>().unwrap();

There’s a small problem here: remember how I said lambda_http would use the 200 status code? In this situation, if the probability is missing, we are returning an error message, but not with a 400 status code! For this, we will need to construct the Response manually, and add it to our use statement. Since we’re doing this manually now, we also need to serialize our error message into a string and add the right Content-Type header.

    // Get the probability from the request
    let parameters = request.query_string_parameters();
    let probability_str = match parameters.first("probability") {
        Some(probability) => probability,
        None => {
            return Ok(Response::builder()
                .status(400)
                .header("content-type", "application/json")
                .body(serde_json::json!({
                    "message": "missing 'probability' in query string",
                }).to_string())
                .unwrap())
        }
    };

As soon as we do this, the compiler complains: we are returning a Response object here, while we return a Value at the end of the function. While we don’t have to specify which type we’re returning precisely, Rust is still a strongly typed language and is expecting to have a single return type for the entire function.

error[E0308]: mismatched types
  --> lambda-explicitness/src/main.rs:34:8
   |
34 |       Ok(serde_json::json!({
   |  ________^
35 | |         "result": value,
36 | |     }))
   | |______^ expected struct `Response`, found enum `Value`
   |
   = note: expected struct `Response<std::string::String>`
                found enum `Value`
   = note: this error originates in the macro `json_internal` (in Nightly builds, run with -Z macro-backtrace for more info)

That means we need to construct a Response object for that case too. Our function is getting a bit more complex already, but this is part of the incremental process I mentioned at the beginning of this post.

    // Send the boolean back
    Ok(Response::builder()
        .body(
            serde_json::json!({
                "result": value,
            })
            .to_string(),
        )
        .unwrap())

Handling invalid probabilities

We can do the same thing when we are parsing the probability now. Instead of dealing with an Option, we have a Result but the same logic applies.

    // Parse the probability into a float
    let probability = match probability_str.parse::<f64>() {
        Ok(probability) => probability,
        Err(_err) => {
            return Ok(Response::builder()
                .status(400)
                .header("content-type", "application/json")
                .body(
                    serde_json::json!({
                        "message": "'probability' is not a float",
                    })
                    .to_string(),
                )
                .unwrap())
        }
    };

If we weren’t using floats as the probability, we could push this even further and validate the probability in the match statement itself. This uses to be possible, but now you’ll get a warning that floating-point types cannot be used in patterns (see the tracking issue here). Trust me: this is the first thing I’ve tried and decided to change course thanks to the compiler warning.

That means we have one last step to do: check if the probability is between 0 and 1. We can still do it quite cleanly by checking if the range 0.0..=1.0 does not contain the probability provided by the user.

    // Return an error if the probability is not between 0 and 1
    if !(0.0..=1.0).contains(&probability) {
        return Ok(Response::builder()
            .status(400)
            .header("content-type", "application/json")
            .body(
                serde_json::json!({
                    "message": "'probability' must be between 0 and 1"
                })
                .to_string()
            )
            .unwrap())
    }

Putting everything together

We now have a program that handles all the edge cases that users could send our way. It added quite a bit of verbosity, but we’re handling all possible errors gracefully. Since we’re repeating the same pattern to generate errors, we could create an ad-hoc function to reduce the amount of code. And here’s the complete program:

1 use lambda_http::{service_fn, Error, IntoResponse, Request, RequestExt, Response};
2 use rand::Rng;
3 
4 #[tokio::main]
5 async fn main() -> Result<(), Error> {
6     lambda_http::run(service_fn(handler)).await?;
7     Ok(())
8 }
9 
10 async fn handler(request: Request) -> Result<impl IntoResponse, Error> {
11     // Get the probability from the request
12     let parameters = request.query_string_parameters();
13     let probability_str = match parameters.first("probability") {
14         Some(probability) => probability,
15         None => return Ok(user_error("missing 'probability' in query string")),
16     };
17 
18     // Parse the probability into a float
19     let probability = match probability_str.parse::<f64>() {
20         Ok(probability) => probability,
21         Err(_err) => return Ok(user_error("'probability' is not a float")),
22     };
23 
24     // Return an error if the probability is not between 0 and 1
25     if !(0.0..=1.0).contains(&probability) {
26         return Ok(user_error("'probability' must be between 0 and 1"));
27     }
28 
29     // Retrieve a thread-local random number generator
30     let mut rng = rand::thread_rng();
31     // Generate the boolean
32     let value = rng.gen_bool(probability);
33 
34     // Send the boolean back
35     Ok(Response::builder()
36         .body(
37             serde_json::json!({
38                 "result": value,
39             })
40             .to_string(),
41         )
42         .unwrap())
43 }
44 
45 /// Create an user error response with the given message
46 fn user_error(msg: &str) -> Response<String> {
47     Response::builder()
48         .status(400)
49         .header("content-type", "application/json")
50         .body(serde_json::json!({ "message": msg }).to_string())
51         .unwrap()
52 }

You might notice that I have left a few unwraps in the code for generating responses. Looking at the body method documentation, we can see that this call could fail if some values provided are incorrect. Because we don’t let the user specify anything here and we’re pretty confident the values we provide are valid, we can safely unwrap the result.

n14n.dev