Intro

This year I have been learning Rust, and I recently came across an excellent post by Tyler Christiansen (@supertylerc) who BTW, looks EXACTLY like Captain Jean-Luc Picard, comparing Python and Go. It's an excellent post (the second in the series) and well worth a read. I am on a similar journey from Python to Rust, so I decided to emulate the Go part of the blog post in Rust as a bit of a fun exersize on Easter Monday 🐇.

In this post, I will show you how to connect to the Pokemon API asynchronously to gather information on the first 150 Pokemon. Unlike in Tylers post, I will not be covering Python or a synchronous version of the Rust code.

Software

The following software was used in this post.

  • Rust - 1.59.0
  • reqwest - 0.11
  • tokio - 1.12.0
  • futures - 0.3.21
  • serde - 1.0.136
  • serde_derive - 1.0.136
  • serde_json - 1.0.79

New Project

Use cargo to generate a new project.

cmd
cargo new pokemon

Dependencies

Add the following libraries to the dependencies section of the cargo.toml file.

Cargo.toml
[dependencies]
reqwest = { version = "0.11", features = ["json"] }
tokio = { version = "1.12.0", features = ["full"] }
futures = "0.3.21"
serde = "1.0.136"
serde_derive = "1.0.136"
serde_json = "1.0.79"

Let's Get Rusty 🦀

I have placed comments in the following code block to try and explain what is going on and how it works. Admittedly, for some of the async code, I barely understand how it works myself.

main.rs
use futures::StreamExt;
use reqwest::{Client, Error};
use serde_derive::Deserialize;
use std::time::{Duration, Instant};

// The number of concurrent requests to execute.
const BUFFER_SIZE: usize = 50;

// Struct to hold the Pokemon data.
// The `Deserialize` macro adds the `Deserialize` trait to the struct.
#[derive(Deserialize)]
struct Pokemon {
    // For the purpose of this exersize I am just capturing 
    // the `id` and `name` fields, all other fields will be ignored.
    id: u32,
    name: String,
}

// The `#[tokio::main]` annotation sets the `main` function up for 
// asynchronous execution. 
#[tokio::main]
async fn main() {
    // Start a timer
    let start = Instant::now();

    // Create a `reqwest client` using the builder pattern.
    // This allows you to set things like the timeout, headers etc..
    let client = Client::builder()
        // Set the timeout value to 2 seconds.
        .timeout(Duration::from_secs(2))
        // Build the client.
        .build()
        // If there are any errors, bail with the message `got an error building client`
        .expect("got an error building client");

    // Create a vector of URLs to query consisting of the first 150 Pokemon.
    let urls = (1..=150).map(|x| format!("https://pokeapi.co/api/v2/pokemon/{}", x));

    // Here we get ready to asynchronously connect to the Pokemon API.
    // We start by creating a iterable `stream` of `futures` out of the `urls` vector.
    let data: Vec<Result<String, Error>> = futures::stream::iter(urls)
        // `StreamExt::map()` performs an action on each iteration of the `stream`
        // and converts it to a new type.
        .map(|url| {
            // Create a new `client` object from the original `&client` reference
            // that is enclosed in this scope.
            let client = &client;
            // Start an async block moving ownership of any captured varibales
            // into the block.
            async move {
                // Send a `get` request to the `url` and `await` the response.
                let resp = client.get(url).send().await?;
                // Return the `response text` as the `String` portion of the
                // `Result<String, Error>` type.
                resp.text().await
            }
        })
        // Kick of the requests asynchronously in batches equal to `BUFFER_SIZE`.
        .buffer_unordered(BUFFER_SIZE)
        // Collect all the responses and add them to the `data` vector.
        .collect()
        // Wait for all requests to finish.
        .await;

    // Due to the asynchronous nature of the API calls, the responses are returned out of order.
    // I want to print the Pokemon out in the order of their ID's lowest to highest.
    //
    // To do this, let's unpack the `data: Vec<Result<String, Error>>` vector so we can deserialize
    // the `response text` into our `Pokemon` struct.
    //
    // The `pokemons: Vec<Pokemon>` vector needs to be mutable so we can sort it later.
    let mut pokemons: Vec<Pokemon> = data
        .iter()
        .map(|d| match d {
            Ok(v) => {
                // Unpack the `Result<String, Error>` and deserialize the text into a Pokemon struct.
                // If there is an error, bail with the error message `broken pokemon`
                let pokemon: Pokemon = serde_json::from_str(v).expect("broken pokemon");
                // Return the pokemon object
                pokemon
            }
            // If any errors exist in `Result<String, Error>`, bail with the error message `broken pokemon`
            Err(_) => panic!("pokemon broken"),
        })
        // Collect all the Pokemon objects into the `pokemons` vector.
        .collect();

    // Sort the pokemons vector by their id. We made `pokemons` mutable 
    // so it can be sorted in place.
    pokemons.sort_by_key(|p| p.id);

    // Iterate over the `pokemons` vector to print the `id` and `name`.
    for p in pokemons {
        println!("{}: {}", p.id, p.name)
    }

    // Stop the timer
    let duration = start.elapsed();
    // Debug printing here ({:?}) to print out `duration` for simplicity.
    println!("Time elapsed: {:?}", duration);
}

I hope you got all that. You still with me? Clear as mud? Good! Let's continue.

Now, use cargo to build a binary that is optimized for release with the --relase flag.

cmd
cargo build --release

The binary pokemon will be generated in the target/release directory.

Gotta Catch Em All

Alright, the moment of truth. Let's hit them so hard and fast they think their surrounded.

cmd
target/release/./pokemon

# Output
1: bulbasaur
2: ivysaur
.
. <output omitted>
.
149: dragonite
150: mewtwo
Time elapsed: 425.068076ms

Holy crab apples Batman, that was fast ⚡ 425.068076ms ⚡

<1 second is pretty darn good and the result is very close with the async Go version from Tylers post.

The results for all 898 Pokemon are around the 1.5 - 2 second mark. I updated the code to query the API in batches of 50 which seemed like a good sweet spot without overloading the API server.

cmd
target/release/./pokemon

# Output
1: bulbasaur
2: ivysaur
.
. <output omitted>
.
897: spectrier
898: calyrex
Time elapsed: 1.630966447s

Outro

In this post, I showed you how to connect to the Pokemon API asynchronously to gather information on the lovable Pokemon. This was a super fun exercise that allowed me to learned a lot about the Rust async model. Thanks again to @supertylerc for the inspiration. Peace out legends ✌️

Tags

rust