Rendering Tidbyt graphics in Rust
One of my other long-term projects has been building new graphics for my Tidbyt in Rust. It has been a slow, silly process in which I celebrate when a single pixel lights up on the device. I’m not even writing firmware or code that runs “on the device” - that’s a stretch goal for someday. I’m just writing a Rust program that renders some WebP graphics and pushes them to Tidbyt’s API.
I’ve had a lot of type 2 fun. I’ve also learned a lot more about Rust.
It’s probably not useful for many people, but the repository is now open source. There’s a little implementation of checking a Fastmail email account to get a count of unread emails and Strava for new runs.
Plus, there’s an implementation of checking weather with weather.gov, UV exposure from an EPA API, and AQI from AirNow. Isn’t it nice to feel the benefits of paying taxes by using these free APIs from government services?
There have been lots of high points to the project so far. The Rust compiler can be really fast when it’s recompiling projects. The error messages it emits can be really helpful. And the quality of the average Rust project seems really high: even a niche crate that I used for this project - bdf was really solid and worked right off the bat.
This is markedly better than my experience with the JavaScript ecosystem and NPM modules: I read through every new module that I add as a dependency and have to sort through the old or overcomplicated duds. A lot of the times, the popular module in the JavaScript ecosystem is kind of not-very-good and has much better alternatives that people should be using instead.
I love anyhow
Why isn’t the anyhow crate part of Rust core? Every time I’ve needed to return Result types from functions, or unwrap Option values within functions that return a Result, I get stuck in some vague dyn gotcha with the borrow-checker, and then I just sprinkle anyhow on it, and it works.
I barely understand how anyhow does its magic, but so far I haven’t needed to: I just use it for everything and it makes Rust much less painful. When I recently removed all of the lazy use of .unwrap() in this project - a method that gets the “successful” values out of Result and Option objects or throws if the Result is an error or the Option is nothing, I was stuck until I realized that I could use the anyhow Context trait which gives Options a .context method that lets you unwrap them in methods that return Result.
Rendering pixel art in Rust
This project is all to run a Tidbyt display. This is what it looked like last time I wrote about it:
The images I’m generating are 64x32 pixels, and the pixels are huge, so aesthetically I want everything to be crisp and non-antialiased. Which is a fun challenge for this project, because a lot of graphics technologies are built for floating-point pixels and anti-aliasing by default.
My first inclination was to use Bevy the game engine for Rust. It’s super cool, but seemed like kind of overkill for this project - if I ever want to run this on the Tidbyt hardware directly, my guess is that Bevy will be too heavyweight.
Bevy seemed to heavyweight, and embedded_graphics seemed too low-level. Raqote is just right. And Raqote being adopted by the huge Servo browser project was a point of trustworthiness. It’s been pretty reliable.
The code to make all of this work together has been kind of hilariously low-level. For example, here’s a snippet of how I’m rendering letters:
for px in glyph.pixels() {
let x = px.0 .0;
let y = px.0 .1;
let white = px.1;
if white {
dt.fill_rect(
start.x + x as f32 + x_offset as f32,
start.y + y as f32 + y_offset as f32,
1., 1., color, &DrawOptions::new(),
)
}
}
That’s right, I use the bdf crate to read bitmap fonts into a map of pixels to on & off values, and render those fonts pixel-by-pixel. I use the font’s glyph advance information to manually move forward to draw the next character. The same kind of process goes for rendering the pixels to a WebP image: pixel by pixel. Like I said, type 2 fun.
On ChatGPT
So, I’ll admit, I use ChatGPT to write bash scripts sometimes. I don’t know bash very well and have no interest in learning it. I don’t use it for harder problems because, brace yourself, I think that cognitive load is good. I know that cars and bicycles exist, but I still go on runs because it’s not just about getting to the destination.
But, at many points in this project I’ve reached for ChatGPT (4.0) in moments of weakness, and it is not very good. It hallucinates methods that don’t exist. It can’t solve borrow checker problems. Four or five broken solutions deep, I just start regretting the CO2 that this conversation has burned and promising myself that I’ll just read more documentation.
The borrow checker
Complaining about the borrow checker in Rust is like writing about NaN in JavaScript or the GIL in Python, but in reality, how bad is it?
In this project, it’s been an intermittent annoyance. I’ll have an hour of coding with everything going according to plan, and then spend an hour trying to refactoring three lines of code into a function. And that is a consistent theme: the more I try to refactor this program into something that’d make sense in Ruby, with small focused functions, the worse the borrow-checking becomes.
To give an example, here’s the start of my render() method:
async fn render(args: &Args) -> Result<()> {
let local: DateTime<Local> = Local::now();
let width = 64i32;
let height = 32i32;
let mut config = WebPConfig::new().map_err(|_s| anyhow!("WebPConfig failed"))?;
config.lossless = 1;
let mut encoder = AnimEncoder::new(width as u32, height as u32, &config);
// ...
I think that maybe I want to get a similar encoder somewhere else in my application, so it’d be nice to have a get_encoder() method that encapsulates those first few lines. So, I refactor those few lines into something like
fn get_encoder() -> Result<AnimEncoder<'static>> {
let width = 64i32;
let height = 32i32;
let mut config = WebPConfig::new().map_err(|_s| anyhow!("WebPConfig failed"))?;
config.lossless = 1;
let mut encoder = AnimEncoder::new(width as u32, height as u32, &config);
encoder
}
And the borrow checker is summoned from the dark place in the woods where it lives:
error[E0515]: cannot return value referencing local variable `config`
--> src/main.rs:369:5
|
368 | let mut encoder = AnimEncoder::new(width as u32, height as u32, &confi...
| ------- `config` is borrowed here
369 | Ok(encoder)
| ^^^^^^^^^^^ returns a value referencing
data owned by the current function
This happened for two major cases:
- The webp crate’s AnimEncoder takes a reference to a WebPConfig.
- The raqote crate’s Image struct takes a reference to a slice of pixel data.
Now, brief intermission from this to thank two people. I complained about these issues as I was hitting them, and Dave Ceddia was extremely helpful on one and Owen Nelson took time out of his Sunday to help me on another. Visit their websites! Appreciate the value of people just being helpful and nice to each other!
Maybe for reasons of performance, the crates that I’ve been using don’t “take ownership” of some of the arguments of AnimEncoder::new or Image::new - maybe you already have the bytes lying around, and you want to create an Image that refers to those bytes without creating a copy of them. This makes sense, but makes it a lot harder to write a method that configures & returns an AnimEncoder struct or reads an image into bytes and returns an Image struct - the bytes have to belong to something.
The solution seems to be creating a struct that owns both the Image and the data it refers to, and both the AnimEncoder and its configuration. This worked well for raqote’s Image type, but still.
I think this is the biggest part of the learning curve of Rust for me. In most other languages, you can pull a few lines of an existing function into a new function and it behaves mostly the same. In Rust, the context really matters – especially for the borrow-checker, but also for the question mark operator, which you can only use in methods that return Result or Option values, and .await, which you can only use in asynchronous functions. JavaScript has the same restriction with its await keyword, but you can still use .then to use Promise values in “synchronous” functions.
I think the next thing for me to learn is how to use the Box and Rc types so I can trade be more free and wild with passing around memory.
Async
The borrow checker is still my enemy, but this time around, asynchronous Rust has been pretty painless. I’m using tokio and thankfully haven’t needed to bridge it to any other async runtime.
I think the coolest moment of all these crates working together was using the cached macro to cache the requests to different APIs so that I don’t hammer Fastmail’s servers.
/// Get the number of unread threads in my inbox
#[cached(time = 120, result = true)]
pub async fn get_email_count() -> Result<(u64, Vec<u64>)> {
It just worked: one line of code and it worked with an async method that returns a Result. A lot of these crates, the work that the Rust community has been producing, are rock-solid.
Running on the device
This whole contraption still generates a Base64-encoded WebP image and sends it to Tidbyt’s web service. It’d be super cool to run on the Tidbyt itself, and they released a development kit that lets you re-flash firmware. A baby step in the direction of self-hosting would be to flash firmware with a local URL to pull the image from - I think I could assign a static IP to a home server. It’d be even cooler to use Tailscale to handle the networking, but I don’t think that they scale down to the tiny chip on this device: judging by the hardware development kit, it’s some kind of ESP32 chip.
Rust does have some support for the ESP32 library, but getting all of that working with the display driver is too big a side project for me, for now.
The actual program
I’ve mostly written about all the joys of implementing the thing, not much about the thing itself.
Tidbyt provides a library called pixlet for developing graphics and applications for the display. For 99% of people, you should use that: it solves all the problems I’ve solved and more.
I wanted to branch out from pixlet, though, first because it’s very locked down: you write pixlet applications in Starlark, which is a Python-like language. You can’t access the filesystem, and there aren’t many Starlark libraries, though the ones maintained by the folks at Google & Tidbyt are high-quality. Plus, learning Rust is a long-term goal for me and this is a fun justification.
The display currently features:
- The outdoor temperature
- A warning about UV radiation or AQI if either is high today
- A count of my unread emails
- How many miles I’ve run in the last 7 days
- The time
Mainly development has been focused on making it more reliable, so that even if Fastmail or NOAA is offline, the display still works, and making it more customizable. It now has a tiny little layout system and a tiny widget system:
trait Widget: Send {
// Gets the width of the given widget
fn measure(&self) -> Point;
fn frame_count(&self) -> u32;
fn render(&self,
dt: &mut DrawTarget,
point: Point,
frame: u32) -> Result<(), Error>;
}
Widgets can measure themselves to be laid out, and then render onto a shared canvas. I want to add animation, but that’s still a work-in-progress. This might’ve been the first time I implemented a trait with methods! That part of Rust clicked immediately: it seems like such an elegant way to define shared behavior.
The widgets are combined in stacks:
let layout = vstack![
hstack![
get_weather().await,
TextWidget::new(
format!("{}", local.format("%l:%M").to_string()),
String::from("#fff"),
)
],
hstack![
TextWidget::new(format!("{} MAIL", count), String::from("#fff")),
ChartWidget::new(&rec_chart)
],
hstack![
TextWidget::new(format!("{:.0} RUN", week_miles), String::from("#fff")),
ChartWidget::new(&miles_chart)
],
hstack![get_aqi().await, get_uv().await]
]
.map(|s| s.set_gap(3.0));
And this was my first time implementing a macro, for vstack! and hstack!. That part felt much hackier: I used macros because managing the ownership of the items was tricky. The horizontal stacks have a little layout algorithm which is kind of amusing because of the nature of the problem: unlike a standard flexbox implementation, I really want to make sure that the pixel boundaries are right. Having an uneven number of pixels might shift the date to the right. The math for this was pretty fun.
This project has been a lot of fun. I feel like I’ve been working at the edge of my abilities the whole time. I’m pathetically proud of rendering some pixels to the screen. The absurdity of gesturing toward this glorified clock that I’ve spent hours coding.
In a way it’s a perfect antidote to the behaviors that the internet and social media have been brainwashing us all into. It’s optimized for nothing but my own personal enjoyment. Not even the open source code is really worth reusing yet. One of my favorite things about the Tidbyt – something that they’ve ‘fixed’ in their second-generation devices - is how amazing the display looks in real life and how terrible it looks in photos. The display is made for eyes, not CMOS sensors. This project was about doing it, not about finishing it.