Operating System development tutorials in Rust on the Raspberry Pi

ℹ️ Introduction

This is a tutorial series for hobby OS developers who are new to ARM's 64 bit ARMv8-A architecture. The tutorials will give a guided, step-by-step tour of how to write a monolithic Operating System kernel for an embedded system from scratch. They cover implementation of common Operating Systems tasks, like writing to the serial console, setting up virtual memory and handling HW exceptions. All while leveraging Rust's unique features to provide for safety and speed.

Have fun!

Best regards,
Andre (@andre-richter)

P.S.: For other languages, please look out for alternative README files. For example, README.CN.md or README.ES.md. Many thanks to our translators 🙌.

📑 Organization

• Each tutorial contains a stand-alone, bootable kernel binary.
• Each new tutorial extends the previous one.
• Each tutorial README will have a short tl;dr section giving a brief overview of the additions, and show the source code diff to the previous tutorial, so that you can conveniently inspect the changes/additions.
  • Some tutorials have a full-fledged, detailed text in addition to the tl;dr section. The long-term plan is that all tutorials get a full text, but for now this is exclusive to tutorials where I think that tl;dr and diff are not enough to get the idea.
• The code written in these tutorials supports and runs on the Raspberry Pi 3 and the Raspberry Pi 4.
  • Tutorials 1 till 5 are groundwork code which only makes sense to run in QEMU.
  • Starting with tutorial 5, you can load and run the kernel on the real Raspberrys and observe output over UART.
• Although the Raspberry Pi 3 and 4 are the main target boards, the code is written in a modular fashion which allows for easy porting to other CPU architectures and/or boards.
  • I would really love if someone takes a shot at a RISC-V implementation!
• For editing, I recommend Visual Studio Code with Rust Analyzer.
• In addition to the tutorial text, also check out the make doc command in each tutorial. It lets you browse the extensively documented code in a convenient way.

Output of make doc

🛠 System Requirements

The tutorials are primarily targeted at Linux-based distributions. Most stuff will also work on macOS, but this is only experimental.

🚀 The tl;dr Version

1. Install Docker Engine.

2. (Linux only) Ensure your user account is in the docker group.

3. Prepare the Rust toolchain. Most of it will be handled on first use through the rust-toolchain.toml file. What's left for us to do is:

   1. If you already have a version of Rust installed:

      cargo install cargo-binutils rustfilt
      cargo install --locked \
        --git https://github.com/FlamingosProject/scip.git \
        --rev a72f816fb4120a58ec7d6c59032d1e43860bc459

   2. If you need to install Rust from scratch:

      curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
      
      source $HOME/.cargo/env
      cargo install cargo-binutils rustfilt
      cargo install --locked \
        --git https://github.com/FlamingosProject/scip.git \
        --rev a72f816fb4120a58ec7d6c59032d1e43860bc459

4. In case you use Visual Studio Code, I strongly recommend installing the Rust Analyzer extension.

🧰 More Details: Eliminating Toolchain Hassle

This series tries to put a strong focus on user friendliness. Therefore, efforts were made to eliminate the biggest painpoint in embedded development as much as possible: Toolchain hassle.

Rust itself is already helping a lot in that regard, because it has built-in support for cross-compilation. All that we need for cross-compiling from an x86 host to the Raspberry Pi's AArch64 architecture will be automatically installed by rustup. However, besides the Rust compiler, we will use some more tools. Among others:

• QEMU to emulate our kernel on the host system.
• scip, a Rust serial console that implements the MiniPush protocol for loading a kernel onto the Raspberry Pi on-demand over UART.
• OpenOCD and GDB for debugging on the target.

There is a lot that can go wrong while installing and/or compiling the correct version of each tool on your host machine. For example, your distribution might not provide the latest version that is needed. Or you are missing some hard-to-get dependencies for the compilation of one of these tools.

This is why we will make use of Docker whenever possible. We are providing an accompanying container that has all the needed tools or dependencies pre-installed, and it gets pulled in automagically once it is needed. If you want to know more about Docker and peek at the provided container, please refer to the repository's docker folder.

📟 USB Serial Output

Since the kernel developed in the tutorials runs on the real hardware, it is highly recommended to get a USB serial cable to get the full experience.

• You can find USB-to-serial cables that should work right away at [1] [2], but many others will work too. Ideally, your cable is based on the CP2102 chip.
• You connect it to GND and GPIO pins 14/15 as shown below.
• Tutorial 5 is the first where you can use it. Check it out for instructions on how to prepare the SD card to boot your self-made kernel from it.
• Starting with tutorial 6, booting kernels on your Raspberry is getting really comfortable. In this tutorial, a so-called chainloader is developed, which will be the last file you need to manually copy on the SD card for a while. It will enable you to load the tutorial kernels during boot on demand over UART.

Development

SDCard

To prepare an SD card for Raspberry Pi 5, erase it as FAT32, mount it at /Volumes/BOOT, and run make prepare-sdcard. This builds the chainloader for the RPi 5 debug UART, copies it as kernel8.img together with the required firmware files, and unmounts the card.

Use raspberry debug probe serial + gpio uart serial

Using two serial ports allows to see both bootloader logs and our kernel logs.

1. Connect RPI serial debugger to early uart and to U port on debugger.

2. Connect other usb to ttl to original gpio pins from the readme (GND, RX, TX)

3. Now you can use one of the programs to talk to the rpi over serial port:

   • make miniterm (uses scip)
   • https://github.com/tio/tio tio /dev/tty.usbmodem12202
   • screen screen /dev/ttyUSB0 115200
   • python with pyserial python3 -m serial.tools.miniterm /dev/ttyUSB0 115200

4. List all devices by ls -l /dev

You will see something like this:

```bash
...
crw-rw-rw-  1 root   wheel        0x9000008 Aug 27 19:05 tty.SLAB_USBtoUART
crw-rw-rw-  1 root   wheel        0x9000004 Aug 27 19:06 tty.usbmodem12202
crw-rw-rw-  1 root   wheel        0x9000006 Aug 27 19:05 tty.usbserial-0001
...
```

In my case:
- `tty.usbmodem12202` is RPI debug probe - this is where rpi bootloader logs are shown
- `tty.SLAB_USBtoUART` is gpio uart - this is where our `kernel` logs are shown
  Try `tty.usbmodem12202` if you do not see any bootloader logs.

5. Connect using tio and power up RPI

# in one terminal
tio /dev/tty.usbmodem12202
# in another terminal
tio /dev/tty.SLAB_USBtoUART

You will see bootloader logs and kernel logs side by side

Chainboot

The host-side terminal and uploader use the FlamingosProject fork of scip instead of the former Ruby miniterm and minipush scripts. The fork adds MiniPush uploads; the serial-console 1.0.1 currently published on crates.io does not include that feature. If the fork is not installed yet, run:

cargo install --git https://github.com/FlamingosProject/scip.git

The same source tree supports two build modes. A normal build produces the kernel that will be sent over UART:

make                    # Raspberry Pi 5 (default)
BSP=rpi4 make           # Raspberry Pi 4

Setting CHAINLOADER=1 builds the UART chainloader instead and writes it to chainloader8.img:

CHAINLOADER=1 make
BSP=rpi4 CHAINLOADER=1 make

For Raspberry Pi 5, the default uses the RP1 UART on GPIO 14/15. To use the firmware-configured debug UART instead, add RPI5_EARLY_UART=1:

CHAINLOADER=1 RPI5_EARLY_UART=1 make

Copy the selected loader to the SD card once. Raspberry Pi firmware must see it as kernel8.img:

CHAINLOADER=1 make copy-kernel-to-sdcard
# or build and install the RPi 5 debug-UART setup plus firmware files:
make prepare-sdcard

Put the card into the Pi and power it on. For the normal development loop, build and upload the regular kernel8.img. scip remains attached as the serial console after the upload completes:

make chainboot

Both modes use 115200 baud. Override the serial device, payload, or baud rate when needed:

DEV_SERIAL=/dev/ttyUSB0 make chainboot
CHAINBOOT_PAYLOAD=path/to/kernel8.img make chainboot
SERIAL_BAUD=115200 make miniterm

With or without chainloader

With the chainloader installed as the SD card’s kernel8.img, every boot works like this:

Firmware → chainloader → wait for UART upload → run kernel from RAM

The uploaded kernel is stored only in RAM, so it disappears after power-off/reset. You must run:

DEV_SERIAL=/dev/tty.usbserial-0001 make chainboot

If you want the Pi to boot the kernel immediately without a host computer, replace the SD-card chainloader with the normal kernel:

CHAINLOADER= make copy-kernel-to-sdcard

Then the flow becomes:

Firmware → normal kernel directly

To restore chainloading later:

CHAINLOADER=1 RPI5_EARLY_UART= make copy-kernel-to-sdcard

Stable kernel tests

The kernel test path uses stable Rust and ordinary Cargo integration-test targets without the host test harness. Each test under kernel/tests is a small no_std, no_main kernel. A native Rust runner converts the test ELF into a boot image, supervises QEMU, propagates the guest exit status, and fails tests that exceed the timeout.

The test commands require qemu-system-aarch64 and rust-objcopy on PATH.

Run the boot smoke test and all integration-test kernels on the QEMU-supported Raspberry Pi 3 BSP:

BSP=rpi3 make test

The targets can also be run separately, or a single integration test can be selected:

BSP=rpi3 make test_boot
BSP=rpi3 make test_integration
BSP=rpi3 TEST=01_timer_sanity make test_integration

Assertions fail through the kernel panic handler. Tests that intentionally provoke a panic call test::expect_panic() immediately before the faulting operation so only that panic is treated as success. RPi5 has no compatible QEMU machine in this repository, so its QEMU-backed test targets report that they are unavailable; RPi5 MMIO mapping invariants are checked at compile time and against the actual layout descriptors during kernel boot instead.

Openocd

make jtagboot
make openocd
make gdb-opt0

See the probe reset proposal for a safe design that requests a firmware reset through SWD when the Debug Probe has no hardware reset signal.

🙌 Acknowledgements

The original version of the tutorials started out as a fork of Zoltan Baldaszti's awesome tutorials on bare metal programming on RPi3 in C. Thanks for giving me a head start!

Translations of this repository

• Chinese
  • @colachg and @readlnh.
  • Need updating.
• Spanish
  • @zanezhub.
  • In the future there'll be tutorials translated to spanish.

License

Licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.