Skip to content

Data Transfer Protocol From Scratch

In this guide, we'll explore how to implement a custom data transfer protocol that allows data to be transferred between an STM32H7 microcontroller and a computer over USB.

We'll be using Rust 🦀 for the implementation, and to ensure data integrity and keep things simple, we'll apply Consistent Overhead Byte Stuffing (COBS) encoding.

The messages will have a maximum length of 64 bytes, which helps us avoid dynamic memory allocation — something crucial when working with embedded systems that have limited resources.

Protocol Spec ​

Our protocol provides a straightforward and reliable way to send messages between a computer (the host) and an STM32H7 microcontroller (the device) using USB. Here's a breakdown of how it works:

  • Data Encoding: We use COBS to encode the data. COBS ensures there are no zero bytes in the data stream, making it easier to detect the boundaries of each message.
  • Message Structure: Each message can be up to 64 bytes long. If needed, the message can be split into smaller chunks. Control bytes are used to indicate whether a chunk is part of an ongoing message, the end of a message, or a specific command.
  • Control Bytes:
    • 0x01: The message continues in the next chunk.
    • 0x02: This is the last chunk of the message.
    • 0x03: Erase command (no data payload).
    • 0x04: Termination command (no data payload).
  • Acknowledgment: After the microcontroller receives each chunk or command, it sends an acknowledgment byte (0x06) back to the host.

Message Structure ​

  • Control Byte (1 byte): Indicates the type of message and whether it’s the last chunk.
  • Payload (0-61 bytes): The actual data, which is encoded with COBS.
  • End Marker (1 byte): Marks the end of the message (always 0x00).

Sender Implementation on PC ​

The sender code on the PC is responsible for preparing, encoding, and sending data chunks to the STM32H7 microcontroller over USB. Let's break down the implementation.

Setting Up USB Communication ​

The first step is to establish a USB connection with the STM32H7 device. We’ll use the rusb library, a Rust wrapper for the libusb[1] library, to handle this.

rust
fn open_device(vendor_id: u16, product_id: u16) -> rusb::Result<DeviceHandle<Context>> {
    let cx = Context::new()?;

    let device = cx
        .devices()?
        .iter()
        .find(|d| {
            d.device_descriptor().ok().map_or(false, |desc| {
                desc.vendor_id() == vendor_id && desc.product_id() == product_id
            })
        })
        .ok_or(rusb::Error::NoDevice)?;

    let handle = device.open()?;
    handle.claim_interface(1)?;
    handle.set_active_configuration(1)?;

    Ok(handle)
}

This function searches for and opens the USB device using its Vendor ID (VID) and Product ID (PID), then claims the right interface and sets the active configuration. Once this setup is done, we can start transferring data.

Command Handling and COBS Encoding ​

Our protocol supports three main commands: Erase, Write, and Bye. These commands are represented by the Command enum, and each has a method to send it over USB.

rust
impl Command {
    fn send(&self, handle: &mut DeviceHandle<Context>) -> rusb::Result<usize> {
        match self {
            Self::Erase => {
                let written = handle.write_bulk(WRITE_ENDPOINT, &[0x03, 0x0], TIMEOUT)?;
                if !wait_for_ack(handle)? {
                    return Err(rusb::Error::Other);
                }
                Ok(written)
            }
            Self::Bye => {
                let written = handle.write_bulk(WRITE_ENDPOINT, &[0x04, 0x0], TIMEOUT)?;
                if !wait_for_ack(handle)? {
                    return Err(rusb::Error::Other);
                }
                Ok(written)
            }
            Self::Write { payload } => {
                let mut total = 0;
                for chunk in prepare_message(&payload) {
                    let written = handle.write_bulk(WRITE_ENDPOINT, &chunk, TIMEOUT)?;
                    if !wait_for_ack(handle)? {
                        return Err(rusb::Error::Other);
                    }
                    total += written;
                }
                Ok(total)
            }
        }
    }
}

Each command is sent as a USB bulk transfer. After sending, the sender waits for an acknowledgment from the STM32H7 device using wait_for_ack, which helps ensure that the data was received correctly.

Preparing Messages with COBS ​

The prepare_message function breaks down the payload into chunks, encodes each chunk with COBS, and adds control bytes to manage the flow of the message:

rust
pub fn prepare_message(data: &[u8]) -> Vec<Vec<u8>> {
    let mut result = Vec::new();
    let chunks = data.chunks(CHUNK_SIZE);
    let num_chunks = chunks.len();

    for (i, chunk) in chunks.enumerate() {
        let mut message = encode_vec(chunk);

        if i == num_chunks - 1 {
            message.insert(0, 0x02); // Last chunk
        } else {
            message.insert(0, 0x01); // Continued chunk
        }
        message.push(0x0); // End marker

        result.push(message);
    }

    result
}

This function ensures that each chunk fits within the 64-byte limit and includes the necessary control information, so the receiver can properly reconstruct the original data.

Sending Data ​

Here’s how the main function uses the Command enum to erase memory, write a payload, and send a termination command:

rust
fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut handle = open_device(VID, PID)?;
    println!("found: {:?}", handle);

    let very_long: Vec<_> = (0..69).map(|v| v).collect();

    Command::Erase.send(&mut handle)?;
    Command::Write { payload: very_long }.send(&mut handle)?;
    Command::Bye.send(&mut handle)?;

    println!("done");
    Ok(())
}

This setup ensures that data is transferred in a controlled manner, with the microcontroller acknowledging each chunk before moving on to the next one.

Receiver Implementation on STM32H7 ​

On the STM32H7 side, the receiver’s job is to decode the incoming COBS-encoded data, interpret the command, and respond as needed.

The STM32H7’s implementation[2] of the Command enum looks like this:

rust
enum Command {
    Erase,
    Write {
        continued: bool,
        data: [u8; 64],
        len: usize,
    },
    Bye,
}

Command Parsing ​

We implement the TryFrom trait to parse raw data into commands. For our purposes, we use a generic byte array type [u8; N]:

rust
impl<const N: usize> TryFrom<[u8; N]> for Command {
    type Error = ParseError;

    fn try_from(value: [u8; N]) -> Result<Self, Self::Error> {
        if let Some((control, payload)) = value.split_first() {
            match control {
                0x01 | 0x02 => {
                    let mut data = [0u8; 64];
                    let len = cobs::decode(&payload, &mut data)
                        .map_err(|_| ParseError::Decoding)?;

                    return Ok(Self::Write {
                        continued: *control == 0x01,
                        data,
                        len,
                    });
                }
                0x03 => return Ok(Self::Erase),
                0x04 => return Ok(Self::Bye),
                _ => return Err(ParseError::Unknown),
            }
        }

        Err(ParseError::Malformed)
    }
}

This method ensures that any malformed or unknown commands are handled appropriately, minimizing the chances of unexpected behavior.

Handling USB Events ​

The usb_handler task manages USB communication on the STM32H7, processing incoming data and sending acknowledgments:

rust
#[task(binds = OTG_FS, local = [usb, buf: [u8; 64] = [0; 64]])]
fn usb_handler(cx: usb_handler::Context) {
    let usb_handler::LocalResources { usb, buf } = cx.local;
    let (usb_dev, serial) = usb;

    loop {
        if !usb_dev.poll(&mut [serial]) {
            return;
        }

        match serial.read(buf) {
            Ok(count) if count > 0 => {
                match Command::try_from(*buf) {
                    Ok(cmd) => match cmd {
                        Command::Erase => defmt::info!("erase"),
                        Command::Bye => defmt::info!("done."),
                        Command::Write {
                            continued,
                            data,
                            len,
                        } => {
                            defmt::info!("write continues? {}, {:?}", continued, data[..len])
                        }
                    },
                    Err(e) => panic!("error parsing command: {:?}", e),
                }

                serial.write(&[0x06, 0x00]).expect("could not send ack");
            }
            Err(e) => panic!("error receiving serial data: {:?}", e),
            _ => {} // no data
        }
    }
}

This function processes incoming USB data, tries to parse it into a command, and then executes the command logic, whether that’s erasing memory, writing data, or terminating. After successfully processing a command, the microcontroller sends an acknowledgment back to the host.


  1. On Windows machines, you might need to use Zadig to install a libusb-compatible driver for the STM32. ↩︎

  2. We need it to be different because on the receiver we will get two variations of Write: a continued and the last one, each with a certain payload length. ↩︎