Formal methods for the unsafe side of the Force
In a recent blog post, Michael Gibson talked about the framework we set up to safely bridge between a single-threaded C++ codebase and a multi-threaded Rust component. This framework gives us an ergonomic way to interact with C++ types that are not fully thread-safe in multi-threaded async Rust.
In the implementation, we effectively reduce all the thread-safety arguments down to the existence of two special primitive types:
SendWrapper<T>: Sendallows you to wrap aT: !Sendand smuggle it across threads.MainThreadTokenis a witness of execution that exists solely on the main thread.
Both play some unusual tricks on the Send and Sync traits that are not expressible in safe Rust. But we can’t just switch to unsafe and gain unlimited power. We are now burdened with the obligation of showing no runtime abnormalities will occur:
![Palpatine: The [strikethrough: dark] unsafe side of [strikethrough: the Force] Rust is a pathway to many abilities some consider to be unnatural…](/img_opt/buLwY_1AQh-1920.jpeg)

By the end, we’ll formally prove the correctness of our primitives in the RustBelt logical framework.1 And along the way, we’ll gradually build up intuitions on the formal semantics of Rust types and what it really means for them to be Send and Sync.
Send & Sync, semi-formally
Let’s start by discussing the Send and Sync bounds a bit more formally to build up intuitions. Paraphrasing the Rustonomicon slightly:
- A type
TisSendif it is safe to expose exclusive access to T (asTor&mut T) across different threads.2 - A type
TisSyncif it is safe to expose shared access toT(as&T) across different threads.
What do we mean by “safe to expose exclusive/shared access”? Well, the end goal here is making T sound to use for external safe code if we add the Send/Sync bounds. This means that if we expose exclusive/shared access across different threads, and interleave that ability with any sequence of public safe operations we expose on T, we should still be unable to trigger any undefined behavior at run-time.
Thinking through all possible external interactions on T and checking each one against the big list of undefined behaviors is a daunting task. Fortunately, by formalizing and reasoning about invariants of T, we can reduce this task into smaller, more manageable steps:
- Identify some predicate
Pto serve as an invariant overT. - Prove that
Preally is an invariant:- Show that constructing a value of
TestablishesP. That is, for functionsU -> Tthat map intoT, try to provePassumingU’s invariant is true. - Dually, show that every safe operation you expose on
Tpreserves the invariantP. That is, for functionsT -> Uthat map out ofT, you can now assumePand try to proveU’s invariant is true.
- Show that constructing a value of
- Looking only at
P, deduce whetherPis logically compatible with cross-thread access. If it is, and since we’ve shown individual safe operations preserveP, we know any sequence of safe operations will still be compatible with cross-thread access.
Owning and sharing
We can use the method of the last section to formalize the semantics of Rust types. The most straightforward way of writing down invariants of a type T is to provide a formal model. In most languages, it means providing a predicate that decides what values are (and aren’t) valid inhabitants of T.
In Rust, however, with its ownership model and attention to low-level programming, modelling a type T instead requires you to demarcate which raw byte patterns in the memory of an executing thread can be considered an owned value of T. Formally, we say every Rust type T has an associated owning predicate to encode these semantics. In formal logic pseudo-code, similar to that used by RustBelt, we can write this as:
T.own(tid, val) := ...This predicate specifies whether the thread tid can have the memory bytes val as a valid owned value of T. For example, the owning predicate of bool is true exactly when the memory value is an initialized single byte with either the bit pattern 0x00 or 0x01; and the owning predicate of u8 is true exactly when the memory value is an initialized single byte (and all bit patterns are valid). Expressed formally, this looks like:
bool.own(tid, val) := (val = [0x00]) ∨ (val = [0x01])
u8.own(tid, val) := val.length = 1Interestingly, since Rust allows creation of shared borrows on types (&T), each type T also has an associated sharing predicate:
T.share('a, tid, val) := ...This predicate decides whether the thread tid can have shared access for lifetime 'a to the memory bytes val as a value of T.
As it turns out, sharing predicates are related to, but not completely determined by the owning predicates. That means one has to specify both the owning and sharing predicates to completely model the semantics of a type. In fact, for example, Cell<u8> and u8 have the exact same owning predicates but very different sharing predicates. That difference allows you to use .set(new_val) to mutate through a &Cell<u8>, but not through a &u8.
Send & Sync, formally
The owning and sharing predicates on T help us capture exactly what it means to have exclusive access (T) and shared access (&T). Under this formalism, we have the following definition of whan a type is Send:
A type
TisSendexactly when its owning predicateT.own(tid, val)is independent from thetidparameter.
This means that whether you can validly own a value of T does not depend on which thread you are on.
Symmetrically, for Sync we have:
A type
TisSyncexactly when its sharing predicateT.share('a, tid, val)is independent from the thread parametertid.3
For example, when T internally owns some U, usually you can only claim T: Send when U: Send. Since T.own(tid, _) likely refers to U.own(tid, _) in some way, T.own is thread-independent only if U.own is.
Similarly, when T internally owns some reference &U, you usually have T: Send only when U: Sync. Since T.own(tid, _) now refers to U.share(tid, _), and T.own is thread-independent only if U.share is.
Formally modelling SendWrapper<T>
Now let’s apply this modelling process to SendWrapper<T>. Here’s its public interface:
unsafe impl<T> Send for SendWrapper<T> {} // ??
unsafe impl<T> Sync for SendWrapper<T> {} // ??
pub struct SendWrapper<T>(T);
impl<T> SendWrapper<T> {
pub fn new(val: T) -> Self;
/// # Safety
/// In general only safe to call from the same thread that originally constructed `self`.
pub unsafe fn unwrap_unchecked(self) -> T;
}
impl<T: Sync> Deref for SendWrapper<T> {
type Target = T;
fn deref(&self) -> &T;
}
impl<T> Drop for SendWrapper<T> {
fn drop(&mut self) {
panic!("Hey, no dropping here!");
}
}First, what are the owning/sharing predicates of SendWrapper<T>? We know SendWrapper<T> has to, in some form, own a T, because we rely on that to expose accessor methods like unwrap_unchecked and the Deref impl. But we can’t directly say:
SendWrapper<T>.own(tid, val) := T.own(tid, val)since we promise SendWrapper<T>: Send even when T: !Send. Instead, we introduce an existential to unbind the tid. We can’t claim the current thread owns T, but we can claim that there exists some thread that owns T. We do the same for sharing:
SendWrapper<T>.own(tid, val) := ∃ tid'. T.own(tid', val)
SendWrapper<T>.share('a, tid, val) := ∃ tid'. T.share('a, tid', val)This results in somewhat curious semantics: owning a SendWrapper<T> on the current thread does not give you full rights to own T. Instead, you get a certificate that some other thread (maybe you, we don’t know) can fully own T. You are only allowed to extract the T from the wrapper if you can prove that it is valid for your thread to directly own the underlying inhabitant of T.4
After we decoupled the owning thread of SendWrapper<T> and that of T, the invariants we specify are trivially thread-independent. That’s why we can claim SendWrapper<T>: Send + Sync. What remains is to show they are compatible with the operations we expose:
- First, in Rust you can always obtain a shared reference from an owned value by borrowing it. That is, we need to start from
SendWrapper<T>.own(tid, val)and deriveSendWrapper<T>.share('a, tid, val). This can be done since knowing some thread can ownTmeans that same thread can perform a borrow and satisfiesT.share. newis the only public constructor, and is effectively the true implicationT.own(tid, val) -> ∃ tid'. T.own(tid', val).unwrap_uncheckedis effectively the implication∃ tid'. T.own(tid', val) ∧ tid = tid' -> T.own(tid, val), with thetid = tid'condition provided by the safety requirement.- The
Derefimplementation requires us to obtain a&Ton the current thread when we only know some thread can borrowT. But remember we also requireTto beSync, and that means if some thread can have&T, all threads (including the current thread) can as well! - Finally the
DroponSendWrapper<T>has to be banned by a run-time panic. Otherwise the default destructor will effectively unwrap the existential without checking and violates our invariants.
The real machine-checkable version of this formalism in RustBelt is done through a separation logic framework implemented in Rocq. There are some more technical details, but the overall idea remains the same. Expand to see the full proof for SendWrapper<T>:
Formally modelling MainThreadToken
Now let’s apply the same modelling process to MainThreadToken. MainThreadToken has a smaller interface:
#[derive(Clone, Copy)]
pub struct MainThreadToken(PhantomData<*mut ()>);
// Owning a `*mut ()` here ensures !Send and !Sync
impl MainThreadToken {
/// # Safety
/// This function must be called from the main fuzzer thread.
pub unsafe fn new() -> Self;
}The owning predicate encodes the requirement that MainThreadToken is only supposed to live on the main thread, and MainThreadToken itself is merely a zero-sized type:
MainThreadToken.own(tid, val) := (tid = MAIN_THREAD_ID) ∧ (val = [])For convenience, we also mark MainThreadToken as Copy, which allows us to freely duplicate them from a shared borrow &MainThreadToken. This requires the sharing predicate to also bear the “only living on the main thread” semantics:
MainThreadToken.share('a, tid, val) := (tid = MAIN_THREAD_ID)This time those predicates’ truth value depends on the exact value of the current thread id tid, and that explains why we have to mark MainThreadToken: !Send + !Sync.
Expand to see the full Rocq proof for the safety of MainThreadToken:
A few unnatural abilities
Phew, that’s quite a lot of sweat and tears for us to finally claim that SendWrapper and MainThreadToken are safe abstractions!
This wasn’t just about the formal proofs we got at the end, though. Thinking through how to model Send and Sync at this level of detail has given us a better understanding of thread safety in Rust.
One fact we came to appreciate along the way is that in safe Rust, the compiler is essentially doing all the proofs that we’re manually verifying here. A good strategy for working with unsafe interfaces, then, is to express that with as much safe code as you can and concentrate the dark side of the Force into a few unnatural abilities reduce the unsafeness to the existence of a few primitive types. This way, we can focus our energy more effectively on proving the correctness of a smaller target.
If you’ve made it through this second post as well, and are still thinking “what I don’t like about this post series is that it’s too short and doesn’t go into the weeds enough,”… well, that’s all we have for you for now, but we can point you in the direction of more good things.
If you want to learn even more about formal Rust semantics, the original RustBelt paper and Ralf Jung’s PhD thesis are great starting points. For formal verification of low-level system programming with concurrency, a general separation logic framework like Iris might be the current best tool.