dryoc/

unsafe_code.rs

//! Every non-test use of `unsafe` in dryoc, and why each is sound.
//!
//! This module contains no items; it only documents the unsafe code inventory.
//!
//! Miri uses portable code where AArch64 assembly or NEON intrinsics don't
//! apply. Protected-memory OS calls are native-only: Miri can't enforce page
//! permissions.
//!
//! Kernels below run only after `has_x86_feature!`/`has_aarch64_feature!`
//! confirms their features — runtime detection with `std`, compile-time target
//! features without it. Either way `true` means the CPU has it.
//!
//! Non-test `unsafe` code is limited to these areas:
//!
//! | Area | Feature gate | Why `unsafe` is required |
//! |-|-|-|
//! | `src/wincode_schema.rs` wincode impls, invoked in `src/dryocbox.rs`, `src/dryocsecretbox.rs` and `src/dryocaead.rs` | `wincode_0_6` | The `impl_wincode_schema!` macro implements the `unsafe` wincode schema traits for each Rustaceous `Vec` box wire format (the public-key and secret boxes, and the AEAD boxes and envelopes of both nonce sizes) from one ordered field list: `size_of` sums the fields' own sizes, `write` and `read` visit the same fields in the same order, and `read` initializes the destination once, with a struct literal of the decoded fields, after all of them decode. |
//! | `src/blake2b/mod.rs` parameter block | Always available | `Params::as_bytes` views the `repr(C, packed)` BLAKE2b parameter block as a `[u8; 64]` so the initialization vector is mixed exactly as specified; both backends call it. The parameter type contains only initialized byte fields, has alignment 1, and its size is checked at compile time. |
//! | `src/protected.rs` protected memory | `protected` on Unix/Windows | Calls OS APIs such as `mlock`, `mprotect`, `VirtualLock`, and `VirtualProtect`, implements page-aligned guarded heap buffers, and exposes exact-size byte-array views over protected heap buffers. Each allocation is initialized through its raw pointer before any slice over it exists, and the OS calls take recorded address ranges rather than slices, so no reference is ever created to no-access pages. |
//! | `src/x86_64.rs` and `src/aarch64.rs` CPU feature tokens | Always available on `x86_64` / little-endian `aarch64` | Not `unsafe` themselves: the zero-sized tokens `Avx2`, `Avx512`, `Avx512Vl`, `Avx512Ifma`, `Bmi2` (x86-64) and `Neon`, `Sve2`, `Sha2`, `Sha3` (AArch64) have a private field, and their only constructors are `new` functions that return a token when `has_x86_feature!`/`has_aarch64_feature!` reports every feature it names (runtime detection with `std`, the compile-time target features without it) (plus `Avx2::avx512vl`/`Avx512::avx512vl`, which detect the rest of the `Avx512Vl` set). `Avx512` and `Avx512Ifma` also require `avx2`: rustc's `avx512f` target feature implies it, but std's `avx512f` detection does not check the CPUID `avx2` bit. Every `#[target_feature]` kernel in the rows below is entered through a safe `#[inline(always)]` wrapper that takes its token by value and makes the one `unsafe` call into the `*_unchecked` kernel, whose features the token proves; `sve2` and `sha3` imply `neon`, so `Sve2` and `Sha3` also cover the `"neon,sve2"` and `"neon,sha3"` kernels. The dispatch enums (`Kernel`, `x86_64::LaneSet`) hold the tokens, so the calls into the wrappers are safe. |
//! | `src/poly1305/poly1305_soft.rs` with `src/poly1305/poly1305_x86_64.rs` Poly1305 bulk backends | Always available on `x86_64` | Calls, through `poly1305_x86_64::full_blocks`, a `#[target_feature(enable = "avx2")]`, `#[target_feature(enable = "avx512f")]` or `#[target_feature(enable = "avx512f,avx512ifma")]` (x86-64) one, through the token wrappers of an `Avx2`, `Avx512` or `Avx512Ifma` token. The kernels use only safe value intrinsics and safe slice loads; the x86-64 ones load message blocks through `x86_64::load`/`load512` (`_mm256_loadu_si256`/`_mm512_loadu_si512` on `&[u8; 32]`/`&[u8; 64]`) and key-power lanes through `x86_64::load_words`/`load_words512` (the same intrinsics on `&[u64; 4]`/`&[u64; 8]`). |
//! | `src/poly1305/poly1305_aarch64.rs` Poly1305 block loops | Always available on little-endian `aarch64` | Two `asm!` loops of `poly_block` text (Poly1305-donna-64 in radix 2^64) on the integer registers: `blocks` over four lanes, each reading its contiguous quarter of the caller's input through a post-incremented pointer (the loop runs exactly `len / 64` times, 16 bytes per lane per iteration), and `blocks1` over one lane (exactly `len / 16` iterations of 16 bytes). Every load is therefore in bounds; otherwise register-only, no stack, flags clobbered. |
//! | `src/salsa20/salsa20_neon.rs` XSalsa20 NEON and SVE2 backends | Always available on little-endian `aarch64` | Calls `#[target_feature(enable = "neon")]`, `#[target_feature(enable = "neon,sha3")]` and `#[target_feature(enable = "neon,sve2")]` keystream kernels through the token wrappers of the `Neon`, `Sha3` or `Sve2` token held by its `Kernel` handle. The NEON kernels use only safe intrinsics and safe slice loads and stores. The SVE2 kernel additionally runs the Salsa20 rounds of one vector set and one scalar block in one `asm!` block (or of half the set's rounds and one scalar block in each of two) of `add`/`xar`/`eor` instructions on registers bound as `inout` operands, with no memory access; a second variant also runs a Poly1305 lane beside the scalar block (`mul`/`umulh`/`adds`/`adcs` on general-purpose registers), whose only memory access is `ldp` loads of its 320 bytes of MAC input through a pointer bound as an `inout` operand and advanced 16 bytes at a time. |
//! | `src/chacha20/chacha20_neon.rs` ChaCha20 NEON and SVE2 backends | Always available on little-endian `aarch64` | Calls a `#[target_feature(enable = "neon")]` or `#[target_feature(enable = "neon,sve2")]` keystream kernel through the token wrappers of the `Neon` or `Sve2` token held by its `Kernel` handle. The NEON kernel uses only safe intrinsics and safe slice loads and stores. The SVE2 kernels additionally run the ChaCha20 rounds in `asm!` blocks of `add`/`xar` instructions — one over the 32 vector registers for full chunks (in a second variant also running a companion block's rounds with `add`/`eor`/`ror` on 16 general-purpose registers, and in a third two Poly1305 lanes over 512 bytes of MAC input with `mul`/`umulh`/`adds`/`adcs` on general-purpose registers, whose only memory access is `ldp` loads of those 512 bytes through two pointers bound as `inout` operands and advanced 16 bytes at a time), one over the 16 low registers for runs of up to four blocks — with the registers bound as `inout` operands and no memory access (the full-chunk and four-block blocks loop twice over five double rounds, on a general-purpose counter operand), called only from the `"neon,sve2"` kernels. |
//! | `src/chacha20/chacha20_x86_64.rs` and `src/salsa20/salsa20_x86_64.rs` ChaCha20 and XSalsa20 AVX2/AVX-512 backends, with the `src/x86_64.rs` helpers | Always available on `x86_64` | Each calls a `#[target_feature(enable = "avx2")]`, `#[target_feature(enable = "avx512f")]` or `#[target_feature(enable = "avx2,avx512f,avx512vl")]` keystream kernel through the token wrappers of the `Avx2`, `Avx512` or `Avx512Vl` token held by the `x86_64::LaneSet` in its `Kernel` handle. The kernels and the shared helpers use safe value intrinsics; the only pointer intrinsics are `_mm256_loadu_si256`/`_mm256_storeu_si256` and `_mm512_loadu_si512`/`_mm512_storeu_si512` in `x86_64::load`/`store`/`load512`/`store512` on `&[u8; 32]`/`&[u8; 64]` references (shared or exclusive as the operation needs), which guarantee exactly that many readable or writable bytes and need no alignment. |
//! | `src/scalarmult_curve25519.rs`, `src/edwards25519/mod.rs` and `src/fe25519/mod.rs` Curve25519 BMI2 roots | Always available on `x86_64` | The X25519 ladder (whole, and the projective `ladder_xz` that X-Wing pairs with the base point under one inversion), `mul_base`, `double_scalar_mul_basepoint_vartime`, `Fe::invert` and `Fe::sqrt_ratio_i` each call a `#[target_feature(enable = "bmi2")]` copy of the same safe, inlined arithmetic through its token wrapper when a `Bmi2` token can be constructed, so the `u128` field products compile to `mulx`. These functions contain no intrinsics or `asm!`; the only unsafe operation is the wrapper's call. |
//! | `src/chacha20/chacha20_x86_64.rs` and `src/salsa20/salsa20_x86_64.rs` scalar double rounds beside the AVX-512 lane sets | Always available on `x86_64` | Each `scalar_double_round` is an `asm!` block of base x86-64 `add`/`xor`/`rol`/`mov` instructions: ten (ChaCha20) or nine (Salsa20) state words are `inout` registers and the rest are 4-byte loads and stores at fixed offsets within a `&mut [u32; 6]` / `&mut [u32; 7]` whose pointer is passed in (`nostack`). They keep the companion block of `xor_chunk_avx512_with_block` on the integer ports, where the compiler would otherwise SLP-vectorise it onto the ports the lane set occupies; called only from that `#[target_feature(enable = "avx512f")]` kernel, which the AVX-512 `Kernel` handles reach through the wrapper of their `Avx512` token. |
//! | `src/argon2/argon2_x86_64.rs` Argon2 AVX2 and AVX-512 block compression | `alloc` on `x86_64` | The safe `Kernel::fill_block` runs the permutation `P` in place on the block by calling the `#[target_feature(enable = "avx2")]` function `permute_avx2_unchecked` or the `#[target_feature(enable = "avx512f")]` function `permute_avx512_unchecked` through the token wrapper of the `Avx2` or `Avx512` token held by its `Kernel` handle. The kernels use safe value intrinsics; block words are loaded and stored through `x86_64::load_words`/`store_words`/`load_words512`/`store_words512` (`_mm256_loadu_si256`/`_mm256_storeu_si256`/`_mm512_loadu_si512`/`_mm512_storeu_si512` on `&[u64; 4]`/`&[u64; 8]` references, shared or exclusive as the operation needs, which guarantee exactly 32 or 64 readable or writable bytes and need no alignment). |
//! | `src/argon2/argon2_neon.rs` Argon2 SVE2 block compression | `alloc` on little-endian `aarch64` (outside Miri) | The safe `Kernel::fill_block` runs the permutation `P` in place on the block by calling the `#[target_feature(enable = "neon,sve2")]` function `permute_unchecked` through the token wrapper of the `Sve2` token held by its `Kernel` handle. Each of its two `asm!` passes (rows, then columns) loads, permutes and stores three 16-word states on the vector registers and five on the integer registers, through pointers derived from the block's one `&mut` borrow and advanced only over the words of those eight states, which are in the block and each belong to one state; it uses no stack and declares every register it writes. |
//! | `src/mlkem/mlkem_x86_64.rs` ML-KEM AVX2 polynomial arithmetic | Always available on `x86_64` | Calls the `#[target_feature(enable = "avx2")]` NTT, inverse NTT and base-multiplication kernels through the token wrappers of the `Avx2` token held by its `Kernel` handle. The kernels use safe value intrinsics; coefficients and twiddle tables are loaded and stored through `x86_64::load_i16s`/`store_i16s` (`_mm256_loadu_si256`/`_mm256_storeu_si256` on `&[i16; 16]` references, shared or exclusive as the operation needs, which guarantee exactly 32 readable or writable bytes and need no alignment). |
//! | `src/keccak/keccak_x86_64.rs` 4-way Keccak-p\[1600\] AVX2 permutation | Always available on `x86_64` | `permute_lanes` (behind `ParSponge`) calls the `#[target_feature(enable = "avx2")]` function `permute4_avx2_unchecked` through the token wrapper of the `Avx2` token held by its `Kernel` handle. The kernel uses safe value intrinsics; the four states (disjoint `&mut [u64; 25]` references from `get_disjoint_mut`) are loaded and stored through `x86_64::load_words`/`store_words` (`_mm256_loadu_si256`/`_mm256_storeu_si256` on `&[u64; 4]` references, which guarantee exactly 32 readable or writable bytes and need no alignment). |
//! | `src/keccak/keccak_aarch64.rs` 2-way Keccak-p\[1600\] SHA3-extension permutation | Always available on little-endian `aarch64` | `ParSponge` and `permute_lanes` call the `#[target_feature(enable = "neon,sha3")]` function `permute2_sha3_unchecked` through the token wrapper of the `Sha3` token held by its `Kernel` handle. The kernel uses only safe value intrinsics and reads and writes the states through their `&mut [u64; 25]` references (disjoint ones from `get_disjoint_mut` for lanes of one array). |
//! | `src/keccak/keccak3_aarch64.rs` 3-way Keccak-p\[1600, 24\] permutation (generated by `keccak3_aarch64.py`) | Always available on little-endian `aarch64` | `Kernel::permute_selected` in `keccak_aarch64.rs` calls the `#[target_feature(enable = "neon,sha3")]` function `permute3_unchecked` through the `permute3` wrapper of the `Sha3` token held by its `Kernel` handle. The function is one `asm!` block with every register it writes an output operand; its only memory accesses are 48 post-incremented loads of the static round-constant table `RC` over its three loop passes and loads and stores in an 80-byte stack frame it allocates below the stack pointer (eight spill slots and the pass counter), which it wipes and releases before it ends. The states are loaded into and stored from its operands through their `&mut [u64; 25]` references (disjoint ones from `get_disjoint_mut` for lanes of one array). |
//! | `src/keccak/keccak_soft.rs` lazy-rotation scalar Keccak-p\[1600\] | Always available on `aarch64` (outside Miri) | `xor_rol` and `bic_rol` are each one register-only `asm!` instruction (`eor` or `bic` with a rotated second operand) on two `u64` inputs and one output, with no memory, stack or flag access (`pure`, `nomem`, `nostack`, `preserves_flags`); the rotation amount is a `const` operand. Under Miri they are the plain Rust expressions. |
//! | `src/blake2b/blake2b_x86_64.rs` BLAKE2b AVX2 and AVX-512VL compression | Always available on `x86_64` (soft backend) | Calls a `#[target_feature(enable = "avx2")]` or `#[target_feature(enable = "avx2,avx512f,avx512vl")]` `compress` through the token wrapper of the `Avx2` or `Avx512Vl` token held by its `Kernel` handle. The kernels use safe value intrinsics; the block and chaining state are loaded and stored through `x86_64::load`/`load_words`/`store_words` (`_mm256_loadu_si256`/`_mm256_storeu_si256` on `&[u8; 32]`/`&[u64; 4]` references, which guarantee exactly 32 readable or writable bytes and need no alignment). |
//! | `src/blake2b/blake2b_aarch64.rs` BLAKE2b rounds | Always available on little-endian `aarch64` (soft backend) | `rounds` is an `asm!` block of base A64 `ldr`/`add`/`ror`/`eor` instructions: the sixteen working-state words are `inout` registers, and the only memory accesses are 192 8-byte loads (two per `G`) at immediate offsets within the 128-byte block whose pointer is passed in (`readonly`, `nostack`, `preserves_flags`). It pins the two-instruction-deep `G` step schedule that LLVM folds back into three. |
//! | `src/chacha20/chacha20_aarch64.rs` scalar ChaCha20 rounds | Always available on `aarch64` | `rounds` is a register-only `asm!` block of base A64 `add`/`ror`/`eor` instructions over the 16 state words, bound as `inout` operands with no memory access (`nomem`, `nostack`); it exists to pin the two-instruction-deep quarter-round schedule that LLVM folds back into three. Used by HChaCha20 and the scalar block function. |
//! | `src/sha256/sha256_aarch64.rs` SHA-256 hardware compression | Always available on little-endian `aarch64` | `Sha256` calls the `#[target_feature(enable = "sha2")]` function `compress_unchecked` through the token wrapper of a `Sha2` token. That function runs the whole block loop in one `asm!` block that reads the caller's `&[[u8; 64]]` blocks and the `K32` table, and reads and writes the eight-word state through its `&mut` pointer; only the listed registers are clobbered and it touches no stack. |
//! | `src/sha512/sha512_aarch64.rs` SHA-512 hardware compression | Always available on little-endian `aarch64` | `Sha512` calls the `#[target_feature(enable = "sha3")]` function `compress_unchecked` through the token wrapper of a `Sha3` token. That function runs the whole block loop in one `asm!` block that reads the caller's `&[[u8; 128]]` blocks and the `K64` table, and reads and writes the eight-word state through its `&mut` pointer; only the listed registers are clobbered and it touches no stack. |
//! | `src/sha512/sha512_aarch64.rs` SHA-512 two-state hardware compression | Always available on little-endian `aarch64` | `Sha512::absorb_key_blocks` calls the `#[target_feature(enable = "sha3")]` function `compress2_unchecked` through the token wrapper of a `Sha3` token. Its single `asm!` block compresses one block into each of two independent states with the two round sequences interleaved (the HMAC inner and outer key blocks); it reads the two `&[u8; 128]` blocks and the `K64` table, reads and writes both eight-word states through their `&mut` pointers, clobbers every vector register and `x3`, and touches no stack. |
//! | `src/edwards25519/edwards25519_x86_64.rs` basepoint table lookup | Always available on `x86_64` | `select_row` calls the `#[target_feature(enable = "avx512f")]` function `edwards25519_x86_64::select_row_unchecked` through the token wrapper of an `Avx512` token. The function uses safe value intrinsics (a vector compare of the digit and masked blends over every entry), reads every table entry regardless of the digit, assembles its result through `x86_64::store_words512`, and writes it into caller-owned storage that `mul_base` wipes. |
//! | `src/edwards25519/edwards25519_neon.rs` basepoint table lookup | `aarch64` with `neon` enabled at build time (every std AArch64 target) | The module is compiled only under `cfg(target_feature = "neon")`. Its `select_row` and `select_row64` (the four-limb table copy) are `#[inline(always)]` functions whose bodies are each one `unsafe` block of NEON intrinsics (`and`/`orr` on vectors built from limbs; `select_row64` loads its limb pairs with `vld1q_u64` from two-element subslices of the table, in bounds and `u64`-aligned); the block is valid because the feature is statically enabled. Each reads every entry of its row regardless of the digit; `select_row` writes the result into caller-owned storage and `select_row64` returns it. |
//! | `src/mlkem/mlkem_neon.rs` ML-KEM NEON polynomial arithmetic and matrix rejection sampling | Always available on little-endian `aarch64` | Calls the `#[target_feature(enable = "neon")]` forward NTT, inverse NTT, NTT-domain multiply-add, reduction, message encoding, ciphertext decompression and rejection-sampling kernels through the token wrappers of the `Neon` token held by its `Kernel` handle. The kernels use safe value intrinsics; the only pointer intrinsics are `vld1q_s16`/`vst1q_s16` in `load`/`store` on `&[i16; 8]`/`&mut [i16; 8]` rows of the polynomials and twiddle tables, and `vld1q_u8`/`vld1q_u16` in `load_u8`/`load_u16` on `&[u8; 16]`/`&[u16; 8]` sampler input bytes and constant tables, which guarantee exactly 16 readable or writable bytes and need no alignment beyond the element type's. |
//! | `src/edwards25519/mod.rs` `mul_base` loop alignment | Always available on `aarch64` | Each of the two loop bodies starts with an `asm!` block holding only a `.p2align 6` directive, which emits `nop` padding and touches no register, memory or flag (`nomem`, `nostack`, `preserves_flags`); it pins the loops' alignment, whose drift with unrelated code cost up to 12%. |
//! | `src/fe25519/fe25519_aarch64.rs` Curve25519 field multiply and square | Always available on `aarch64` | `mul`, `square` and `square_chain` are register-only `asm!` blocks of base A64 integer instructions (`mul`, `umulh`, `adds`/`adc`, `extr`, `madd`, `and`) computing one radix-2^51 field product; every written register is a declared output or scratch operand and the blocks are `pure`, `nomem`, `nostack`. |
//! | `src/fe25519/safegcd.rs` divsteps of the Curve25519 inversion | Always available on `aarch64` (outside Miri) | `divsteps_59` is one register-only `asm!` loop of base A64 integer instructions (`csel`, `cinv`, `ccmp`, `tst`, `and`, `add`, shifts) running the 59 branch-free divsteps of one inversion batch; every written register is a declared output or scratch operand, the flags are clobbered, the only branch is the fixed-count loop, and the block is `pure`, `nomem`, `nostack`. |
//! | `src/fe25519/fe64_aarch64.rs` four-limb Curve25519 field arithmetic for the X25519 ladder and the basepoint multiplication | Always available on `aarch64` | `Fe64::add`, `sub`, `mul`, `square` and `mul_121666_add` are register-only `asm!` blocks of base A64 integer instructions (`mul`, `umulh`, `adds`/`adcs`/`adc`, `subs`/`sbcs`, `csel`, `add`/`sub`) computing one result modulo `2^256 - 38`; every written register is a declared output or scratch operand and the blocks are `pure`, `nomem`, `nostack`. |
//! | `src/wasm32.rs` WebAssembly `simd128` loads and stores, used by the `src/chacha20/chacha20_wasm32.rs`, `src/salsa20/salsa20_wasm32.rs` and `src/mlkem/mlkem_wasm32.rs` kernels | `wasm32` built with the `simd128` target feature | `load`/`store` and `load_i16s`/`store_i16s` call `v128_load`/`v128_store` on `&[u8; 16]`/`&mut [u8; 16]` and `&[i16; 8]`/`&mut [i16; 8]` references, which guarantee exactly 16 initialized readable or writable bytes; both instructions are unaligned (align 1) accesses. The kernels, and the Poly1305 and Keccak `simd128` kernels, otherwise use only safe value intrinsics. `simd128` is a compile-time target feature, so there is no runtime check or `#[target_feature]` call to justify. |
//! | `src/argon2/argon2_soft.rs` round output stores | `wasm32` built with the `simd128` target feature | `store_word` writes each of a round's sixteen output words with `core::ptr::write_volatile` through a `&mut u64` into the block, so the pointer is valid, aligned and initialized. A volatile store is not a seed for LLVM's SLP vectorizer, which would otherwise pair the four `G` columns into `i64x2` lanes whose multiply engines emulate at half the scalar speed; other builds use a plain assignment. |
//! | `src/argon2/argon2_soft.rs` `G` XOR-rotations | Always available on `aarch64` (outside Miri) | `xor_ror` is one register-only `asm!` instruction (`eor` with a rotated second operand) on two `u64` inputs and one output, with no memory, stack or flag access (`pure`, `nomem`, `nostack`, `preserves_flags`); the rotation amount is a `const` operand. Other targets and Miri use the plain Rust expression. |
//! | `src/utils.rs` word-wise zeroization | Always available | `zeroize_bytes`, `zeroize_u32s`, `zeroize_u64s` and `zeroize_i16s` (behind `WideZeroizing`) view the 16-byte-aligned middle of a byte, `u32`, `u64` or `i16` slice as `u128`s (`align_to_mut`) and clear each with a volatile store, so wiping a buffer costs one store per sixteen bytes instead of one per element. Unaligned ends use the `zeroize` crate. |
//! | `src/pwhash.rs` `PwHash::into_parts` | `alloc` | Uses `ManuallyDrop` and reads each owned field exactly once so the hash's drop-time zeroization does not erase the value while transferring ownership to the caller. |
//!
//! Test-only unsafe (libsodium/Argon2 checks, protected-memory probes) isn't
//! part of the runtime API.