Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,8 @@ jobs:
- name: Clang
run: nix develop .#clang --command bash -c 'export CC=gcc; just ffi test cpp && just ffi moq-cpp'

# The C++ header with the compilers the job above lacks, through the same checks.
# The C++ header with the compilers the job above lacks, through the same checks, and on
# macOS the Swift with Apple's toolchain.
ffi-cpp:
name: Export moq-net to C++ (${{ matrix.name }})
runs-on: ${{ matrix.os }}
Expand Down Expand Up @@ -234,6 +235,12 @@ jobs:
- name: Export
run: just ffi moq-cpp

# The Swift through the checks the merge gate runs with nixpkgs' Swift on Linux, then
# moq's session, reusing the builds above.
- name: Swift
if: runner.os == 'macOS'
run: just ffi test swift && just ffi moq-swift

# The Kotlin on ART, shrunk by R8 as an app shrinks it. x86_64, as moq's Android build:
# this image ships the Android SDK and NDK, and its KVM runs an x86_64 emulator at speed.
ffi-android:
Expand Down
5 changes: 5 additions & 0 deletions ffi/MIGRATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

## Unreleased

- Every export writes `swift/<prefix>.swift`, whose runtime claims the C names
`<prefix>_atomic_load`, `<prefix>_atomic_store`, `<prefix>_atomic_add`,
`<prefix>_atomic_exchange`, `<prefix>_atomic_cas`, and `<PREFIX>_ATOMICS_H`. An item that
clashes with one, or whose Swift name clashes with another's or with a standard library
name the Swift uses, such as `Data`, takes a `rename`.
- Kotlin returns bytes as a `Bytes`, not a `ByteArray`: Rust's buffer as `bytes.buffer`, a
read-only `ByteBuffer`, copied up to 1 KiB and otherwise a direct one lent uncopied
until `bytes.close()` frees it. Read it in `use { }`, or copy it out with
Expand Down
76 changes: 66 additions & 10 deletions ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ A `[replace]` from the source it replaces, which no `[patch]` can carry, is an e
Beside the C header, `include/<prefix>.hpp` holds the same API in C++, and
`kotlin/<prefix>.kt` holds it in Kotlin, over JNI functions the crate builds with
`--features kotlin`, and `kotlin/<prefix>.pro` the R8 and ProGuard rules keeping what
those functions look up by name.
those functions look up by name, and `swift/<prefix>.swift` holds it in Swift, over the C
header as a clang module.

```toml
# ffi.toml, beside the package's Cargo.toml unless --manifest-path says otherwise.
Expand Down Expand Up @@ -272,6 +273,52 @@ in as a `ByteArray`, bytes returned as their owner, and anything else encoded in
Android library ships it as a consumer rule (`consumerProguardFiles`), which its AAR
carries to each app that shrinks it.

## Swift

`swift/<prefix>.swift` is Swift over the C ABI, for Apple's platforms and Linux. It imports
the header as the clang module `<Prefix>FFI`, which `swift/module.modulemap` declares
beside `swift/<prefix>_atomics.h`, the atomics its runtime needs that Swift's standard
library has only from macOS 15: a build puts `swift/` on its import path and links the
static library. It needs Swift 6 and, for `Swift.Duration`, macOS 13 or iOS 16, and builds
in either language mode.

- **Names** are as Kotlin's, at the module's top level: types in PascalCase without the
prefix, members in camelCase without their type's name (`TrackProducer.appendGroup`). A
keyword is quoted in backticks, and a member named as one Swift gives its type, such as
`init`, takes a trailing `_`. Parameters have no labels, so a call reads as Rust's, and a
constructor such as `new` stays a static function: `Counter.new(1)`.
- **Objects** are final classes owning their handle, freed in `deinit`, or sooner by
`close()` once no call holds it; a call after it traps. A `self` method, or passing the
object by value, closes it once the call returns. `Clone` is `clone()`, and `PartialEq`
is `==`. A field is a computed property, and its setter a function, `setFirst(value)`,
since setting an object moves it in and closes its handle, which an assignment would
hide. An object is `Sendable`, since Rust checks each borrow at run time.
- **Records** are structs and **enums** Swift enums, with a variant's data as associated
values, all `Sendable`. `PartialEq` is `==`, through Rust's; `Display` is `description`,
`Debug` `debugDescription`, and `Default` a static `default()`.
- **Bytes** are `Data`. Passed in at a call's top level, Rust reads Swift's own; nested in
another value, they are copied in. Bytes Rust returns lend its buffer until Swift
releases the `Data`, though `Data` copies a few bytes inline, freeing Rust's at once.
Strings are `String`, copied, `char` `Unicode.Scalar`, `usize` `UInt`, `Option<T>` `T?`,
a tuple a Swift tuple, though a tuple of one is its element, and `Duration`
`Swift.Duration`. An `Err` is thrown: an exported type as itself, which conforms to
`Swift.Error`, and any other in a generated `<T>Error` holding it.
- A **path** is a `String`: one Rust returns that is not UTF-8 traps. A `SocketAddr`,
`SocketAddrV4`, or `SocketAddrV6` is the runtime's `SocketAddr`, its IP's 4 or 16 bytes
and its port, flow label, and scope, whose family Rust checks. A `Vec<T>` or `&[T]` is
an array, and passing a `Vec` by value closes each object in it once the call returns.
- **Traits** are `Sendable` protocols, which Rust releases once it drops an implementation.
- **Closures** lent for a call are `@Sendable` function types, which Rust calls only during
the call, from any thread. An object it lends is closed once the call returns.
- **Futures** are `async throws` functions, which resume on the task's executor, never
inside a wake. Cancelling the task frees the future, losing nothing, and throws
`CancellationError`. Each call is its own future, so it has one poller. The call polls it
once before suspending, and makes the signal it parks on only if that poll pends; the
signal latches a wake that lands before it parks.
- A **waker** is a `@Sendable () -> Void`, which Rust may call from any thread.
- **The contract** is checked once, at the first call into the library: a Swift file and a
library from different builds trap.

## IR

[`src/ir.rs`](src/ir.rs) is the API as each generator reads it, with no language decided:
Expand All @@ -280,10 +327,12 @@ its Rust path, its span, and its docs. A type is in the exact form Rust takes or
(`&str` apart from `String`, `Box<dyn Trait>` apart from `impl Trait`), so a generator can
borrow where Rust borrows. [`src/abi.rs`](src/abi.rs) decides the C ABI from it once,
names and signatures, so [`src/c.rs`](src/c.rs), [`src/cpp.rs`](src/cpp.rs),
[`src/kotlin.rs`](src/kotlin.rs), and [`src/glue.rs`](src/glue.rs) agree; the runtime the
glue includes is [`runtime/rt.rs`](runtime/rt.rs), the one the C++ header includes
[`runtime/rt.hpp`](runtime/rt.hpp), and the ones Kotlin and its JNI glue include
[`runtime/rt.kt`](runtime/rt.kt) and [`runtime/jni.rs`](runtime/jni.rs).
[`src/kotlin.rs`](src/kotlin.rs), [`src/swift.rs`](src/swift.rs), and
[`src/glue.rs`](src/glue.rs) agree; the runtime the glue includes is
[`runtime/rt.rs`](runtime/rt.rs), the one the C++ header includes
[`runtime/rt.hpp`](runtime/rt.hpp), the ones Kotlin and its JNI glue include
[`runtime/rt.kt`](runtime/rt.kt) and [`runtime/jni.rs`](runtime/jni.rs), and the one the
Swift includes [`runtime/rt.swift`](runtime/rt.swift).

## Census

Expand All @@ -296,12 +345,13 @@ refusal, which is a bug in the exporter's inference rather than in the crate.
## Tests

```bash
just ffi test # export the fixtures, build them, and drive api from C, C++, and Kotlin
just ffi moq # export moq's crates at the pinned revision, build them, and run moq/moq.c, moq/moq.cpp, and moq/moq.kt
just ffi test # export the fixtures, build them, and drive api from C, C++, Kotlin, and Swift
just ffi moq # export moq's crates at the pinned revision, build them, and run moq/moq.c, moq/moq.cpp, moq/moq.kt, and moq/moq.swift
just ffi moq-cpp # the same export, running only moq/moq.cpp
just ffi moq-swift # the same export, running only moq/moq.swift
just ffi census # census each into target/census
nix develop .#android --command just ffi android # the Kotlin test, shrunk by R8, on the Android device adb reaches
nix develop .#bench --command just ffi bench # frame writes and reads, through the export from C, C++, and Kotlin, and moq's rs/moq-c
nix develop .#bench --command just ffi bench # frame writes and reads, through the export from C, C++, Kotlin, and Swift, and moq's rs/moq-c
```

The `api` fixture covers each rule once, and names the `dep` fixture's types, which
Expand All @@ -316,7 +366,12 @@ each runtime check, which must throw or abort with its message. `just ffi androi
same on ART: the Kotlin and `tests/api.kt`, shrunk by R8 into an app's dex with only the
generated rules keeping what JNI looks up, against the library built by the NDK for the
device's ABI, which ANDROID_HOME and ANDROID_NDK_HOME locate. Nightly runs it on an
x86_64 emulator at API 24, the oldest moq's Android library supports, and at API 35. The `reject`
x86_64 emulator at API 24, the oldest moq's Android library supports, and at API 35.
`tests/api.swift` runs the script through the Swift, built as a module in Swift 6's
language mode and type-checked in Swift 5's, checks its futures the same way and that
releasing frees, and breaks each runtime check, which must trap or abort with its message.
It runs with nixpkgs' Swift on Linux, which builds none for aarch64-linux
([quest](/quest/m2/uniffi-swift-arm.md)), and nightly with Apple's on macOS. The `reject`
fixture's expected errors, each with its notes, are `fixtures/reject/expected.txt`.
moq's crates export through `target/moq-ffi`, an empty library on each that `just ffi moq`
writes in place of moq-ffi's remainder. `moq/moq.c` publishes and subscribes to a moq-net
Expand All @@ -327,7 +382,8 @@ serves a broadcast from a moq-tokio server bound to port 0 to a client that read
QUIC, both on moq-tokio's fallback runtime, whose threads wake C's. `moq/moq.cpp` does the
same through the C++ header, short of the catalog's field, at C++17 without exceptions or
RTTI, as OBS builds it, serving on an executor's thread and blocking on each read, and
`moq/moq.kt` through the Kotlin, serving in a coroutine and suspending on each read.
`moq/moq.kt` through the Kotlin, serving in a coroutine and suspending on each read, and
`moq/moq.swift` through the Swift, serving in a task and awaiting each read.

The C++ builds take `CXX`, or `c++`, with every warning an error and `-Wshadow`, which
stands in for MSVC's warnings that a name hides another. On Windows they take MSVC's `cl`
Expand Down
32 changes: 32 additions & 0 deletions ffi/fixtures/api/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,22 @@ pub enum Event {
Done,
}

/// An enum whose variant `Init` names an initializer in Swift, and the C union member
/// holding its data.
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum Stage {
Init(u32),
Done,
}

/// The stage after `stage`: `Init` counts down to `Done`.
pub fn advance(stage: Stage) -> Stage {
match stage {
Stage::Init(0) | Stage::Done => Stage::Done,
Stage::Init(n) => Stage::Init(n - 1),
}
}

/// The error a [`Counter`] returns.
#[derive(Clone, Debug, PartialEq)]
pub enum Error {
Expand Down Expand Up @@ -480,6 +496,11 @@ impl Channel {
}
}

/// `n` halved if even, at once: a future whose output may be `None`.
pub async fn halved(n: u32) -> Option<u32> {
n.is_multiple_of(2).then_some(n / 2)
}

/// `value`, on its third poll: each before wakes its waker before it returns pending, so
/// the wake lands before the poller waits for one, even past a first poll Kotlin makes
/// with no waker.
Expand Down Expand Up @@ -1305,6 +1326,17 @@ pub fn stray() -> std::path::PathBuf {
]))
}

/// Lends [`stray`]'s path to `visit`.
#[cfg(unix)]
pub fn visit_stray(visit: impl Fn(&std::path::Path)) {
visit(&stray());
}

/// `duration` plus `by`, unless past [`Duration::MAX`]: Rust's seconds are a `u64`.
pub fn extended(duration: Duration, by: Duration) -> Option<Duration> {
duration.checked_add(by)
}

/// Each path's parent, if it has one.
pub fn parents(paths: Vec<std::path::PathBuf>) -> Vec<Option<std::path::PathBuf>> {
paths
Expand Down
48 changes: 46 additions & 2 deletions ffi/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ run *args:
census checkout="": (standin checkout)
cargo run --locked --quiet -- --config moq/ffi.toml --manifest-path target/moq-ffi/Cargo.toml --census target/census

# Time frame writes through the export, from C, C++, and Kotlin, and through moq's hand-written rs/moq-c, in the bench shell: nix develop .#bench.
# Time frame writes through the export, from C, C++, Kotlin, and Swift, and through moq's hand-written rs/moq-c, in the bench shell: nix develop .#bench.
bench checkout="": (moq checkout)
#!/usr/bin/env bash
set -euo pipefail
Expand All @@ -62,14 +62,23 @@ bench checkout="": (moq checkout)
c++ -O2 -std=c++17 -Wall -Wextra -Werror -I "$out/include" moq/bench/export.cpp "$out/target/release/libmoq.a" -lpthread -ldl -lm -o target/bench-export-cpp
rm -rf target/bench-kt
kotlinc -Werror -cp "$KOTLINX_COROUTINES" "$out/kotlin/moq.kt" moq/bench/export.kt -d target/bench-kt
swift=true
if [ "{{ os() }}-{{ arch() }}" = linux-aarch64 ]; then
echo "skipping the Swift: nixpkgs builds no Swift for aarch64-linux (quest/m2/uniffi-swift-arm.md)" >&2
swift=false
else
{{ just_executable() }} swift-module -O
swiftc -O -warnings-as-errors $({{ just_executable() }} swift-target 15) -I target/swift -I "$out/swift" moq/bench/export.swift target/swift/libMoq.a "$out/target/release/libmoq.a" -lpthread -ldl -lm -o target/bench-export-swift
fi
for _ in 1 2 3; do
target/bench-moq-c
target/bench-export
target/bench-export-cpp
kotlin -Djava.library.path="$out/target/release" -cp "target/bench-kt:$KOTLINX_COROUTINES" ExportKt
if $swift; then target/bench-export-swift; fi
done

# Export moq's crates' C ABI, build it, and run moq/moq.c, moq/moq.cpp, and moq/moq.kt against it, from the pinned moq revision or a checkout.
# Export moq's crates' C ABI, build it, and run moq/moq.c, moq/moq.cpp, moq/moq.kt, and moq/moq.swift against it, from the pinned moq revision or a checkout.
moq checkout="": (moq-cpp checkout)
#!/usr/bin/env bash
set -euo pipefail
Expand All @@ -81,6 +90,11 @@ moq checkout="": (moq-cpp checkout)
rm -rf target/moq-kt
kotlinc -Werror -cp "$KOTLINX_COROUTINES" "$out/kotlin/moq.kt" moq/moq.kt -d target/moq-kt
kotlin -Djava.library.path="$out/target/release" -cp "target/moq-kt:$KOTLINX_COROUTINES" MoqKt
if [ "{{ os() }}-{{ arch() }}" = linux-aarch64 ]; then
echo "skipping moq/moq.swift: nixpkgs builds no Swift for aarch64-linux (quest/m2/uniffi-swift-arm.md)" >&2
else
{{ just_executable() }} moq-swift-run
fi

# Export and build moq's crates' C ABI as `just ffi moq` does, and run only moq/moq.cpp against it, built by $CXX or the platform's compiler: MSVC's on Windows, from a developer environment.
moq-cpp checkout="": (moq-export checkout)
Expand All @@ -98,6 +112,36 @@ moq-cpp checkout="": (moq-export checkout)
fi
target/moq-cpp-test

# Export and build moq's crates' C ABI as `just ffi moq` does, and run only moq/moq.swift against it, through Swift on PATH: Apple's on macOS.
moq-swift checkout="": (moq-export checkout) moq-swift-run

# Build the exported Swift as the module Moq, and run moq/moq.swift against it.
[private]
moq-swift-run:
#!/usr/bin/env bash
set -euo pipefail
out=target/export/moq
read -ra native < target/moq-native
{{ just_executable() }} swift-module
swiftc -warnings-as-errors $({{ just_executable() }} swift-target 15) -parse-as-library -I target/swift -I "$out/swift" moq/moq.swift target/swift/libMoq.a "$out/target/release/libmoq.a" "${native[@]}" -o target/moq-swift-test
target/moq-swift-test

# Build target/export/moq's Swift as the module Moq in target/swift, at Swift 6's language mode. Its deployment target is the oldest its `Swift.Duration` allows.
[private]
swift-module *flags:
#!/usr/bin/env bash
set -euo pipefail
out=target/export/moq/swift
rm -rf target/swift
mkdir -p target/swift
swiftc -warnings-as-errors $({{ just_executable() }} swift-target 13) -swift-version 6 {{ flags }} -parse-as-library -module-name Moq -emit-module -emit-module-path target/swift/Moq.swiftmodule -emit-library -static -I "$out" "$out/moq.swift" -o target/swift/libMoq.a

# swiftc's target at this macOS deployment target on macOS, and nothing elsewhere.
[private]
swift-target macos:
#!/usr/bin/env bash
if [ "{{ os() }}" = macos ]; then echo "-target $(uname -m)-apple-macos{{ macos }}"; fi

# Export moq's crates' C ABI into target/export/moq and build it, with the libraries a program linking it needs besides in target/moq-native.
[private]
moq-export checkout="": (standin checkout)
Expand Down
Loading