Repository navigation
test(ietf): verify pinned Moxygen conformance under a relay profile #6497
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
fb4cbe3
chore(quest): claim quest/m1/ietf-extension/README.md
kixelated d96126c
Pin Moxygen conformance and document relay contract conflicts
kixelated 7047e72
Merge remote-tracking branch 'origin/main' into HEAD
kixelated 9cfeb32
test(ietf): verify Moxygen conformance under the relay profile
kixelated 2808a9a
test(ietf): gate the nightly on cells that can pass
kixelated File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| name: IETF conformance | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| workflow_call: | ||
|
|
||
| jobs: | ||
| moxygen: | ||
| runs-on: ubuntu-24.04 | ||
| environment: ${{ github.ref == 'refs/heads/main' && 'cache-write' || '' }} | ||
| timeout-minutes: 60 | ||
| steps: | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| - uses: ./.github/actions/nix | ||
| - uses: ./.github/actions/rust-cache | ||
| with: | ||
| access-key-id: ${{ secrets.MBX_R2_ACCESS_KEY_ID }} | ||
| secret-access-key: ${{ secrets.MBX_R2_SECRET_ACCESS_KEY }} | ||
| - name: Pinned Moxygen through the relay under the conformance profile | ||
| run: nix develop --command just test ietf-conformance | ||
| env: | ||
| MOQ_TEST_RUNS: ${{ runner.temp }}/ietf-conformance | ||
| MOQ_TEST_KEEP: 1 | ||
| - name: Keep conformance reports and endpoint logs | ||
| if: ${{ !cancelled() }} | ||
| uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7 | ||
| with: | ||
| name: ietf-conformance | ||
| path: ${{ runner.temp }}/ietf-conformance/ | ||
| retention-days: 7 | ||
| if-no-files-found: ignore | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| --- a/moxygen/moqtest/MoQTestClient.cpp | ||
| +++ b/moxygen/moqtest/MoQTestClient.cpp | ||
| @@ -554,6 +554,10 @@ | ||
| XLOG(DBG1) << "MoQTest DEBUGGING: calling onObjectStatus"; | ||
|
|
||
| ObjectHeader header = objHeader; | ||
| + if (header.status == ObjectStatus::END_OF_TRACK) { | ||
| + validateEndOfTrack(state, header); | ||
| + return; | ||
| + } | ||
| // Validate the received data | ||
| if (header.status != ObjectStatus::END_OF_GROUP) { | ||
| XLOG(ERR) | ||
| @@ -594,6 +598,35 @@ | ||
| } | ||
| } | ||
|
|
||
| +void MoQTestClient::validateEndOfTrack( | ||
| + const ReceiveState& state, | ||
| + const ObjectHeader& header) { | ||
| + // A subscription that runs to the track's end may learn where it ended from | ||
| + // an END_OF_TRACK placed directly after the last object, either inline or | ||
| + // opening the next group. It carries no payload and ticks off nothing. | ||
| + const AbsoluteLocation location{header.group, header.id}; | ||
| + const AbsoluteLocation inlineEnd{window_.last.group, window_.last.object + 1}; | ||
| + const AbsoluteLocation nextGroup{window_.last.group + 1, 0}; | ||
| + if (state.type != ReceivingType::SUBSCRIBE || !window_.endOfTrack || | ||
| + endOfTrackReceived_ || | ||
| + (location != inlineEnd && location != nextGroup)) { | ||
| + recordSemanticsFailure( | ||
| + folly::to<std::string>( | ||
| + "Unexpected End of Track at group=", | ||
| + header.group, | ||
| + " id=", | ||
| + header.id, | ||
| + " Expected group=", | ||
| + nextGroup.group, | ||
| + " id=0 or group=", | ||
| + inlineEnd.group, | ||
| + " id=", | ||
| + inlineEnd.object)); | ||
| + return; | ||
| + } | ||
| + endOfTrackReceived_ = true; | ||
| +} | ||
| + | ||
| void MoQTestClient::onEndOfStream() { | ||
| XLOG(DBG1) << "MoQTest DEBUGGING: calling onEndOfStream"; | ||
| } | ||
| @@ -793,6 +826,16 @@ | ||
| publishDoneReceived_ = true; | ||
| } | ||
|
|
||
| +namespace { | ||
| +// The origin alternates priority by group, but advertises the even-group value | ||
| +// as the track's publisher priority. A relay that ranks egress by track | ||
| +// priority forwards that value for every group; anything else still fails. | ||
| +bool priorityMatches(uint64_t groupID, uint8_t priority) { | ||
| + return priority == publisherPriorityForGroup(groupID) || | ||
| + priority == kMoQTestPublisherPriority; | ||
| +} | ||
| +} // namespace | ||
| + | ||
| void MoQTestClient::validateSubgroupHeader( | ||
| const ReceiveState& state, | ||
| uint64_t groupID, | ||
| @@ -811,8 +854,11 @@ | ||
| return; | ||
| } | ||
|
|
||
| + // A relay may end the track on a stream of its own, past the last group. | ||
| + // No payload group exists there to end; onObjectStatus checks its contents. | ||
| + const bool terminal = groupID > window_.last.group; | ||
| auto expectEndOfGroup = subgroupCarriesLastObject(params_, subgroupID); | ||
| - if (options.containsLastInGroup != expectEndOfGroup) { | ||
| + if (!terminal && options.containsLastInGroup != expectEndOfGroup) { | ||
| recordSemanticsFailure( | ||
| folly::to<std::string>( | ||
| "End of Group Signal Mismatch for group=", | ||
| @@ -839,7 +885,7 @@ | ||
| // The publisher may elide the priority from the wire, but the value the | ||
| // subscriber ends up with must still be the one the publisher chose. | ||
| auto expectedPriority = publisherPriorityForGroup(groupID); | ||
| - if (priority != expectedPriority) { | ||
| + if (!priorityMatches(groupID, priority)) { | ||
| recordSemanticsFailure( | ||
| folly::to<std::string>( | ||
| "Subgroup Priority Mismatch for group=", | ||
| @@ -876,7 +922,7 @@ | ||
| } | ||
|
|
||
| auto expectedPriority = publisherPriorityForGroup(header.group); | ||
| - if (header.priority != expectedPriority) { | ||
| + if (!header.priority || !priorityMatches(header.group, *header.priority)) { | ||
| recordSemanticsFailure( | ||
| folly::to<std::string>( | ||
| "Datagram Priority Mismatch for group=", | ||
| --- a/moxygen/moqtest/MoQTestClient.h | ||
| +++ b/moxygen/moqtest/MoQTestClient.h | ||
| @@ -254,6 +254,7 @@ | ||
| ReceiveState& state, | ||
| const std::optional<TrackAlias>& trackAlias, | ||
| const ObjectHeader& objHeader); | ||
| + void validateEndOfTrack(const ReceiveState& state, const ObjectHeader& header); | ||
| void onEndOfStream(); | ||
| void onError(ReceiveState& state, ResetStreamErrorCode); | ||
| void onPublishDone(); | ||
| @@ -318,6 +319,8 @@ | ||
| // Last group the joining FETCH half covers; unset when there is no join. | ||
| std::optional<uint64_t> fetchHalfLastGroup_; | ||
| bool publishDoneReceived_{false}; | ||
| + // A track ends at most once. | ||
| + bool endOfTrackReceived_{false}; | ||
| // A datagram that arrives after its deadline is as good as dropped. | ||
| uint64_t datagramDrops_{0}; | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| { | ||
| moxygen = { | ||
| owner = "facebookexperimental"; | ||
| repo = "moxygen"; | ||
| rev = "c27ae6358df62a1195da479b2e07c50dd8174279"; | ||
| hash = "sha256-ZbTb3kOyFYL0MfALDgoQC50NX+OR6geC4iUXc/phSik="; | ||
| }; | ||
| folly = { | ||
| owner = "facebook"; | ||
| repo = "folly"; | ||
| rev = "e89841cb61bf8f5283cda05b1bd35de99557a557"; | ||
| hash = "sha256-PMOouxfCqzzkdzIC3lO0iwSi82YjwQ5empXOUSV1j8E="; | ||
| }; | ||
| mvfst = { | ||
| owner = "facebook"; | ||
| repo = "mvfst"; | ||
| rev = "00fb3329dc43c6095668083d92a626031aa1eead"; | ||
| hash = "sha256-tTDiMMGZ82THJnrZc5V4xgScm+RaQiAgYUXbGLtY3qs="; | ||
| }; | ||
| proxygen = { | ||
| owner = "facebook"; | ||
| repo = "proxygen"; | ||
| rev = "05d1c685b865a2c90f6ab13aef6f01ccd53c7931"; | ||
| hash = "sha256-ogSop9Vc1Arfa9FOyhxo1F8n2QQ+ytBmlB7SK7Am0xE="; | ||
| }; | ||
| wangle = { | ||
| owner = "facebook"; | ||
| repo = "wangle"; | ||
| rev = "a39ad0756413158253824d08fa3d212dcfac4e42"; | ||
| hash = "sha256-OpmCdhEokiH0kTzsDGchHMtwHsvE/2ZjZ61ph3SeF0Y="; | ||
| }; | ||
| fizz = { | ||
| owner = "facebookincubator"; | ||
| repo = "fizz"; | ||
| rev = "327e167726bdde385a4fbb368ea148aeb2641acb"; | ||
| hash = "sha256-tQC/2apy2BS5+Ckb7U3DRwwsMs3Zbv5sUvW/g+wzywo="; | ||
| }; | ||
| fast_float = { | ||
| owner = "fastfloat"; | ||
| repo = "fast_float"; | ||
| rev = "v8.0.0"; | ||
| hash = "sha256-shP+me3iqTRrsPGYrvcbnJNRZouQbW62T24xfkEgGSE="; | ||
| }; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| { | ||
| lib, | ||
| stdenv, | ||
| fetchFromGitHub, | ||
| cmake, | ||
| ninja, | ||
| pkg-config, | ||
| gperf, | ||
| python3, | ||
| perl, | ||
| openssl, | ||
| glog, | ||
| gflags, | ||
| fmt, | ||
| double-conversion, | ||
| boost, | ||
| libevent, | ||
| libsodium, | ||
| zstd, | ||
| zlib, | ||
| c-ares, | ||
| libunwind, | ||
| }: | ||
| let | ||
| pins = import ./moxygen-sources.nix; | ||
| sources = lib.mapAttrs (_: pin: fetchFromGitHub pin) pins; | ||
| in | ||
| stdenv.mkDerivation { | ||
| pname = "moxygen-conformance"; | ||
| version = "unstable-2026-10-23"; | ||
| src = sources.moxygen; | ||
| cmakeDir = "../standalone"; | ||
| # The relay conformance profile: the client accepts track-level priority and | ||
| # an END_OF_TRACK directly after the last object. Every other check stays. | ||
| patches = [ ./moxygen-profile.patch ]; | ||
| nativeBuildInputs = [ | ||
| cmake | ||
| ninja | ||
| pkg-config | ||
| gperf | ||
| python3 | ||
| perl | ||
| ]; | ||
| buildInputs = [ | ||
| openssl | ||
| glog | ||
| gflags | ||
| fmt | ||
| double-conversion | ||
| boost | ||
| libevent | ||
| libsodium | ||
| zstd | ||
| zlib | ||
| c-ares | ||
| libunwind | ||
| ]; | ||
| # The standalone client uses blockingWait, which needs Folly's concrete executor. | ||
| postPatch = | ||
| lib.concatStringsSep "\n" ( | ||
| lib.mapAttrsToList (name: source: '' | ||
| cp -R ${source} .vendor-${name} | ||
| chmod -R u+w .vendor-${name} | ||
| patchShebangs .vendor-${name} | ||
| '') (builtins.removeAttrs sources [ "moxygen" ]) | ||
| ) | ||
| + '' | ||
| substituteInPlace moxygen/moqtest/CMakeLists.txt \ | ||
| --replace-fail 'moqtest_client PUBLIC' \ | ||
| 'moqtest_client PUBLIC Folly::folly_executors_manual_executor' | ||
| ''; | ||
| preConfigure = lib.concatStringsSep "\n" ( | ||
| lib.mapAttrsToList (name: _: '' | ||
| cmakeFlagsArray+=("-DFETCHCONTENT_SOURCE_DIR_${lib.toUpper name}=$PWD/.vendor-${name}") | ||
| '') (builtins.removeAttrs sources [ "moxygen" ]) | ||
| ); | ||
| cmakeFlags = [ | ||
| "-DBUILD_TESTS=OFF" | ||
| "-DBUILD_MOQTEST=ON" | ||
| "-DBUILD_SAMPLES=OFF" | ||
| "-DFETCHCONTENT_FULLY_DISCONNECTED=ON" | ||
| ]; | ||
| ninjaFlags = [ | ||
| "moqtest_client" | ||
| "moqtest_server" | ||
| ]; | ||
| installPhase = '' | ||
| runHook preInstall | ||
| mkdir -p $out/bin $out/libexec/moxygen/moqtest $out/share/moxygen | ||
| for executable in moqtest_client moqtest_server; do | ||
| install -m755 moxygen/moqtest/$executable $out/libexec/moxygen/moqtest/$executable | ||
| ln -s $out/libexec/moxygen/moqtest/$executable $out/bin/$executable | ||
| done | ||
| install -m755 $src/moxygen/moqtest/conformance_test.sh $out/share/moxygen/conformance_test.sh | ||
| cp ${./moxygen-sources.nix} $out/share/moxygen/sources.nix | ||
| runHook postInstall | ||
| ''; | ||
| meta = { | ||
| description = "Pinned Moxygen client and origin for relay conformance"; | ||
| homepage = "https://github.com/facebookexperimental/moxygen"; | ||
| license = lib.licenses.asl20; | ||
| platforms = lib.platforms.linux; | ||
| }; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.