Conversation
The three video export tests were skipped as flaky or slow. Locally they pass every time, but their 2-3s timeouts have little headroom: a warm run took up to 6s, and runs under CPU contention took up to 20s. Raise the timeouts to 60s, which only bounds how long a real failure takes to report, and compare the exported duration with a tolerance rather than exact equality, since re-encoding can shift it slightly.
Contributor
|
| App Name | Jetpack | |
| Configuration | Release-Alpha | |
| Build Number | 34703 | |
| Version | PR #26086 | |
| Bundle ID | com.jetpack.alpha | |
| Commit | f5a9e85 | |
| Installation URL | 6gfa8gik6hui8 |
Contributor
|
| App Name | WordPress | |
| Configuration | Release-Alpha | |
| Build Number | 34703 | |
| Version | PR #26086 | |
| Bundle ID | org.wordpress.alpha | |
| Commit | f5a9e85 | |
| Installation URL | 7cj93t60r3gg8 |
Contributor
|
Maybe we should try using a smaller video, to see if the unit tests can run faster? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Re-enables the three skipped video export tests. They weren't broken — their 2–3s timeouts left too little headroom for an AVFoundation transcode on a busy or freshly reset simulator.
Summary
MediaURLExporterTests.testThatURLExportingVideoWorks/testThatURLExportingVideoWithoutGPSWorks— skipped in Sidebar: Enhance split view behavior #23627 as "too flaky in iOS 18"ItemProviderMediaExporterTests.testThatVideoIsExported— skipped in 2023 as "takes too long"validateVideoExportcompares the exported duration withaccuracy: 0.01instead of exact==— the export re-encodes (AVAssetExportPresetHighestQuality), so the output duration isn't guaranteed to match the source bit-for-bit.testThatVideoIsExportedalready uses the same tolerance.Root Cause
Neither skip commit records a failure, but the timings make the cause clear. Each test transcodes a 3.47s 640×360 fixture; on an idle machine that takes ~0.5s, but the tail is long:
reset_simulator: truedoes)Even the idle case exceeded the old 3s timeout. 60s gives ~3× headroom over the worst case we could produce.
Test Plan
Local stress on the CI configuration (Xcode 27.0, iOS 27.0 simulator), via
test-without-buildingon one build. Per-test counts are first-attempt executions (-test-iterationsruns fixed iterations, no retries):Simulator device failed to launch org.wordpress), before any test ran — a harness issue, not these tests.WordPressUnitTestsruns on an erased simulator — green, 6/6 video executions passedPinghubWebSocketTests.testReceiveManyMessage(13/20 runs),BlogJetpackTests.testSyncBlogsAndSignOut(7/20),ReaderTopicSwiftTest.testReaderSiteTopicUpdated(2/20),CommentDetailViewModelTests.actionsWaitForReplyCountAfterDetailRenders(1/20).