Skip to content

fix(flags): report evaluation failures instead of missing flags - #318

Open
marandaneto wants to merge 2 commits into
mainfrom
fix/spec211-failed-flags
Open

marandaneto wants to merge 2 commits into
mainfrom
fix/spec211-failed-flags

Conversation

@marandaneto

@marandaneto marandaneto commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

💡 Motivation and Context

A failed remote flag record is present, but it does not contain a usable flag value. Direct getters currently report flag_missing for that record. This makes evaluation failures look like missing flags.

Report evaluation_failed instead, including after inconclusive local evaluation. Keep response-wide errors_while_computing_flags diagnostics independent. Null getter results, event deduplication and suppression, healthy sibling flags, and existing value/payload filtering stay unchanged. No public API changes.

Implements the failed-direct-read telemetry requirement from PostHog/sdk-specs#211. This is a narrow fix, not a claim of complete SDK-spec conformance.

Includes the existing pnpm change core patch intent. Release tooling will propagate the Laravel patch and exact core dependency update. No package versions or Laravel code are changed. Latest main was merged normally with signed commits.

💚 How did you test it?

Post-merge validation on PHP 8.5.10:

  • Focused and neighboring PHPUnit suites passed: 579 tests, 2,322 assertions, covering failed telemetry, flag errors, direct getters, evaluation snapshots and email clients.
  • Scoped PHP_CodeSniffer, composer api:check, strict Composer validation and all 16 adapter fidelity/report tests passed.
  • The new regression's 12 pre-fix failures were verified in earlier implementation evidence. Final-commit Authoring test audit and isolated autoreview found no actionable issues.
  • The full 2,719-test suite has one known failure in EmailsClientTest::testAnOversizedReplyIsNotReadOrRetried: its second response reports network_error/service_unavailable instead of response_too_large/application_error. Independent matched baseline and patched runs reproduced this same failure before publication. The focused email suite passes. Transport debugging is outside this flag fix.
  • Full-tree PHP_CodeSniffer reports existing warnings outside the changed files, with no errors. No local coverage driver is available. Other PHP versions and the Laravel matrix were not run locally.
  • CI passed all 33 registered checks on d3290ff0dcb17131d2b6ff7bd6e0fb7113b8f661, including PHPUnit on PHP 8.2-8.5, coverage, public API, formatting, all seven Laravel combinations and all three compliance transport profiles with report-completeness checks. No CI retry or follow-up fix was needed.

📝 Checklist

  • All commits are signed (see Commit signing)
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed. (No documentation change needed.)
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm change to generate a change intent file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Pi/OpenAI agents prepared and reviewed this narrow fix using Git, PHP/Composer, pnpm and GitHub CLI. The isolated autoreview helper used its configured Claude Code provider with tools disabled. The driving maintainer is assigned as DRI. No shareable session URL is available.

The accepted patch was preserved before a normal signed merge of main. The existing release intent was retained without version bumps or Laravel code changes. The unrelated baseline email transport failure was disclosed rather than folded into this flag patch. Human review is required before merge.

@marandaneto marandaneto self-assigned this Nov 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

posthog-php-lib_curl Compliance Report

Date: 2026-11-12T10:56:33.801018+00:00
Duration: 116624ms

⚠️ Some Tests Failed

46/47 tests passed, 1 failed


Capture Tests

⚠️ 29/30 tests passed, 1 failed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 29ms
Format Validation.Event Has Uuid ✅ 520ms
Format Validation.Event Has Lib Properties ✅ 523ms
Format Validation.Distinct Id Is String ✅ 523ms
Format Validation.Token Is Present ✅ 523ms
Format Validation.Custom Properties Preserved ✅ 522ms
Format Validation.Event Has Timestamp ✅ 524ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 525ms
Retry Behavior.Retries On 503 ✅ 5835ms
Retry Behavior.Does Not Retry On 400 ✅ 2528ms
Retry Behavior.Does Not Retry On 401 ✅ 2526ms
Retry Behavior.Respects Retry After Header ❌ 6335ms
Retry Behavior.Implements Backoff ✅ 16248ms
Retry Behavior.Retries On 500 ✅ 5631ms
Retry Behavior.Retries On 502 ✅ 5632ms
Retry Behavior.Retries On 504 ✅ 5629ms
Retry Behavior.Max Retries Respected ✅ 17046ms
Deduplication.Generates Unique Uuids ✅ 237ms
Deduplication.Preserves Uuid On Retry ✅ 5628ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10841ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5633ms
Deduplication.No Duplicate Events In Batch ✅ 528ms
Deduplication.Different Events Have Different Uuids ✅ 524ms
Compression.Sends Gzip When Enabled ✅ 523ms
Batch Format.Uses Proper Batch Structure ✅ 523ms
Batch Format.Flush With No Events Sends Nothing ✅ 522ms
Batch Format.Multiple Events Batched Together ✅ 511ms
Error Handling.Does Not Retry On 403 ✅ 2528ms
Error Handling.Does Not Retry On 413 ✅ 2529ms
Error Handling.Retries On 408 ✅ 5638ms

Failures

retry_behavior.respects_retry_after_header

Retry delay too short: 804ms < 2500ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 524ms
Request Payload.Flags Request Uses V2 Query Param ✅ 526ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 525ms
Request Payload.Flags Request Omits Authorization Header ✅ 524ms
Request Payload.Token In Flags Body Matches Init ✅ 526ms
Request Payload.Groups Round Trip ✅ 524ms
Request Payload.Groups Default To Empty Object ✅ 527ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 522ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 525ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 525ms
Request Lifecycle.No Flags Request On Init Alone ✅ 520ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 509ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 528ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 522ms
Retry Behavior.Retries Flags On 502 ✅ 625ms
Retry Behavior.Retries Flags On 504 ✅ 622ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 522ms

@github-actions

Copy link
Copy Markdown
Contributor

posthog-php-fork_curl Compliance Report

Date: 2026-11-12T10:56:34.543117+00:00
Duration: 116650ms

⚠️ Some Tests Failed

46/47 tests passed, 1 failed


Capture Tests

⚠️ 29/30 tests passed, 1 failed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 41ms
Format Validation.Event Has Uuid ✅ 531ms
Format Validation.Event Has Lib Properties ✅ 534ms
Format Validation.Distinct Id Is String ✅ 535ms
Format Validation.Token Is Present ✅ 534ms
Format Validation.Custom Properties Preserved ✅ 533ms
Format Validation.Event Has Timestamp ✅ 536ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 534ms
Retry Behavior.Retries On 503 ✅ 5860ms
Retry Behavior.Does Not Retry On 400 ✅ 2537ms
Retry Behavior.Does Not Retry On 401 ✅ 2538ms
Retry Behavior.Respects Retry After Header ❌ 6351ms
Retry Behavior.Implements Backoff ✅ 16283ms
Retry Behavior.Retries On 500 ✅ 5653ms
Retry Behavior.Retries On 502 ✅ 5652ms
Retry Behavior.Retries On 504 ✅ 5651ms
Retry Behavior.Max Retries Respected ✅ 16286ms
Deduplication.Generates Unique Uuids ✅ 543ms
Deduplication.Preserves Uuid On Retry ✅ 5650ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10865ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5651ms
Deduplication.No Duplicate Events In Batch ✅ 544ms
Deduplication.Different Events Have Different Uuids ✅ 535ms
Compression.Sends Gzip When Enabled ✅ 535ms
Batch Format.Uses Proper Batch Structure ✅ 532ms
Batch Format.Flush With No Events Sends Nothing ✅ 521ms
Batch Format.Multiple Events Batched Together ✅ 523ms
Error Handling.Does Not Retry On 403 ✅ 2535ms
Error Handling.Does Not Retry On 413 ✅ 2535ms
Error Handling.Retries On 408 ✅ 5651ms

Failures

retry_behavior.respects_retry_after_header

Retry delay too short: 811ms < 2500ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 525ms
Request Payload.Flags Request Uses V2 Query Param ✅ 523ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 524ms
Request Payload.Flags Request Omits Authorization Header ✅ 524ms
Request Payload.Token In Flags Body Matches Init ✅ 524ms
Request Payload.Groups Round Trip ✅ 524ms
Request Payload.Groups Default To Empty Object ✅ 524ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 525ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 524ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 524ms
Request Lifecycle.No Flags Request On Init Alone ✅ 519ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 519ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 528ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 525ms
Retry Behavior.Retries Flags On 502 ✅ 628ms
Retry Behavior.Retries Flags On 504 ✅ 626ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 538ms

@github-actions

Copy link
Copy Markdown
Contributor

posthog-php-socket Compliance Report

Date: 2026-11-12T10:56:42.803641+00:00
Duration: 116008ms

⚠️ Some Tests Failed

46/47 tests passed, 1 failed


Capture Tests

⚠️ 29/30 tests passed, 1 failed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 26ms
Format Validation.Event Has Uuid ✅ 521ms
Format Validation.Event Has Lib Properties ✅ 523ms
Format Validation.Distinct Id Is String ✅ 523ms
Format Validation.Token Is Present ✅ 524ms
Format Validation.Custom Properties Preserved ✅ 523ms
Format Validation.Event Has Timestamp ✅ 538ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 523ms
Retry Behavior.Retries On 503 ✅ 5832ms
Retry Behavior.Does Not Retry On 400 ✅ 2525ms
Retry Behavior.Does Not Retry On 401 ✅ 2524ms
Retry Behavior.Respects Retry After Header ❌ 6332ms
Retry Behavior.Implements Backoff ✅ 16245ms
Retry Behavior.Retries On 500 ✅ 5632ms
Retry Behavior.Retries On 502 ✅ 5631ms
Retry Behavior.Retries On 504 ✅ 5631ms
Retry Behavior.Max Retries Respected ✅ 16245ms
Deduplication.Generates Unique Uuids ✅ 528ms
Deduplication.Preserves Uuid On Retry ✅ 5631ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10836ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5633ms
Deduplication.No Duplicate Events In Batch ✅ 528ms
Deduplication.Different Events Have Different Uuids ✅ 523ms
Compression.Sends Gzip When Enabled ✅ 526ms
Batch Format.Uses Proper Batch Structure ✅ 522ms
Batch Format.Flush With No Events Sends Nothing ✅ 519ms
Batch Format.Multiple Events Batched Together ✅ 512ms
Error Handling.Does Not Retry On 403 ✅ 2525ms
Error Handling.Does Not Retry On 413 ✅ 2524ms
Error Handling.Retries On 408 ✅ 5630ms

Failures

retry_behavior.respects_retry_after_header

Retry delay too short: 802ms < 2500ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 522ms
Request Payload.Flags Request Uses V2 Query Param ✅ 521ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 521ms
Request Payload.Flags Request Omits Authorization Header ✅ 520ms
Request Payload.Token In Flags Body Matches Init ✅ 522ms
Request Payload.Groups Round Trip ✅ 520ms
Request Payload.Groups Default To Empty Object ✅ 521ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 521ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 521ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 521ms
Request Lifecycle.No Flags Request On Init Alone ✅ 526ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 499ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 524ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 522ms
Retry Behavior.Retries Flags On 502 ✅ 624ms
Retry Behavior.Retries Flags On 504 ✅ 623ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 522ms

@marandaneto
marandaneto marked this pull request as ready for review November 12, 2026 12:13
@marandaneto
marandaneto requested a review from a team as a code owner November 12, 2026 12:13
@veria-ai

veria-ai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

PR overview

No security issues were found in this pull request.

Reviews

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants