Skip to content

Leaderboard submission: decting-v0.13 (Qwen3-0.6B + LoRA, targeted sampling) - #218

Merged
liamdugan merged 2 commits into
liamdugan:mainfrom
woldy:main
Oct 13, 2026
Merged

liamdugan merged 2 commits into
liamdugan:mainfrom
woldy:main

Conversation

@woldy

@woldy woldy commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

Detector: decting-v0.13

Method: Qwen3-0.6B + LoRA (r=16) sequence classification, bf16. Successor to decting-v0.10 (currently on the leaderboard) — from char-TF-IDF to a fine-tuned 0.6B model.

Training: RAID train split targeted sampling (25,968 rows — human with wiki/reddit domain boost, base-model floors for gpt2/mpt/mistral/cohere/gpt3, sampling-decoding priority, 30% synonym-attack augmentation; title-grouped, no test leakage) + 2,945 rows of 2026-generation-model text (Kimi K2.5 / GLM-5.3 / DeepSeek-V4.1 / GPT-6-Sol outputs from public corpora).

Validation beyond RAID:

  • Stress-tested on unseen 2025 flagships (arena data, 12 families x 300): AUROC 0.998
  • Blind test on unseen 2026 flagships (5 generators incl. GPT-6-Sol): AUROC 0.9999, 100% detection at 0.5
  • v0.10 TF-IDF baseline on the same sets: 0.49-0.53 (random) — the upgrade is qualitative

Files:

  • leaderboard/submissions/decting-v0.13/predictions.json — full test set (672,000 rows)
  • leaderboard/submissions/decting-v0.13/metadata.json

Thanks for maintaining RAID!

@woldy
woldy deployed to raid-main October 10, 2026 02:28 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

decting-v0.13

Release date: 2026-10-08

I've committed detailed results of this detector's performance on the test set to this PR.

Warning

Failed to find threshold values that achieve False Positive Rate(s): (['1%']) on all domains. This submission will not appear in the main leaderboard for those FPR values; it will only be visible within the splits in which the target FPR was achieved.
On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an AUROC of 92.11 and a TPR of 96.89% at FPR=5%.
Without adversarial attacks, it achieved AUROC of 93.06 and a TPR of 98.80% at FPR=5%.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

@liamdugan

Copy link
Copy Markdown
Owner

Hi @woldy please let me know if you'd like me to merge this to the leaderboard

@woldy

woldy commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

Yes, please merge. Thanks! We'll follow up with an attack-augmented v0.14 once it's validated.

@liamdugan
liamdugan merged commit e66f14e into liamdugan:main Oct 13, 2026
2 checks passed

This branch was successfully deployed

1 active deployment
raid-main — 06819a98 Deployed Oct 12, 2026 by liamdugan via evaluate #388
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants