Rubric AI feedback delivery modes

[adi-herwana-nus]

Motivation

Rubric AI grading could only produce draft comments: every generated comment waited for a staff member to accept it before the student saw it. For large cohort sizes, this becomes unmanageable for staff to individually click through, rate, and approve each comment.

This PR gives rubric grading similar controls to how Codaveri feedback is delivered
(none / draft / publish).

Rubric grading also needs a mode Codaveri feedback does not. Rubric answers are graded both when a student submits a
single answer and when they finalise the whole submission, so "publish" comes in two variants: publish as soon as the student submits an answer, and publish when the student finalises their submission (and all answers within it)

Change

1. The setting

Course Settings > Assessments > AI Rubric Grading > Feedback comment delivery is a radio group, like its
Codaveri counterpart:

Option	Value	Behaviour
Generate draft comments for staff approval	draft	Unchanged, and the default. Staff accept or reject each comment.
Publish comments when submission is finalised	publish_on_finalise	Comments are drafts while the student is attempting. Finalising publishes them, and comments generated after that are published directly.
Publish comments immediately	publish_on_answer_submit	The student sees each comment as soon as their answer is graded, and can reply while still attempting.
Generate no comments	none	Answers are still graded against the rubric, but no comment is written.

The setting lives in the assessments component settings hash (no migration), defaults to draft, and is
validated against AiGeneratedPostService::FEEDBACK_WORKFLOWS. It is an ordinary assessment setting that
course managers and owners can change. It is not behind the system-admin gate that guards the model
configuration, because it is a teaching decision rather than a cost or provider lever.

How it is implemented. AiGeneratedPostService is the single place both producers of AI comments go
through (RubricAutoGradingService and ApplyEvaluationsJob), so it decides everything:

• none returns before any post is created. Unlike Codaveri's none, this cannot skip the whole job, because
  the same LLM call also produces the grade.
• In the publish modes the post is created published and the topic is not marked pending, so these
  comments never enter the staff pending queues.
• Only drafts are updated in place when an answer is re-graded. In a publish mode, a re-graded answer gets a
  new comment instead of rewriting one the student may already have read.

publish_on_finalise needs two steps. The finalise callback runs before the grading jobs it triggers, so
at that point the comments for those answers do not exist yet:

1. When a comment is created, it is published if the submission is no longer being attempted.

2. When the submission is finalised, Submission#publish_ai_generated_feedback publishes the AI drafts already
   on it and clears their pending flags. Human drafts are left alone, and so are comments staff already
   accepted.

   The drafts are published one at a time with update!, because each post's callbacks save its final text into
   the feedback rating. The pending flags are cleared for all of them in a single update_all, which also bumps
   updated_at, since read tracking depends on it (acts_as_readable on: :updated_at). That saves two queries per
   draft. What remains, about 17 queries per draft, comes from the posts' own save callbacks.

Together these cover comments generated both before and after finalising, without a race between them.

Decisions made while building it:

• No notifications. Published comments arrive without a notification, matching Codaveri's publish mode.
• Applying from the rubric playground always creates drafts. That is a staff action that can affect a
  whole class at once, so ApplyEvaluationsJob passes force_draft: true.
• Ratings stay unset in the publish modes, as in Codaveri. The rating card only appears on drafts, so
  nobody is asked to rate an auto-published comment. The generated text is still saved as
  original_feedback, so the rating record exists if a way to rate published comments is added later.

2. Published comments appear without a page refresh

After grading finishes, the client reloads the answer and looks for the AI comment in the response. Two things
kept a published comment from arriving:

• Server: the reload response only included the comment for staff, and only while it was a draft. It now
  includes the latest AI comment the viewer is allowed to see: staff see any state, students only published
  comments. This follows the same rule the full page load uses. The logic is in a shared
  _ai_generated_comment.json.jbuilder partial.
• Client: AUTOGRADE_RUBRIC_SUCCESS was only dispatched when the response contained a category breakdown,
  which students don't receive until their submission is published. It now fires when either the breakdown or
  the comment is present, and both are optional. gradingResults keeps the existing breakdown when an action
  carries none.

Rubric-graded forum-post answers had the same problem, because their response never included the comment at
all. They now use the shared partial too.

3. Step-by-step assessments: a rubric question is passed once it has been submitted

In step-by-step assessments, the Continue and Finalise buttons, and the step a student can reach on page load
(maxStep), depend on whether each answer is correct. Rubric grading has no notion of a wrong answer, which
caused two problems:

• With "Allow submission with incorrect answers" on, students aren't shown grades, so the server never sent
  the result that the Continue button checks. With "show MCQ answer" also on, Continue was permanently
  disabled for rubric questions.
• A failed grading run, or a question with AI grading turned off, never marked the answer as correct, so the
  student could not move past that step.

The new rule is that a rubric-graded question is passed once the student has submitted it at least once,
however long grading takes and whether or not it succeeds.

• Server step (maxStep): QuestionsConcern#correctly_answered_question_ids also counts questions with
  grading_mode: rubric that have a submitted answer. This is a plain query on the question's grading mode.
• Buttons: rubric answers now always send an explanation, from a new _rubric_explanation.json.jbuilder
  partial. Its correct is true once the answer has been submitted and null before that. It carries no
  grade, so it no longer needs the check on whether the viewer may see grades. The client code is unchanged.

4. Removing the autogradable: false placeholder

The RBR question had been hardcoded to autogradable: false, as a placeholder until it was decided when AI
feedback should be auto-published. This PR settles that, so the question now reports its real
auto_gradable?. Two client components relied on the placeholder:

• The RBR answer box only rendered when !question.autogradable, a condition copied from TextResponse,
  where an auto-gradable question shows a different input instead. RBR has no other input, so with the real
  value the box would have disappeared. The condition is removed.
• The explanation panel's heading now depends on a new gradingMode field on the question, sent for every
  question type. Rubric questions always show "Answer submitted", and other types behave as before. This
  also fixes rubric-graded forum-post answers, which already reported their real autogradable and so were
  showing a green "Correct" banner.

5. Comment state and comment UI converted to TypeScript

The two reducers that store the AI comment, and the question comment thread that displays it, are converted to
TypeScript. The work is split into three commits so git history stays connected:

• c586eecedd only renames the five files (100% renames; they still contain the old JS at that commit).
• f43a775135 converts the posts and topics reducers.
• 73d2caa272 converts commentForms, Comments and CommentCard.

Reducers. posts, topics and commentForms are now createSlice reducers. They react to the bundle's
existing legacy action types by matching on the type, through a shared isOneOf helper in
assessment/utils/matchers.ts. gradingResults.ts now uses the same helper too, which removes its four inline
type checks and the Action / UnknownAction types that existed only to support them. Topic in types.ts
now declares the full payload the server sends (id, questionId, submissionQuestionId, postIds), where it
previously declared only postIds.

Each slice keeps the old state shape and reacts to exactly the same actions. The old and new action lists were
compared for posts and topics. For commentForms, the old and new reducers were run through the same 25
actions, covering every action type it handles, and the state matched after each one. The deliberate
differences are:

• A topic that isn't loaded, or a batch action with no posts or topics, is ignored where the old reducers would
  have thrown.
• commentForms now starts with isUpdatingComment and annotations defined, where they were previously
  missing until the first relevant action. Its submitting flags are always booleans. It no longer has
  annotationsDelayedComment, which nothing read or wrote.

Components.

• Comments is a function component using typed selectors instead of connect and PropTypes. It reads the post
  ids and the posts map separately and builds the list during render. Returning a newly built array from a
  selector would re-render on every store update, and react-redux warns about that in development.
• CommentCard is a function component typed against CommentPostMiniEntity. publishComment is now optional:
  Annotations.jsx never passes it, so the old card would have thrown if an annotation had been a draft.
• AiFeedbackCommentCard's hand-written post type is now a Pick of the shared type.

CommentCard also replaces the deprecated ConfirmationDialog with Prompt, using DeleteButton's
confirmMessage prop, which several other delete buttons in the app already use. The delete isn't awaited, so
the prompt still closes when you confirm and can't be confirmed twice while the request is running.
manually_graded_spec.rb now clicks Prompt's confirm button.

The DOM hooks the feature specs use are unchanged: #topic_<id>, #edit_post_<id>, #post_<id>, and the
edit-comment / delete-comment classes. Annotations.jsx and ReadOnlyEditor.jsx, which read the same
state and render CommentCard, needed no changes.

6. AI feedback ratings are no longer sent to students

The comment payload sent generatedRating for every AI comment, with no check on who was viewing it. That
object holds the AI's original text, the staff-edited text and the staff member's score, so when staff edited a
draft before publishing it, the student's browser received the AI's original wording and the rating. Forum
posts had the same problem with RagWise answers, where the rating also carries the answer's faithfulness and
relevance scores.

Both views (_post.json.jbuilder and the forum's _post_list_data.json.jbuilder) now send the rating only
when can?(:update, rating), the same check the rating endpoints use. That is granted to course staff only, so
exactly the people who can rate a comment receive its rating. No client changes were needed: every component
reads generatedRating only on drafts, which students never receive.

Locales

• The new setting's strings are in en, ko and zh.
• The Codaveri "draft" option was reworded to match ("…as a draft for staff approval").
• A separate locale bug is fixed: CodaveriSettings.codaveriSystemPromptDescription was shortened in
  translations.ts in 06e3f2778c (2025-09-28), but the locale files were never updated. The ko and zh
  versions still had the old paragraph with {br} placeholders, which the form never passes, so Korean and
  Chinese users were probably seeing a broken or raw message there.

Tests

• AiGeneratedPostService: one example per mode, including force_draft producing a draft under a
  publish mode, publish_on_finalise producing a draft while attempting and a published comment after, and
  original_feedback still being saved with rating left unset in the publish modes.
• Finalising a submission: AI drafts are published and their pending flags cleared, human drafts are left
  alone, and draft mode is unaffected. There is also a case for a comment staff accepted before
  finalising: if the student has since replied, the topic must still be pending after finalising. Without the
  workflow_state: 'draft' filter, the release would clear that flag and hide the student's question from
  staff.
• Reload response: a student gets the published comment and never a draft, even a newer one; staff get the
  latest comment in any state; with partial submission allowed, a student gets explanation.correct == true
  once they have submitted.
• next_unanswered: a drafted rubric answer does not pass the step; a submitted one does, even though
  grading never set correct.
• Course and settings controller: the workflow defaults to draft, rejects unsupported values, and saves
  for a course manager.
• Rating visibility: a student's reload of a rated, published comment contains no generatedRating and
  none of the AI's original wording, while staff get the rating. The same checks cover a rated RagWise answer
  in a forum topic, for a student and for a teaching assistant.
• Reducers: posts.test.ts and topics.test.ts (5 cases each: loading, create/update, the optional AI
  comment, no comment leaving state unchanged, and delete), and commentForms.test.ts (5 cases: boxes emptied
  on load while in-progress edits are kept, a new comment's lifecycle, a new annotation on a file with none yet,
  an edit saved then deleted, and grading emptying a file's annotation boxes).
• Feature spec: manually_graded_spec.rb covers the comment UI end to end. Its delete step was updated for
  Prompt. It was not run locally and will run on CI.

Each new spec was checked against the code it guards: with that code removed or reverted, the spec fails on
the intended assertion.

Deployment notes

No migration. Every existing course reads as draft, so nothing changes until a course opts into another
mode. Switching modes affects only comments created afterwards.

Known limitations

• A grading failure in the same session still blocks Continue until the page is refreshed. The client's
  grading-failure handler resets the question's result to null. After a refresh the student can move on.

Follow-ups (not in this PR)

• Course::Discussion::Post runs mark_self_as_read twice on every update. It's registered on both
  after_save and after_update. This affects every post save, and is most of the per-draft cost when
  finalising a submission publishes its drafts.
• CommentPostMiniEntity.createdAt is typed as Date, but the API sends a string. Comments wraps it in
  String(...) where it builds the AI card's React key. Fixing the shared type affects the discussion bundle too.
• topicShape in the submission bundle's propTypes.js declares posts, but the field is postIds. It is still used for annotation topics (with the correct field, so this bug is only cosmetic), so it is best fixed along with the eventual migration of the annotations code to TypeScript.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot review overview

🟡 Changes recommended

Finalisation can publish drafts intended for staff approval, and student responses introduce an avoidable N+1 lookup.

Review effort: Lite
Findings: 1

Open (1)

• Release filtering publishes unapproved student-triggered drafts · New