Repository navigation
Add SpeechRecognitionResult Timestamps #191
Description
Description
The Web Speech API currently does not expose the start and end timestamps of the source audio corresponding to a transcription result (SpeechRecognitionResult).
While SpeechRecognitionEvent.timeStamp provides a single timestamp for the event itself, it does not convey the start time, end time, or duration of the acoustic speech segment within the audio stream.
Without acoustic bounds for each result, web applications cannot associate transcribed text with media timelines:
- Automated Subtitling & Caption Tracks: Cannot construct frame-accurate subtitle cues (
VTTCue(start, end, text)) for recorded video or audio. - Interactive "Click-to-Seek" Transcripts: Cannot jump media playback (
mediaElement.currentTime = result.speechStartTime) to the exact moment a phrase was spoken. - Live Video Conferencing & Recording Sync: In applications like Google Meet or WebRTC calls, captions displayed live cannot be accurately aligned with the recorded video track for post-meeting review or synchronized with video playout delay buffers.
- Text-Based Media Editing: Tools that allow users to edit audio/video by cutting or rearranging transcript text require precise acoustic boundaries for each segment.
Specification
Expose speech segment bounds on the SpeechRecognitionResult interface in seconds as a double, measured on the timeline of the audio stream consumed by the recognizer (audiostart).
Web IDL
partial interface SpeechRecognitionResult {
// Start timestamp of the recognized speech segment in seconds relative to the audio stream (0.0s).
readonly attribute double speechStartTime;
// End timestamp of the recognized speech segment in seconds relative to the audio stream.
readonly attribute double speechEndTime;
};Usage Example: Subtitling & "Click-to-Seek" Navigation
const mediaElement = document.querySelector('video');
const track = mediaElement.addTextTrack('captions', 'English', 'en');
track.mode = 'showing';
const recognition = new SpeechRecognition();
recognition.continuous = true;
recognition.interimResults = false;
recognition.onresult = (event) => {
for (let i = event.resultIndex; i < event.results.length; ++i) {
const result = event.results[i];
if (!result.isFinal) continue;
const transcriptText = result[0].transcript;
// 1. Create frame-accurate WebVTT cue directly from seconds
const cue = new VTTCue(result.speechStartTime, result.speechEndTime, transcriptText);
track.addCue(cue);
// 2. Click-to-seek transcript element
const item = document.createElement('p');
item.textContent = `[${result.speechStartTime.toFixed(1)}s] ${transcriptText}`;
item.onclick = () => {
mediaElement.currentTime = result.speechStartTime;
mediaElement.play();
};
document.getElementById('transcript').appendChild(item);
}
};
recognition.start(mediaElement.captureStream().getAudioTracks()[0]);Security & Privacy Considerations
- Fingerprinting Mitigation: To mitigate micro-architectural timing attacks and hardware fingerprinting, user agents may reduce the resolution of
speechStartTimeandspeechEndTimeor introduce jitter, in accordance with user agent security and privacy policies (consistent with [[HR-TIME-3]] and [[HTML]]).
Alternatives Considered
- Existing
SpeechRecognitionEvent.timeStamp: Represents a single point in time (the event occurrence in document time). It does not provide the duration or start/end acoustic boundaries of the utterance within the stream, making subtitle cue generation (VTTCue) and click-to-seek impossible. - Separate Events (
speechstart/speechend): Fire only once at session voice-activity boundaries, cannot be correlated to individual stream results in continuous recognition, and do not provide segment start/end bounds per result. DOMHighResTimeStamp(Milliseconds): Semantically represents time relative to document origin rather than a media stream timeline, and causes friction with existing W3C media APIs (HTMLMediaElement.currentTime,BaseAudioContext.currentTime, andVTTCue) which universally operate in seconds as adouble.
Links & References
- Explainer: speech-recognition-result-timestamps.md (PR #206)
- Spec PR: PR #192
- Interactive Demos:
- ChromeStatus: Web Speech API: Add SpeechRecognitionResult Timestamps
Metadata
Metadata
Assignees
Labels
No labels
Activity
Update SpeechRecognitionResult with audio timing attributes
Add Explainer for SpeechRecognitionResult timestamps
Add Explainer for SpeechRecognitionResult timestamps (#193)
padenot commentedon Jul 15, 2026
https://webaudio.github.io/web-speech-api/#speechreco-result:~:text=For%20all%20these,endpointer%29%2E
In Firefox, this accounts for input latency (for direct microphone access), various buffering stages e.g. in Media playback, IPC boundaries, and the ASR model inherent latency. Output latency is available on the web platform to have super tight synchronization for e.g. auto-captionning.
This is workable, and we have built neat demos that show that alignment works well (e.g. putting words below the wave form). It does however require doing a little bit of arithmetic, but you get high flexibility in return.
@alan33d, @evanbliu, what does Chrome do?
michaelwasserman commentedon Jul 24, 2026
Can you point to the demos? I'm not too experienced here and more of a visual learner, so I vibe coded a demo (source) to try learning about the status quo.
I tried testing the same in FF with about:config media.webspeech.* flags all set true, but it wasn't working; lmk if you have tips.
I'm hoping we can continue to explore the viability of using the existing API surface, but new timestamps may prove useful if not necessary.
padenot commentedon Jul 27, 2026
Our implementation isn't merged.
alan33d commentedon Jul 27, 2026
Thanks for the suggestion. The current proposal addresses a developer request to detect speech recognition latency degradation mid-audiostream. From @michaelwasserman 's demo using the existing API surface, we noticed the existing event updates will only allow websites to accurately detect the latency of speech recognition at the beginning of the stream which is not enough granularity for the developer's use case. Thus we are proposing this change to get us the latency updates on an individual
SpeechRecognitionResultcall basis.michaelwasserman commentedon Jul 27, 2026
I published an improved demo and Chrome seems to fire
speechstart,speechend, etc. just once over the course of the recognition session. While the API might permit firing those events around each recognition result, clients would have a hard time correlating separate events and results. It seems like including the timing information in the results is a vast ergonomic improvement.padenot commentedon Jul 28, 2026
The timestamp information on those events is supposed to be in the timeline and time domain of the media, is that not the case in Chrome? See my link in #191 (comment)
Thanks for providing a demo, I'll run it on our implementation.
michaelwasserman commentedon Jul 29, 2026
The events do seem to be in the time domain of the media, but:
soundstartandspeechstartonly fire once per continuous recognition session, so they can't be used for continual latency checks of results emitted through the session.speechstart/end events per recognition session, and implementations were updated to support that, correlating those events with the respective recognition results is not entirely trivial for each API client interested in latency information.timeStampto match the correspondingspeechstartor most recent estimate of aspeechendevent, wouldn't enable clients to calculate latency information IIUC.Adding start and end timestamps to each result would seem to clearly and ergonomically give solve the client pain points around timeline association and continual latency detection. Please let us know what you find in your implementation.
alan33d commentedon Aug 21, 2026
Update on
SpeechRecognitionResultTimestampsFollowing recent discussions in the WebAudio WG meeting and review feedback from @padenot and @ReillyG, we have updated both the Specification and the Explainer with the following:
Unit of Measurement (
doublein Seconds):audioStartTimeandaudioEndTimefromDOMHighResTimeStamp(milliseconds) todoublerepresenting seconds.BaseAudioContext.currentTimeandHTMLMediaElement.currentTime), avoiding unit conversions when piping audio streams between Web Audio andVTTCue.Non-Nullable Attributes:
?) to keep the specification clean, without introducing artificial sentinel values (e.g.0or-1).Stream-Relative Timeline & Document Alignment:
performance.timeOrigin/event.timeStamp) to support live translation, caption alignment, and real-time processing latency tracking.Privacy & Precision:
HTMLMediaElement.currentTimeprecedent, allowing user agents to coarsen timer resolution or add jitter in accordance with their security and isolation policies without mandating a hardcoded quantization value in the normative text.Focus on Timeline Association: Streamlined the proposal to focus on the core capability that currently cannot be done on the web: associating transcripts with media timelines (generating
VTTCuesubtitle tracks, "click-to-seek" transcript navigation, recorded meeting alignment, and text-based media editing).Attribute Naming: Renamed
audioStartTime/audioEndTimetospeechStartTime/speechEndTimeto mirrorspeechstart/speechend.Spec & Explainer Links: Updated links to PR #192 (spec) and PR #206 (explainer), along with live interactive demos.
Update SpeechRecognitionResult with audio timing attributes
Update SpeechRecognitionResult with speech timing attributes (#192)