Speed up MAUI app sync (#208) - #209
Merged
Merged
Conversation
- Pull only posts modified since the last sync; detect server deletions with a new lightweight GET api/posts/ids endpoint (falls back to the full list on first sync or on servers without the endpoint). - Save pulled posts in one SQLite transaction and remove deleted posts in one batch. - Fetch full details for unread posts with at most 5 concurrent requests. - Run sync on the thread pool and throttle progress events to ~250 ms. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kq68qA7Jv7puAF3Kc6GSFY
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kq68qA7Jv7puAF3Kc6GSFY
fboucher
added a commit
that referenced
this pull request
Sep 30, 2026
* fix(#173): use Id ?? RowKey for checking and opening offline post HTML (#179) * feat(#176): add sync progress bar and status updates for downloading offline post text (#180) * should have done that long ago * fix: resolve Android MAUI app crash when opening URLs (issue #177) (#181) * feat: add up/down reordering for categories and notes in summary editor (issue #92) (#183) Co-authored-by: Frank Boucher <2404846+fboucher@users.noreply.github.com> * feat: add text size slider and fix text wrapping on offline reader page (issue #178) (#182) * bumb version * feat: implement async background post content extraction to save HTML in blob storage (#185) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache (#186) * feat: implement async background post content extraction to save HTML in blob storage * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache * feat: configure protected Azure Function Parser API and key propagation (#187) * feat: configure protected Azure Function Parser API and key propagation * bump: version to 1.4.0 --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * update missing env variable * fix * chore: bump version to 1.4.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(#190): update actions/cache to v5 for Node.js 24 compatibility in OpenCode triage workflow (#191) * Update version to 1.4.2, add sync progress tracking, and reader page controls (#192) * chore: bump version to 1.4.2 * feat: show detailed sync progress messages and progress bar on Posts page * feat: add back button and text size slider at both top and bottom of reader page * ci: include version in android apk filename (#193) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * Fix document sync stuck and improve sync progress and offline fallback (fixes #196) (#199) * fix: constrain reader text size slider and add step buttons (fixes #195) (#198) * feat: Add markdown saving for reading notes and bump version to 1.4.3 * feat: add MAUI storage tab (#200) * feat: add MAUI storage tab for offline posts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: refine MAUI storage controls Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: highlight storage actions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: bump application version to 1.4.3 (#201) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(ci): fix timeout and rate limit failures in OpenCode Daily Triage workflow (#204) (#205) * v1.4.3 - Enhances MAUI sync, local storage, and reader (#202) * fix(#173): use Id ?? RowKey for checking and opening offline post HTML (#179) * feat(#176): add sync progress bar and status updates for downloading offline post text (#180) * should have done that long ago * fix: resolve Android MAUI app crash when opening URLs (issue #177) (#181) * feat: add up/down reordering for categories and notes in summary editor (issue #92) (#183) Co-authored-by: Frank Boucher <2404846+fboucher@users.noreply.github.com> * feat: add text size slider and fix text wrapping on offline reader page (issue #178) (#182) * bumb version * feat: implement async background post content extraction to save HTML in blob storage (#185) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache (#186) * feat: implement async background post content extraction to save HTML in blob storage * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache * feat: configure protected Azure Function Parser API and key propagation (#187) * feat: configure protected Azure Function Parser API and key propagation * bump: version to 1.4.0 --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * update missing env variable * fix * chore: bump version to 1.4.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(#190): update actions/cache to v5 for Node.js 24 compatibility in OpenCode triage workflow (#191) * Update version to 1.4.2, add sync progress tracking, and reader page controls (#192) * chore: bump version to 1.4.2 * feat: show detailed sync progress messages and progress bar on Posts page * feat: add back button and text size slider at both top and bottom of reader page * ci: include version in android apk filename (#193) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * Fix document sync stuck and improve sync progress and offline fallback (fixes #196) (#199) * fix: constrain reader text size slider and add step buttons (fixes #195) (#198) * feat: Add markdown saving for reading notes and bump version to 1.4.3 * feat: add MAUI storage tab (#200) * feat: add MAUI storage tab for offline posts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: refine MAUI storage controls Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: highlight storage actions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: bump application version to 1.4.3 (#201) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(ci): prevent timeout and rate limits in opencode daily triage (#204) --------- Co-authored-by: Frank Boucher <2404846+fboucher@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(ui): add dedicated quickadd popup and MinimalLayout for bookmarklet (#203) (#207) * v1.4.3 - Enhances MAUI sync, local storage, and reader (#202) * fix(#173): use Id ?? RowKey for checking and opening offline post HTML (#179) * feat(#176): add sync progress bar and status updates for downloading offline post text (#180) * should have done that long ago * fix: resolve Android MAUI app crash when opening URLs (issue #177) (#181) * feat: add up/down reordering for categories and notes in summary editor (issue #92) (#183) Co-authored-by: Frank Boucher <2404846+fboucher@users.noreply.github.com> * feat: add text size slider and fix text wrapping on offline reader page (issue #178) (#182) * bumb version * feat: implement async background post content extraction to save HTML in blob storage (#185) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache (#186) * feat: implement async background post content extraction to save HTML in blob storage * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache * feat: configure protected Azure Function Parser API and key propagation (#187) * feat: configure protected Azure Function Parser API and key propagation * bump: version to 1.4.0 --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * update missing env variable * fix * chore: bump version to 1.4.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(#190): update actions/cache to v5 for Node.js 24 compatibility in OpenCode triage workflow (#191) * Update version to 1.4.2, add sync progress tracking, and reader page controls (#192) * chore: bump version to 1.4.2 * feat: show detailed sync progress messages and progress bar on Posts page * feat: add back button and text size slider at both top and bottom of reader page * ci: include version in android apk filename (#193) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * Fix document sync stuck and improve sync progress and offline fallback (fixes #196) (#199) * fix: constrain reader text size slider and add step buttons (fixes #195) (#198) * feat: Add markdown saving for reading notes and bump version to 1.4.3 * feat: add MAUI storage tab (#200) * feat: add MAUI storage tab for offline posts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: refine MAUI storage controls Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: highlight storage actions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: bump application version to 1.4.3 (#201) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fixes CI timeout and rate limits in triage workflow (#206) * fix(#173): use Id ?? RowKey for checking and opening offline post HTML (#179) * feat(#176): add sync progress bar and status updates for downloading offline post text (#180) * should have done that long ago * fix: resolve Android MAUI app crash when opening URLs (issue #177) (#181) * feat: add up/down reordering for categories and notes in summary editor (issue #92) (#183) Co-authored-by: Frank Boucher <2404846+fboucher@users.noreply.github.com> * feat: add text size slider and fix text wrapping on offline reader page (issue #178) (#182) * bumb version * feat: implement async background post content extraction to save HTML in blob storage (#185) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache (#186) * feat: implement async background post content extraction to save HTML in blob storage * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache * feat: configure protected Azure Function Parser API and key propagation (#187) * feat: configure protected Azure Function Parser API and key propagation * bump: version to 1.4.0 --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * update missing env variable * fix * chore: bump version to 1.4.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(#190): update actions/cache to v5 for Node.js 24 compatibility in OpenCode triage workflow (#191) * Update version to 1.4.2, add sync progress tracking, and reader page controls (#192) * chore: bump version to 1.4.2 * feat: show detailed sync progress messages and progress bar on Posts page * feat: add back button and text size slider at both top and bottom of reader page * ci: include version in android apk filename (#193) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * Fix document sync stuck and improve sync progress and offline fallback (fixes #196) (#199) * fix: constrain reader text size slider and add step buttons (fixes #195) (#198) * feat: Add markdown saving for reading notes and bump version to 1.4.3 * feat: add MAUI storage tab (#200) * feat: add MAUI storage tab for offline posts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: refine MAUI storage controls Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: highlight storage actions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: bump application version to 1.4.3 (#201) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(ci): fix timeout and rate limit failures in OpenCode Daily Triage workflow (#204) (#205) * v1.4.3 - Enhances MAUI sync, local storage, and reader (#202) * fix(#173): use Id ?? RowKey for checking and opening offline post HTML (#179) * feat(#176): add sync progress bar and status updates for downloading offline post text (#180) * should have done that long ago * fix: resolve Android MAUI app crash when opening URLs (issue #177) (#181) * feat: add up/down reordering for categories and notes in summary editor (issue #92) (#183) Co-authored-by: Frank Boucher <2404846+fboucher@users.noreply.github.com> * feat: add text size slider and fix text wrapping on offline reader page (issue #178) (#182) * bumb version * feat: implement async background post content extraction to save HTML in blob storage (#185) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache (#186) * feat: implement async background post content extraction to save HTML in blob storage * feat: improve sync html download, always display read post button, and remove ILocalHtmlCache * feat: configure protected Azure Function Parser API and key propagation (#187) * feat: configure protected Azure Function Parser API and key propagation * bump: version to 1.4.0 --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * update missing env variable * fix * chore: bump version to 1.4.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(#190): update actions/cache to v5 for Node.js 24 compatibility in OpenCode triage workflow (#191) * Update version to 1.4.2, add sync progress tracking, and reader page controls (#192) * chore: bump version to 1.4.2 * feat: show detailed sync progress messages and progress bar on Posts page * feat: add back button and text size slider at both top and bottom of reader page * ci: include version in android apk filename (#193) Co-authored-by: fboucher-os <fboucheros+github@gmail.com> * Fix document sync stuck and improve sync progress and offline fallback (fixes #196) (#199) * fix: constrain reader text size slider and add step buttons (fixes #195) (#198) * feat: Add markdown saving for reading notes and bump version to 1.4.3 * feat: add MAUI storage tab (#200) * feat: add MAUI storage tab for offline posts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: refine MAUI storage controls Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: highlight storage actions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: bump application version to 1.4.3 (#201) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(ci): prevent timeout and rate limits in opencode daily triage (#204) --------- Co-authored-by: Frank Boucher <2404846+fboucher@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(ui): add browser bookmarklet support for saving URLs (#203) * feat(ui): add dedicated quickadd popup and MinimalLayout for bookmarklet (#203) --------- Co-authored-by: Frank Boucher <2404846+fboucher@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Speed up MAUI app sync (#208) (#209) * Speed up MAUI sync: delta pull, batched saves, off-UI-thread (#208) - Pull only posts modified since the last sync; detect server deletions with a new lightweight GET api/posts/ids endpoint (falls back to the full list on first sync or on servers without the endpoint). - Save pulled posts in one SQLite transaction and remove deleted posts in one batch. - Fetch full details for unread posts with at most 5 concurrent requests. - Run sync on the thread pool and throttle progress events to ~250 ms. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kq68qA7Jv7puAF3Kc6GSFY * Restore CRLF line endings in LocalDataService and PostNoteClient Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kq68qA7Jv7puAF3Kc6GSFY --------- Co-authored-by: Claude <noreply@anthropic.com> * chore: bump application version to 1.4.4 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: fboucher-os <fboucheros+github@gmail.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Frank · project thread
Fixes #208
Before: every time the Posts page opened, the MAUI app downloaded all ~1500 posts, saved them one by one, made an extra HTTP call per unread post, and did all of it on the UI thread while re-rendering the progress bar for every post. The app froze during sync.
After: after the first sync, only posts changed since the last sync are downloaded and saved, all in one SQLite transaction. Sync runs off the UI thread, and the progress bar updates about every 250 ms. Posts deleted on the server are still removed locally, and HTML for read posts is still pruned.
How
GET api/posts/idsreturns only post ids (read and unread). It projects just the key columns, so it stays small.PullAsynccallsGetPostsModifiedAfter(lastSync ?? MinValue). Deletions are detected by comparing local posts withGetPostIds(). On the first sync the full list is already in hand, so the ids call is skipped. If the server is older and has no ids endpoint (404), sync falls back to the full list, so the app still works against an API that hasn't been redeployed yet.SavePostsAsync, which runs in one transaction and preservesIsPendingSync. Server-deleted posts are removed with the newRemovePostsAsyncin one transaction. The end state is the same as the oldDeletePostAsync+MarkSyncedAsyncpair. Duplicate rows for one post (the API joins posts with notes) are de-duplicated.GetPostfor the full details, but at most 5 requests run at once (Parallel.ForEachAsync).SyncService.SyncAsyncstarts the work withTask.Run, which covers every caller (Posts page, Storage page, app resume, connectivity change). All progress and conflict handlers already marshal withInvokeAsync.SyncHtmlAsynclogic is unchanged apart from the progress throttle.Tests
SyncServiceTests: existing pull tests updated for batched save. New tests cover delta pull withlastSync, deletion via ids, fallback when the ids endpoint is missing, first sync skipping the ids call, a 1500-post single-batch save, de-duplication, progress throttling, and boundedGetPostconcurrency.LocalDataServiceTests:RemovePostsAsync, andSavePostsAsyncpreserving the pending-sync flag.DeltaApiTests:GET api/posts/idsreturns read and unread posts and excludes deleted ones.NoteBookmark.MauiApp.Testspasses (57/57). The API tests build but weren't run here because they need Docker for Azurite, so CI covers them. The MAUI app project itself wasn't built locally (no MAUI workload);SyncServiceis compiled through the test project.🤖 Generated with Claude Code
https://claude.ai/code/session_01Kq68qA7Jv7puAF3Kc6GSFY
Generated by Claude Code