Skip to content

fix(local-fs): Remove empty directories after deleting objects - #755

Merged
jan-auer merged 6 commits into
mainfrom
fix/local-fs-empty-dirs
Nov 5, 2026
Merged

jan-auer merged 6 commits into
mainfrom
fix/local-fs-empty-dirs

Conversation

@jan-auer

@jan-auer jan-auer commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

The filesystem backend stores each object key segment as one directory level. When an object was deleted, its directories stayed on disk. Self-hosted and local deployments therefore collected a growing number of empty directories, which can slow down the filesystem.

Deleting an object now also removes its directories while they are empty, up to the storage root. Cleanup stops at the first directory that still holds other objects. A failed cleanup never fails the delete.

This cleanup does not take any lock. Object locks are per object, but directories are shared by many objects, so locking every directory level would be expensive. Instead, this relies on the fact that directory removal is atomic and only succeeds on empty directories, so it can never remove stored objects. The only race is a writer that has just created a directory but has not yet placed its file in it. Writers now handle this: if the directory is gone when they create their draft or publish a resumable upload, they create it again and retry, up to three attempts. The delete releases the object lock before it starts the cleanup.

Fixes FS-158

Changelog Entry

  • The filesystem backend now removes empty object directories when an object is deleted or a write fails.

@jan-auer
jan-auer requested a review from a team as a code owner November 3, 2026 13:37
@linear-code

linear-code Bot commented Nov 3, 2026

Copy link
Copy Markdown

FS-158

@jan-auer
jan-auer added this pull request to stack #756 November 3, 2026 13:37
@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Changelog Preview

This PR lands in these workspaces:

  • objectstore-server: CalVer
📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


Bug Fixes

  • The filesystem backend now removes empty object directories when an object is deleted or a write fails. by jan-auer in #755

🤖 This preview updates automatically when you update the PR.

@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.83333% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.88%. Comparing base (7d411dd) to head (13a2be5).

Files with missing lines Patch % Lines
objectstore-service/src/backend/local_fs.rs 95.83% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #755      +/-   ##
==========================================
+ Coverage   91.86%   91.88%   +0.01%     
==========================================
  Files         119      119              
  Lines       22978    23059      +81     
==========================================
+ Hits        21109    21188      +79     
- Misses       1869     1871       +2     
Components Coverage Δ
Rust Backend 94.85% <95.83%> (+<0.01%) ⬆️
Rust Client 89.13% <ø> (ø)
Python Client 91.64% <ø> (ø)

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit fc6d205. Configure here.

Base automatically changed from worktree-fs-226-local-fs-path-traversal to main November 5, 2026 14:17
@jan-auer
jan-auer force-pushed the fix/local-fs-empty-dirs branch from 97bcaee to c19dfc4 Compare November 5, 2026 14:17
Dropping an unpublished draft removed its empty directories synchronously, usually on an async worker. Inside a Tokio runtime, the removal now runs on the blocking thread pool. Without a runtime, it still runs in place.
@jan-auer
jan-auer merged commit 664095f into main Nov 5, 2026
26 checks passed
@jan-auer
jan-auer deleted the fix/local-fs-empty-dirs branch November 5, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants