Skip to content

[SYSTEMDS-3986] Create class for Sparsity-based Optimization - #2630

Open
ywcb00 wants to merge 4 commits into
apache:mainfrom
ywcb00:feat/recompile/sparsity/rewrite/mmchain
Open

ywcb00 wants to merge 4 commits into
apache:mainfrom
ywcb00:feat/recompile/sparsity/rewrite/mmchain

Conversation

@ywcb00

@ywcb00 ywcb00 commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Hello there,
this PR adds a separate class for sparsity-based optimizations and for related pre-computations to estimate sparsity. It is appended to the dynamic rewrites.
For now, the sparsity optimizers replicates the sparsity-based matrix multiplication rewrites. In the future, this can be extended to complete re-creation of the computation graph.
All the best,
David

…te class sparsity optimizer to perform complete rewrites of isolated mm chain statement blocks
…zer to being a hop rewrite rule instead of statement block rewrite

	NOTE: hop rewrite rules are also executed after computing the previous statementblock

feat(main/hops/rewrite/SparsityOptimizer.java): replicate the optimization from normal sparsity-based mm chain rewrite
	create the method to "sculpt" a mm chain, which is allowed to completely rewrite the mm chain and ingest a new hop structure

feat(test/functions/rewrite/SparsityOptimizerTest.java): create preliminary junit test for the sparsity optimizer
@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.82353% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.03%. Comparing base (9fb64c3) to head (1d437c3).

Files with missing lines Patch % Lines
...g/apache/sysds/hops/rewrite/SparsityOptimizer.java 82.81% 3 Missing and 8 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2630      +/-   ##
============================================
+ Coverage     71.49%   72.03%   +0.54%     
- Complexity    51157    51184      +27     
============================================
  Files          1646     2101     +455     
  Lines        198715   228744   +30029     
  Branches      38640    38657      +17     
============================================
+ Hits         142072   164787   +22715     
- Misses        45475    52777    +7302     
- Partials      11168    11180      +12     
Flag Coverage Δ
java 71.50% <83.82%> (+<0.01%) ⬆️
python 75.61% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…ilar to the mm chain rewrite class

chore(test/functions/rewrite/SparsityOptimizerTest.java): set the estimator type explicitly

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant