Repository navigation
refactor(config): replace grid workspace env vars with MSEConfig - #3436
Merged
Merged
Conversation
Remove GPTQMODEL_MSE_GRID_BUDGET_MB and GPTQMODEL_CLIP_GRID_BUDGET_MB. The per-chunk workspace budgets for the vectorized GPTQ/QQQ MSE shrink grid and AWQ clip grid now live on a new MSEConfig dataclass exposed as QuantizeConfig.mse_config (grid_budget_mb / clip_grid_budget_mb). None preserves the previous device-derived defaults (16 MiB CPU, 256 MiB GPU); 0 still forces single-candidate chunking. QQQ threads the budget through Quantizer.configure; the AWQ processor reads qcfg.mse_config directly. The knob is calibration-time tuning only and is not serialized into checkpoint meta.
Contributor
|
I'll fix CI failures and address comments from users with write access that start with 'Devin'.
|
…nfig
- `mse` on QuantizeConfig now takes an MSEConfig (norm exponent +
grid_budget/clip_grid_budget); a bare float still normalizes to
MSEConfig(norm=...), and dicts merge field-wise for dynamic overrides.
- Budgets are byte counts (ints) or 'K'/'M'/'G' postfixed strings
("256M", "1G"); None keeps the device default (16M CPU / 256M GPU),
0 forces single-candidate chunking.
- QQQ's Quantizer now accepts qcfg like the GPTQ one and reads the MSE
grid budget from it; the configure(grid_budget_mb) shim is dropped.
- Checkpoint meta still serializes the float norm as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the
GPTQMODEL_MSE_GRID_BUDGET_MB/GPTQMODEL_CLIP_GRID_BUDGET_MBenvironment variables from #3435 with a proper config object on
QuantizeConfig, folding the legacy floatmseexponent into it: the fieldis now
mse, taking anMSEConfig.What Changed
quantization/config.py:MSEConfignow carriesnorm(the MSE exponent)plus
grid_budget/clip_grid_budget. Budgets are byte counts (ints) orpostfixed strings (
"256M","1G", optionalB);Nonekeeps thedevice-derived default (16M CPU / 256M GPU) and
0forces single-candidatechunking (and one-token AWQ product tiles). The
msefield lives onBaseQuantizeConfig, soGPTQConfig,QQQConfig, andAWQConfigallaccept it.
mse=2.4→MSEConfig(norm=2.4); dicts mergefield-wise, and per-layer
dynamicoverrides replacenormwhile keepingconfigured budgets.
quantizer.py:find_params/_mse_shrink_gridread the exponent and gridbudget from
qcfg.mse.qqq.py:Quantizernow acceptsqcfgin its constructor like the GPTQQuantizerand reads the grid budget fromself.qcfg.mse; theconfigure(grid_budget_mb=...)shim is gone. InternalQuantizer()callsites pass
self.qcfg.awq_processor.py/qqq_processor.py/gptq_processor.py: read budgetsand
normfromqcfg.mse; dynamicmseoverrides go through_normalize_mse.models/writer.py+config.pymeta payload: checkpoint meta stillserializes the float exponent (
meta["mse"] = mse.norm) so saved configsround-trip identically. Budgets are calibration-scratch tuning knobs, not
part of quantization identity.
Tests
(
mse=MSEConfig/mse={...}dicts); the tiny-budget forced-chunkingpaths still hold.
Review Requirements
Kernel Accuracy Requirements
chunk-size targets into the vectorized grid evaluators from perf(quantizer): vectorize MSE shrink grid and AWQ clip grid evaluation #3435, whose
bitwise-equality vs the sequential oracle was established there.
Notes
normsemantics are unchanged:0disables the MSE search; AWQ ignores itand only uses
clip_grid_budget.test_serialization.pycases fail with HF-downloadOSErrorin this environment (no network access to checkpoints).
Link to Devin session: https://app.devin.ai/sessions/2eb9e33e49b54734a60b2552d71b0aca
Open in Devin Desktop: https://app.devin.ai/desktop/session/2eb9e33e49b54734a60b2552d71b0aca?variant=devin
Requested by: @Qubitium