Use the current SIMD target's width for the transposed KV cache - #1027

Closed

Magnushst wants to merge 305 commits into

google:mainfrom

Magnushst:fix/flash-attention-nf

This pull request is big! We're only showing the most recent 250 commits

Commits on Dec 16, 2025

Makes the entire runtime_config passed into the activations constructor.

Balazs Racz
authored and
copybara-github
committed
baa69df
Upgrade GitHub Actions for Node 24 compatibility
salmanmkc
committed
b66aa11
Merge branch 'dev' into upgrade-github-actions-node24
salmanmkc
authored
a4c78d4

Commits on Dec 17, 2025

Separate monolithic gemma_lib library into more specific cc_library targets.

Balazs Racz
authored and
copybara-github
committed
596bdfe
Use hn::Sub for vector subtraction in flash attention.

pculliton
authored and
copybara-github
committed
b8a409d

Commits on Dec 18, 2025

No public description

cushon
authored and
copybara-github
committed
142e6a7
Internal changes

Fireronin
authored and
copybara-github
committed
6661d3a
Avoid using Row() for unaligned storage.

Balazs Racz
authored and
copybara-github
committed
0ac55f7

Commits on Dec 19, 2025

Internal changes

Fireronin
authored and
copybara-github
committed
b73a9ed
Internal changes

Fireronin
authored and
copybara-github
committed
08a0760
Merge pull request #810 from salmanmkc:upgrade-github-actions-node24
copybara-github
committed
11aa16a
Add int8 quantization stats

jan-wassenberg
authored and
copybara-github
committed
1605925

Commits on Jan 7, 2026

Internal changes

Fireronin
authored and
copybara-github
committed
2ee1fac
Move AssertClose to test_util, add U16

jan-wassenberg
authored and
copybara-github
committed
aeade05

Commits on Jan 8, 2026

Allow overriding hardcoded max_seq_len by cmdline argument seq_len.

Balazs Racz
authored and
copybara-github
committed
384c390
Internal change / remove unused PrintSpeed

jan-wassenberg
authored and
copybara-github
committed
42e9cf5
Build fix for Arm SVE (explicit namespace qualification)

The gemma.cpp Authors
authored and
copybara-github
committed
95592a5

Commits on Jan 9, 2026

Build fix for Arm SVE (invalid template arg on op)

jan-wassenberg
authored and
copybara-github
committed
6d43d6e
Internal changes

Fireronin
authored and
copybara-github
committed
16a7ba2

Commits on Jan 29, 2026

Internal changes

Fireronin
authored and
copybara-github
committed
60eed01

Commits on Feb 5, 2026

Fix paligemma: must subtract image tokens from prompt length

jan-wassenberg
authored and
copybara-github
committed
2751a19
Automated Code Change

h-joo
authored and
copybara-github
committed
7c19b31

Commits on Feb 9, 2026

Internal change plus add U8 type, check MatPtrT type at compile time

jan-wassenberg
authored and
copybara-github
committed
56fa6e4
Internal changes

Fireronin
authored and
copybara-github
committed
7e5310b

Commits on Feb 13, 2026

Added wheat_from_chaff_test to test the ability of a model to find a needle in a haystack of data.

theraysmith
authored and
copybara-github
committed
76d7951

Commits on Feb 18, 2026

Internal changes

Fireronin
authored and
copybara-github
committed
c669634
Internal changes

The gemma.cpp Authors
authored and
copybara-github
committed
34739fd

Commits on Feb 19, 2026

Internal changes

Fireronin
authored and
copybara-github
committed
7dc9890

Commits on Feb 23, 2026

Internal change

jan-wassenberg
authored and
copybara-github
committed
463a368

Commits on Feb 24, 2026

Implementation of tiled attention with bf16 and circular buffers which reduces memory requirements by 4x on longer context on gemma models.

Fireronin
authored and
copybara-github
committed
df162ea

Commits on Feb 25, 2026

Improve instrumentation for ViT parts

jan-wassenberg
authored and
copybara-github
committed
c6587ef

Commits on Feb 27, 2026

Use a struct to manage the mapping between `AttentionImpl` enum values and their string names, simplifying `GetAttentionImplName` function. Add a test to ensure all valid `AttentionImpl` enums have…

Viktor Shipitsin
authored and
copybara-github
committed
d8a123e
remove const to fix windows builds

Fireronin
authored and
copybara-github
committed
bdba3bf
Add FastGelu activation function in a newly created created fast_ops-inl.h files.

Nikhil0250
authored and
copybara-github
committed
dd268dd

Commits on Mar 2, 2026

Add ability to load custom models which are fully described by the ModelConfig blob.

Miguel Lobo
authored and
copybara-github
committed
f7f5fd5
Rewrote flash attention to use BF16, transpose k and v, rewrote the task distribution, increase parallelism on decode, and use double the registers for the core of flash attention.

theraysmith
authored and
copybara-github
committed
16c1b29
No public description

The gemma.cpp Authors
authored and
copybara-github
committed
a3d9949
Fix Windows warnings

jan-wassenberg
authored and
copybara-github
committed
fbd44ce
Rollback of erroneous rollback.

theraysmith
authored and
copybara-github
committed
49cb438

Commits on Mar 3, 2026

Change to use faster exponent function

Fireronin
authored and
copybara-github
committed
539d9bb

Commits on Mar 4, 2026

Implement FastSigmoid.

Nikhil0250
authored and
copybara-github
committed
6721ddd

Commits on Mar 5, 2026

Fixed tsan error.

theraysmith
authored and
copybara-github
committed
79e640a
Use CappedTag to prevent potential out of bound reads.

Nikhil0250
authored and
copybara-github
committed
5081341
Fix VLM prefill batch size - prompt+tokens

jan-wassenberg
authored and
copybara-github
committed
8d9b992

Commits on Mar 6, 2026

internal change

merrymeerkat
authored and
copybara-github
committed
d6c7576
Fixed msan error by fixing padding of k_cache and v_cache

theraysmith
authored and
copybara-github
committed
d2806fb

Commits on Mar 9, 2026

Int8 + microscaling support for kv cache formats.

Fireronin
authored and
copybara-github
committed
029cfd0
Replaced attention in ViT with flash - 8x speedup of image tokenizer on AMD

theraysmith
authored and
copybara-github
committed
bea8b1c
Separate profiler output for image token generation

jan-wassenberg
authored and
copybara-github
committed
70cb9cf

Commits on Mar 11, 2026

Upgrade GitHub Actions to latest versions
salmanmkc
committed
3187ee0
Improved timing for image tokens

jan-wassenberg
authored and
copybara-github
committed
cab77f8

Commits on Mar 12, 2026

Merge pull request #866 from salmanmkc:upgrade-github-actions-node24-general
copybara-github
committed
e728d45
Add phase markers to stderr for high verbosity levels.

The gemma.cpp Authors
authored and
copybara-github
committed
d6e836c
Fix int8

Fireronin
authored and
copybara-github
committed
197c1a0
Add/use MaybePrint; also ShowConfig in non-interactive builds

jan-wassenberg
authored and
copybara-github
committed
529c201

Commits on Mar 13, 2026

Fix testing::SrcDir() path resolution in wheat_from_chaff_test

Nikhil0250
authored and
copybara-github
committed
0110ddf

Commits on Mar 18, 2026

Utilities to convert between different encodings of kv cache

Fireronin
authored and
copybara-github
committed
1a5226e

Commits on Mar 19, 2026

Add min_verbosity to MaybePrint

jan-wassenberg
authored and
copybara-github
committed
ceb7020
Change calculation from (ax+b)/(cx+d) to (x + b')/(c'x+ d') this replaces a MulAdd with Add reducing port contention on modern cpus and thus increasing throughput.

Nikhil0250
authored and
copybara-github
committed
5014473
Use paralell blend chain path in FastSigmoid on architectures having >=32 registers

Nikhil0250
authored and
copybara-github
committed
90f3de7
Disable SVE (except SVE2_128) for MatMul due to compiler crash

jan-wassenberg
authored and
copybara-github
committed
79f2bf7
Warning fix: cast enum for HWY_ABORT %d

jan-wassenberg
authored and
copybara-github
committed
1dedcfd

Commits on Mar 24, 2026

Updates to tests to use kv_transcodign library to reduce theris code size

Fireronin
authored and
copybara-github
committed
8a5e37e
Use Lookup8 and detail::IsFull(d) in FastSigmoid

Nikhil0250
authored and
copybara-github
committed
259b757
Improvements to inference using int8 compressed kv's

Fireronin
authored and
copybara-github
committed
f56d18d

Commits on Mar 25, 2026

Warning fix (size_t vs u64 in format string)

jan-wassenberg
authored and
copybara-github
committed
c0064bd
Internal changes

Fireronin
authored and
copybara-github
committed
2344488

Commits on Mar 27, 2026

Fix namespace references in api_client.cc
brendandahl
committed
20f2570
Support building with Emscripten
brendandahl
committed
b652581
Merge pull request #833 from brendandahl:emscripten-cmake
copybara-github
committed
0da94e5

Commits on Mar 30, 2026

Replace remaining occurrences of Exp with FastExpMinusOrZero in flash attention.

Nikhil0250
authored and
copybara-github
committed
8d2fcb3

Commits on Apr 2, 2026

Use HWY_MEMBER_VAR_MAYBE_UNUSED for members

jan-wassenberg
authored and
copybara-github
committed
3892763

Commits on Apr 7, 2026

Reformat

Nikhil0250
authored and
copybara-github
committed
f01cc59
Use FastExpMinusOrZero in Softmax().

Nikhil0250
authored and
copybara-github
committed
70513a1
Merge pull request #891 from texasich:clean-gcc15-fix
copybara-github
committed
366b143

Commits on Apr 8, 2026

Fixed bug in PackedBytes - not using override_rows.

theraysmith
authored and
copybara-github
committed
a29e2fc

Commits on Apr 13, 2026

Tested and benchmarked OneDNN BRGeMM integration against dev branch
bbhattar
committed
09ddbf4

Commits on Apr 14, 2026

Increased max_tbatch_size to kMaxBatchSize. Gives 1.5x speed-up for prefill on both intel and AMD machines

theraysmith
authored and
copybara-github
committed
221d8df

Commits on Apr 16, 2026

Commits on Apr 17, 2026

Merge pull request #896 from sharadboni:fix-tensor-ppm-overflow
copybara-github
committed
14a7447

Commits on Apr 21, 2026

Remove old attention, superseded by flash

jan-wassenberg
authored and
copybara-github
committed
d58a23d

Commits on Apr 23, 2026

Commits on Apr 27, 2026

Replace Tanh with FastTanh in flash_attention

Nikhil0250
authored and
copybara-github
committed
71fa50b
Tighten bounds in TestFastGelu to catch any regressions

Nikhil0250
authored and
copybara-github
committed
188be40

Commits on Apr 28, 2026

Fix rounding in kv transcoding

Fireronin
authored and
copybara-github
committed
1f21a29

Commits on May 5, 2026

Commits on May 6, 2026

Commits on May 7, 2026

Merge branch 'dev' into feature/onednn-brgemm
bbhattar
authored
0e8d31a

Commits on May 8, 2026

Automated Code Change

h-joo
authored and
copybara-github
committed
4dd473a

Commits on May 11, 2026

Merge branch 'dev' into feature/onednn-brgemm
bbhattar
authored
dd78e91
Internal change

stollem
authored and
copybara-github
committed
646d91c
Merge branch 'dev' into feature/onednn-brgemm
bbhattar
authored
3a2212e

Commits on May 12, 2026

Add enum to string function

Fireronin
authored and
copybara-github
committed
bdd4cde
Merge branch 'dev' into feature/onednn-brgemm
bbhattar
authored
93b38fb

Commits on May 15, 2026

Update kaggle links to Gemma v3. Refs #911

jan-wassenberg
authored and
copybara-github
committed
06f7f23

Commits on May 18, 2026

Remove transposing of Q's in tiled attention kernel

Fireronin
authored and
copybara-github
committed
860cd0b

Commits on May 19, 2026

Fix flash_attention_test link error on multi-target SIMD builds
plawanrath
committed
1984298

Commits on May 20, 2026

Merge branch 'dev' into feature/onednn-brgemm
bbhattar
authored
b2df643

Commits on May 26, 2026

Minor fix to include what is used in flash_attention.h.

birukw
authored and
copybara-github
committed
5c05eca

Commits on May 27, 2026

Commits on May 28, 2026

Merge upstream/dev: resolve CMakeLists.txt comment conflict
plawanrath
committed
e8605a8
Merge branch 'dev' into feature/onednn-brgemm
bbhattar
authored
22875ca

Commits on May 29, 2026

Merge pull request #918 from plawanrath:feat/gemma3-lm-only
copybara-github
committed
e58e56c
Remove UB when used with not aligned data

Fireronin
authored and
copybara-github
committed
ec17ae5
Miscellaneous cleanup.

birukw
authored and
copybara-github
committed
f3e57ec
[Security] Add missing blob-size assertion in ReadAllToBF16 weight loader
0xASTRA
committed
ab4fa85
Merge branch 'dev' into feature/onednn-brgemm
bbhattar
authored
c3787f8

Commits on Jun 2, 2026

Commits on Jun 3, 2026

Merge pull request #903 from Intel-tensorflow:feature/onednn-brgemm
copybara-github
committed
f7b2d1d
Merge #903: OneDNN BRGeMM Micro-Kernel Integration for BF16 MatMul

The gemma.cpp Authors
authored and
copybara-github
committed
3325f6f

Commits on Jun 4, 2026

Re-land BRGEMM

jan-wassenberg
authored and
copybara-github
committed
f8b430a

Commits on Jun 10, 2026

Helper functions to use arm matrix multiplication instructions in a way that is testable on other platofrms

Fireronin
authored and
copybara-github
committed
f5e78cc
Cleanup of unit tests

Fireronin
authored and
copybara-github
committed
0b0418b

Commits on Jun 11, 2026

Fix Windows build: shell was passing 3 instead of 3.5 to CMake, update version

jan-wassenberg
authored and
copybara-github
committed
970403a
[MatMul] Add SFC(Space Filling Curves), specifically the hilbert curve into matmul-inl.h.

Nikhil0250
authored and
copybara-github
committed
6e23806

Commits on Jun 12, 2026

Fix mul overflow and strncpy bounds check

jan-wassenberg
authored and
copybara-github
committed
b9492de

Commits on Jun 17, 2026

Clear unconsumed args after help message to avoid aborting message

The gemma.cpp Authors
authored and
copybara-github
committed
5dd1769

Commits on Jun 19, 2026

[MatMul] Increase reps while autotuning for SFC traversals in MatMul.

Nikhil0250
authored and
copybara-github
committed
d366809
[Gemma] Fix weights loader assertion crash in kReadBF16 mode.

Nikhil0250
authored and
copybara-github
committed
9ba4dfe

Commits on Jun 23, 2026

Commits on Jun 29, 2026

Arm bfmmla based flash attention kernel

Fireronin
authored and
copybara-github
committed
6a15707

Commits on Jul 2, 2026

Namespace-qualify all calls to fp_arith-inl.h so we can move it to hwy

jan-wassenberg
authored and
copybara-github
committed
1ff4078

Commits on Jul 3, 2026

Also namespace-prefix in dot_test.cc

jan-wassenberg
authored and
copybara-github
committed
ca62893
Move non-SIMD TwoSum into fp_arith.h for use by fp_arith-inl.h

jan-wassenberg
authored and
copybara-github
committed
6558e0b
Add attention kernels optimized for arm's i8mm instruction.

Fireronin
authored and
copybara-github
committed
02548ac

Commits on Jul 6, 2026

Support for Gemma 4 MoE 26B text model

pculliton
authored and
copybara-github
committed
3d4b97f

Commits on Jul 8, 2026

Automated Code Change

h-joo
authored and
copybara-github
committed
b44ada5

Commits on Jul 9, 2026

Add kv data type flag

Fireronin
authored and
copybara-github
committed
cf9f223

Commits on Jul 10, 2026

Better spreading of work for tiled attention

Fireronin
authored and
copybara-github
committed
e83725c

Commits on Jul 14, 2026

Re write of x86 kernel to better utilize memory

Fireronin
authored and
copybara-github
committed
2fdd75d

Commits on Jul 15, 2026

For flash_tiled_attention... fixed support for models with mixed qkv dim.

Fireronin
authored and
copybara-github
committed
6241d52
Adjust work spliting to subtasks, about 1.5x faster prefill on zen5 machines, and +- neutral on older ones. Importantly to fully utilize this it's important to use high prefill token_batch like 4k

Fireronin
authored and
copybara-github
committed
b94f446
Add CONTRIBUTING

jan-wassenberg
authored and
copybara-github
committed
23148b0

Commits on Jul 16, 2026

Gemma 4 E2B text model

pculliton
authored and
copybara-github
committed
e8c6416

Commits on Jul 17, 2026

Pass runtime_config_ to the KVCache constructor within GemmaEnv in benchmark_helper.cc. This ensures that kv_cache_type and attention_impl settings are properly propagated when using non-default at…

Fireronin
authored and
copybara-github
committed
7a8c621

Commits on Jul 20, 2026

Add DeepseekV4 Flash support: inference, converter, MTP speculative decoding. Tested via running the model to check whether output made sense.
iStrzalka
committed
2895479

Commits on Jul 21, 2026

Merge pull request #959 from iStrzalka:deepseek-v4-flash
copybara-github
committed
374e102

Commits on Jul 22, 2026

Native tokenizer

Zijian Yi
authored and
copybara-github
committed
4b26b03
Fix Python converter Bazel targets
maasir554
committed
de37fee
Add T5Gemma S/S config and tensor metadata
maasir554
committed
2f0da09
Add T5Gemma safetensors conversion
maasir554
committed
6f6abc3
Add T5Gemma encoder-decoder runtime
maasir554
committed
a606f6a
Document T5Gemma conversion and runtime
maasir554
committed
733d930
Sync config bindings with upstream Gemma4
maasir554
committed
d9cb293
Address T5Gemma review feedback
maasir554
committed
d5f8230
Address follow-up T5Gemma review comments
maasir554
committed
e73ed8d
Implement VNNI based flash attention that uses SumOfMulQuadAccumulate

Fireronin
authored and
copybara-github
committed
f1b8d5e

Commits on Jul 23, 2026

add build flag to enable profilier
RaviTriv
committed
52b1f3a
Merge branch 'dev' into t5gemma-support
maasir554
authored
e0ad749

Commits on Jul 24, 2026

Commits on Jul 27, 2026

Internal change.

pculliton
authored and
copybara-github
committed
b01c579
fix crash in non default attention implementation
RaviTriv
committed
5da6268
pass attention and kv cache flags to eval harness
RaviTriv
committed
33a6e86
document attention flag
RaviTriv
committed
df84e19

Commits on Jul 28, 2026

[gemma_cpp] Add support for Qwen3 models (0.6B, 1.7B and 4B).

Zijian Yi
authored and
copybara-github
committed
891e43d
Gemma4 can now use tiled attention implementation

Fireronin
authored and
copybara-github
committed
be9ced9
Merge branch 'dev' into fix-attention-flag
jan-wassenberg
authored
3ece6bf
Merge pull request #967 from RaviTriv:fix-attention-flag
copybara-github
committed
a5c3476
Helper for doing benchmarks, supports prefill , generation in both single query and multi query for generation

Fireronin
authored and
copybara-github
committed
c48a9ae

Commits on Jul 29, 2026

Commits on Jul 30, 2026

small batch and compressed weight matmul benchmarks
RaviTriv
committed
e317a62
add neon fast path for sfp weight decode
RaviTriv
committed
c3a1e90

Commits on Aug 3, 2026

moved onednn scratchpad to MatMulEnv.per_cluster
romirdes
committed
cb32c29
use IsF32 instead of IsSame
romirdes
committed
bfce5c2
moved onednn weights cache to MatMulEnv.per_cluster in case of concurrent execution
romirdes
committed
d476c4b

Commits on Aug 4, 2026

Optimize Gemma 4 MoE router input buffering and scaling:

Sdccoding
authored and
copybara-github
committed
c9493b8
[gemma_cpp] add tokenizer benchmark test

Zijian Yi
authored and
copybara-github
committed
ef5bd65

Commits on Aug 5, 2026

Merge branch 'dev' into neon-weight-decode
jan-wassenberg
authored
d41fa6a

Commits on Aug 7, 2026

enable fast path on sve_128
RaviTriv
committed
a333fb8
Serialization: EnumValid uses switch, not sentinel; add framing

pculliton
authored and
copybara-github
committed
4ebb1d5

Commits on Aug 9, 2026

enable fast path on all sve targets
RaviTriv
committed
3ad4ed5

Commits on Aug 10, 2026

Merge pull request #971 from RaviTriv:neon-weight-decode
copybara-github
committed
b325eae

Commits on Aug 11, 2026

Merge branch 'dev' into feature/onednn-matmul-threadpool
romirdes
authored
ce03b37

Commits on Aug 12, 2026

Fix buffer overflow in compression int_test.

The gemma.cpp Authors
authored and
copybara-github
committed
99d1848

Commits on Aug 13, 2026

Dspark support for converter and inference

Fireronin
authored and
copybara-github
committed
a52782f

Commits on Aug 14, 2026

Merge pull request #970 from Intel-tensorflow:feature/onednn-matmul-threadpool
copybara-github
committed
d2249b4
Gemma 4 E2B Vision

pculliton
authored and
copybara-github
committed
c63aeed
Internal change.

pculliton
authored and
copybara-github
committed
b956b6c

Commits on Aug 16, 2026

Report dependency download progress during CMake configure
Mikyx-1
committed
298f0e1
divide logits by temperature before exp
RaviTriv
committed
ff1572b
Internal Changes

The gemma.cpp Authors
authored and
copybara-github
committed
2de3466

Commits on Aug 19, 2026

Deduplicate packed matrix splitting

Fireronin
authored and
copybara-github
committed
cf8463c
Allow 0-valued string fields

jan-wassenberg
authored and
copybara-github
committed
20bc1fe

Commits on Aug 21, 2026

Commits on Aug 22, 2026

Store pre_att_rms_out as BF16
Mikyx-1
committed
6769a7a

Commits on Aug 25, 2026

Update build.yml
jan-wassenberg
authored
639f9df
Internal change.

pculliton
authored and
copybara-github
committed
d92b46d

Commits on Aug 26, 2026

Merge pull request #989 from google:jan-wassenberg-patch-3
copybara-github
committed
f920af6
- Define MxFp4Stream format with e8m0 block scale and e2m1 packed weights.

Fireronin
authored and
copybara-github
committed
ee051c9
Merge pull request #987 from Mikyx-1:bf16-pre-att-rms-out-dev
copybara-github
committed
cf3f670
Fix double google initialization, testonly targets

jan-wassenberg
authored and
copybara-github
committed
f9ab5df
Fix incorrect format string (TypeName vs type enum)

jan-wassenberg
authored and
copybara-github
committed
4adc22e

Commits on Aug 27, 2026

Compress the embedding table to SFP at load time

Mikyx-1
authored and
le_hoang_viet
committed
1e2fe4d
Use model config for dense RoPE theta
Mikyx-1
committed
854220b
Chunk packed tensors larger than 1GB across IO batches in MakeBatches.

Fireronin
authored and
copybara-github
committed
7824eb9
Add Gemma 4 E4B model configurations to gemma.cpp

ZacharyGarrett
authored and
copybara-github
committed
22f5f68

Commits on Aug 28, 2026

Use Highway scalar conversions in weights test
Mikyx-1
committed
4b49a8f
Vectorize the embedding max-absolute scan
Mikyx-1
committed
a075d4e
Exposes Type::kMXFP4 to Python configs and maps it to MxFp4Stream in SbsWriterImpl, allowing export of models quantized to Microscaling FP4.

Fireronin
authored and
copybara-github
committed
0a03a3a
Adds `export_gemma4_lm_sbs` to convert Gemma 4 LM safetensors checkpoints into Highway .sbs models, handling QKV projection fusing, per-layer embeddings (PLE) gating/projection/norm, and skip scali…

Fireronin
authored and
copybara-github
committed
4f13d35
Merge pull request #994 from Mikyx-1:fix/rope-theta-config
copybara-github
committed
a13c15e
update sentencepiece version to 0.2.2

Fireronin
authored and
copybara-github
committed
dd92195
Add ability to benchmark prefill in attention benchmark binary

Fireronin
authored and
copybara-github
committed
e7bc886
Use Highway namespace alias in weights
Mikyx-1
committed
f3407af
Internal Changes

The gemma.cpp Authors
authored and
copybara-github
committed
c69af34

Commits on Aug 31, 2026

Merge pull request #993 from Mikyx-1:sfp-embedding-dev
copybara-github
committed
156fbd0
Support for spin in attention benchmark and better handling of max length

Fireronin
authored and
copybara-github
committed
f0111c7

Commits on Sep 1, 2026

Avoid dangerous span usage - must not bind to temporary.

jan-wassenberg
authored and
copybara-github
committed
78a9496

Commits on Sep 2, 2026

Previously, any token selection with row > 0 wrote to

Fireronin
authored and
copybara-github
committed
6ad49a0

Commits on Sep 3, 2026

Introduce --mxfp4 and --lossless flags to convert_dsv4:

Fireronin
authored and
copybara-github
committed
3fdb7f2
Extend CompressAndTransposeQueriesMatrixAccumulationImpl and

Fireronin
authored and
copybara-github
committed
f153c3b
1. Parallelize grouped low-rank output projection (SumHeads) across NUMA

Fireronin
authored and
copybara-github
committed
30b178e
Internal Changes

The gemma.cpp Authors
authored and
copybara-github
committed
f0e467c

Commits on Sep 4, 2026

skip kv cache allocation for kv-shared layers
RaviTriv
committed
b68753b
Avoid repeated reallocation of the TopK buffer
Magnushst
committed
41b451e
Merge pull request #1011 from Magnushst:perf/topk-reserve
copybara-github
committed
dd04362
1. Add --profile flag to attention_benchmark to output Highway profiler

Fireronin
authored and
copybara-github
committed
61cc7d0
Merge pull request #1012 from RaviTriv:skip-shared-kv-alloc
copybara-github
committed
1658f88

Commits on Sep 8, 2026

Internal changes

Fireronin
authored and
copybara-github
committed
dc3914f
Internal changes

Fireronin
authored and
copybara-github
committed
90a7a84
In TileFlashAttentionSVBlockBF16, 256-bit InterleaveLower and InterleaveUpper

Fireronin
authored and
copybara-github
committed
2f19ffb
Quantize softmax probabilities S into [0, 255] uint8_t and utilize unsigned-by-signed (uint8 * int8) matrix multiplication for SV accumulation, improving quantization precision. Add SIMD emulation …

Fireronin
authored and
copybara-github
committed
824b1b8

Commits on Sep 10, 2026

Decompose Gelu and FastGelu into GeluCdf and FastGeluCdf, reformulating the

Fireronin
authored and
copybara-github
committed
5321954

Commits on Sep 11, 2026

Use the current SIMD target's width for the transposed KV cache
Magnushst
committed
866d403