Repository navigation
Conversation
layers/fused_ew.h, with CPU references and SYCL kernels: - fused_rms: RMSNorm into F32 / BF16 rows (fused_rms_q8 wrote only q8 blobs) - fused_swiglu: the GLU node with SiLU, into BF16 rows or a q8 blob - FUSED_GELU_ERF: an exact-GELU epilogue for fused_linear and qlinear (oneDNN eltwise_gelu_erf); the old bool argument still means tanh - fused_lnorm without affine (null weight and bias) - fused_rope_kv_table: RoPE from per-token cos/sin tables (rope_2d), with a BF16 output so a batch of views can be read in place by the attention modules/fused_lm.h: a Qwen2 / Llama decoder layer as 8 fused nodes. modules/encoder: EncCfg::gelu_erf, and fold_layer_scale for folding LayerScale into resident BF16 weights. tests/test_fused_sycl_ops checks every new node against the CPU reference. pi0.5's actions are bit-identical to before (default and W8A8).
sandlbn
force-pushed
the
sycl-fused-nodes
branch
from
October 25, 2026 00:53
16c4317 to
f7dac23
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SYCL: fused nodes for the SmolLM / Qwen2 / InternViT / DiT layers
Shared groundwork for the per-model SYCL PRs that follow (SmolVLA, Evo-1, VLA-Adapter, GR00T). Nothing here changes an existing model's graph: π0.5's actions are bit-identical to main, default and with
VLA_PI05_W8A8=all.New in
layers/fused_ew.h, each with a CPU reference and a SYCL kernel insrc/sycl/vla_sycl_fused.cpp:fused_rms: RMSNorm into F32 or BF16 rows.fused_rms_q8only wrote INT8 blobs.fused_swiglu: the GLU node with SiLU (the Llama / Qwen MLP), into BF16 rows or an INT8 blob. It isfused_gegluwith a flag.fused_linear(..., FUSED_GELU_ERF)and the same forqlinear, using oneDNNeltwise_gelu_erf. The oldbool geluargument still means tanh.fused_lnormwithout an affine (null weight and bias).fused_rope_kv_table: RoPE from per-token cos / sin tables (therope_2dform), with an optional BF16 output, so a batch of views can be read in place by the attention.Shared modules:
modules/fused_lm.h: a Qwen2 / Llama decoder layer as 8 fused nodes. The steps are RMSNorm, q|k|v (+ bias), RoPE into F16 K/V buffers, attention, o + residual, RMSNorm, gate|up, SwiGLU, and down + residual. q|k|v and gate|up are concatenated at load, and the residual stream stays F32.EncCfg::gelu_erffor the fused SigLIP-style blocks.fold_layer_scale(modules/encoder.h): folds LayerScale into the resident BF16 weights, so the residual add can be a GEMM epilogue.Why
Adds the five fused nodes the models were missing. A layer goes from ~20 kernels to 7-9, it's key because at 50 rows each kernel launch costs more than reading the weights.
Verified
-Wall -Wextra(first-party code)ctestpassesvla_predict_checkdiff), or the change ismeant to move it and a LIBERO sweep is below
vla_predict_checkπ0.5 (pi05-libero.gguf, 2 views): bit-identical to main, default and W8A8.Archs and backends tested:
x86 Linux / PTL