Automated performance optimisation & GPU kernel generation

Squeezing speed out of hot code (GPU kernels, numerical routines, inference paths) is slow expert work that depends on the exact hardware. An evolutionary or agentic LLM loop proposes many code variants, compiles and benchmarks each on the real target, rejects anything that fails a correctness check, and feeds the best results back as context for the next round. The outcome is measured runtime and energy savings, with a human reviewing the final change.

Problem & value

• Today: a small group of performance engineers hand-tunes kernels (tiling, memory coalescing, fusion, tensor-core use, vectorisation) per hardware generation. Every new GPU, framework version or model architecture can invalidate earlier tuning. Most teams simply accept default library performance and pay for the extra compute.
• AI value: LLMs know common optimisation patterns and can write CUDA, Triton or SIMD code; automated benchmarking turns "plausible" code into measured improvements. Search over many variants covers more of the design space than one engineer can try.
• Evidence (vendor-reported, not independent): Google DeepMind reports that AlphaEvolve found a scheduling heuristic for its data-centre orchestrator that continuously recovers about 0.7% of Google's worldwide compute, sped up a kernel in the Gemini architecture by 23% (about 1% shorter training time), and gained up to 32.5% on a FlashAttention kernel [1][2]. Treat these as existence proofs from a frontier lab with in-house hardware, not as expected results for a typical company. Benchmarks such as KernelBench show that general LLMs often fail to beat PyTorch baselines without an iterative, feedback-driven loop [3].

Variants

• GPU kernel generation: CUDA, Triton, CUTLASS/CuTe, HIP/ROCm, Pallas (TPU) kernels for attention, GEMM, normalisation, quantised ops, MoE routing.
• Inference hot-path tuning: operator fusion, batching/scheduling heuristics and cache policies in serving stacks (vLLM, TensorRT-LLM) - complements model compression.
• HPC and numerical code: CPU vectorisation, MPI/OpenMP tuning, sparse-matrix and stencil kernels in simulation and research codes.
• Algorithm and heuristic discovery: evolving scoring functions, schedulers, compiler heuristics or mathematical constructions (AlphaEvolve-style) where a programmatic evaluator exists [2].
• Application-level profiling assistant: LLM reads flame graphs and profiler traces and proposes hotspot fixes in ordinary backend code (lower autonomy, mainstream value).
• Private use: hobbyist and student tuning of local-inference kernels; low value, same tools.

How it works

1. baseline code + reference tests + hardware spec
2. LLM proposes N variants (mutations / rewrites, informed by profiler feedback and prior winners)
3. sandboxed compile (nvcc / Triton / clang)
4. reject build failures
5. correctness gate (compare to reference outputs, tolerance per dtype, multiple random seeds, edge shapes)
6. benchmark on target hardware (warm-up, repeated runs, clock locking, variance)
7. profile (Nsight Compute / rocprof)
8. feed metrics back to the LLM
9. selection (evolutionary database: keep best + diverse variants)
10. next generation
11. human review of winner
12. regression suite
13. merge

• Proposer: a code-capable reasoning LLM, often an ensemble of a fast model for breadth and a stronger one for depth (the AlphaEvolve pattern) [2]. Reinforcement learning on speedup rewards is an emerging alternative to pure prompting.
• Evaluator: the decisive part. It is a deterministic harness that returns correctness and timing; it must be hard to game. Public reports show that weak harnesses can be exploited (e.g. reusing memory from earlier runs or skipping work), which inflated early claimed speedups for the Sakana "AI CUDA Engineer" until fixed [4].
• Human-in-the-loop: engineers define the objective, tolerance and shapes up front, review the winning diff, and run the full test and numerical-stability suite before release.

Data preparation

The inputs are the code to optimise (kernels, hot paths, solver heuristics), a correctness reference, and a reliable benchmark on the target hardware. In an enterprise these live in Git repositories, performance-test labs and CI runners; in smaller teams often on one developer's workstation.

Typical gaps: no reproducible benchmark, noisy timings from shared machines, missing reference implementation or unit tests for numerical correctness, no profiling data, and unclear performance targets.

Preparation steps: 1. Profile first (Nsight, perf, py-spy) and pick one hot spot with measurable share of runtime. 2. Build a deterministic benchmark with fixed inputs and warm-up, run on isolated hardware. 3. Write a correctness oracle: the reference implementation plus tolerance-based tests on varied inputs. 4. Containerise the build environment so every candidate is compiled identically. 5. Sandbox execution of generated code.

Tools: pytest, Google Benchmark, Nsight Compute, Docker. Share of effort: low to moderate, around 15 to 25 percent, but the oracle quality decides whether results can be trusted. GUIDE-003 and PREP-018 (building test sets with experts) help.

Implementation paths

Which path for whom

• enterprise (B): Optimisation loops need your own benchmarks and target hardware, so a custom harness calling cloud model APIs gives the best results without training anything.
• developer (B): Frontier model APIs generate the best kernel candidates today; the evaluation harness runs locally on your own GPU.

Path A - Buy (off-the-shelf)

Few turnkey products exist. Options are (1) vendor-provided tuned libraries and compilers that embed autotuning (NVIDIA cuDNN/CUTLASS, TensorRT-LLM, PyTorch torch.compile with Inductor, OpenAI Triton autotuner), (2) cloud services such as Google Cloud's AlphaEvolve early-access offering for selected customers, and (3) AI-assisted profilers inside general coding agents (CODE-001/002). EU-based: few dedicated vendors; check with your GPU/HPC consultancies. Choose A when the bottleneck is in a standard operator that vendors already optimise.

Path B - Build on cloud APIs (recommended starting point)

• Models: frontier reasoning/code models (Claude, Gemini, GPT) or Mistral's code models (EU provider) via API in EU regions where available; code and benchmarks rarely contain personal data.
• Loop: open-source frameworks OpenEvolve (AlphaEvolve-inspired) or ShinkaEvolve provide the evolutionary database, prompt sampling and evaluator hooks [5]; combine with a generic agent framework for profiler-driven iterations.
• Hardware: the benchmark GPUs are the cost centre - rent target hardware (EU cloud or on-prem) and run the evaluator in isolated containers; LLM tokens are usually minor.
• Choose B when: you have one or two hot kernels or services and in-house GPUs for the evaluator.

Path C - Self-host (open source)

• Models: open-weight code/reasoning models such as Qwen3-Coder, DeepSeek-V3/R1-class, gpt-oss or Mistral Devstral/Codestral, served with vLLM; kernel-specialised fine-tunes (e.g. from the KernelBench community) are emerging.
• Stack: OpenEvolve, Triton, CUDA toolkit, KernelBench-style harnesses, Nsight/rocprof.
• Hardware: HW3 for the proposer (30-120B model) plus the target GPUs for the evaluator; small teams can start with one workstation GPU (HW2) and a ~30B model.
• Choose C when: source code is highly confidential or the search runs for weeks.

Path D - Custom training

RL fine-tuning of a code model on a speedup reward, with the real compiler and hardware in the reward loop, is the research frontier [3][4]. It needs a fleet of benchmark GPUs and a robust, exploit-resistant verifier; rarely justified outside AI labs, chip vendors and large inference operators.

State of the art (2026)

• Evolutionary LLM search (AlphaEvolve, OpenEvolve and successors) is the dominant pattern for discovering faster code and heuristics, provided a fast automatic evaluator exists [2][5].
• KernelBench (Stanford) is the reference benchmark: 250 PyTorch workloads at three levels, scored with a "fast_p" metric (correct and faster than baseline by factor p). Early results showed frontier models producing correct, faster kernels in only a minority of tasks; iterative feedback from compiler errors and profilers improves results substantially [3].
• Triton lowers the barrier compared with raw CUDA, so many systems target it first [6].
• Limits and failure modes: reward hacking of the benchmark; kernels fast only for tested shapes or one GPU model; subtle numerical drift (fp16/bf16/fp8 accumulation); race conditions that pass tests intermittently; gains that vanish against a well-tuned vendor library; maintenance burden of unreadable generated code; non-portable results across GPU generations.

Shared resources & combinations

• Same stack: llm-code + agent-framework as CODE-002; test and sandbox infrastructure from CODE-004; hardware-verification loops similar to CODE-019 (RTL, testbenches).
• Upstream/downstream: CODE-021 (large-scale refactoring) for rolling out accepted changes; CODE-009 for CI/CD benchmark runners; CODE-018 for tracing the optimisation agent.
• Related but distinct: OPT-015 (FinOps) optimises resource allocation and cloud spend, not code.
• Shared hardware: the benchmark GPU pool can double as inference/training capacity during idle time.

Effort, cost & KPIs

• Path A: days; gains limited to what vendors ship.
• Path B: 2-4 months to build a trustworthy evaluator and CI integration; cost drivers are benchmark GPU hours and engineer time for harness design, not tokens.
• Path C: 3-6 months; adds model serving operations.
• Path D: 6+ months with a specialised team.
• KPIs: speedup vs. strongest baseline (not naive baseline), p50/p99 latency, throughput per GPU, energy per inference or per job, accepted-variant rate, correctness-regression escapes, GPU-hours consumed per accepted improvement, engineer review time.

People & change

Involve: management only needs to approve budget and GPU time; the department lead of engineering or HPC picks the target and success threshold. End users are the performance engineers who validate every winning variant. IT supplies hardware and sandboxed runners, IT security reviews the sandbox because the loop executes generated code (GUIDE-013).

Works council: no. The system optimises code and measures programs, not people, so §87(1) no. 6 BetrVG is not triggered. Keep it that way: do not attach per-engineer productivity metrics. Still inform the council under §90(1) no. 3 BetrVG if the work process of the performance team changes noticeably. A DPO or DPIA is not needed because no personal data is processed.

AI literacy: support engineers' skills (Art. 4 AI Act) with a workshop on benchmark validity, reward hacking (a variant that games the benchmark) and numerical-correctness checks (GUIDE-006).

Pilot and communication: start with a single kernel or hot path with a clear speed-up goal and a time box of four to six weeks; set a go/no-go threshold in advance (GUIDE-009). Typical concern: "generated code is unmaintainable or subtly wrong." Answer: every accepted variant passes the oracle, goes through normal code review and is documented, so humans stay responsible.

Risks, regulation & compliance

• EU AI Act: minimal risk; the system optimises software and processes no persons. Art. 4 AI literacy applies. If the optimised code runs inside a high-risk AI system or safety component, the change must go through that system's conformity and change management [7].
• GDPR: normally not applicable (code and benchmarks); keep production data out of test inputs, and use synthetic or anonymised inputs.
• Safety and quality: for medical, automotive or industrial software (see CODE-020), generated optimisations need qualified tool and verification processes; numerical changes can alter validated results.
• IP and licensing: check licences of generated code that may resemble open-source kernels; proprietary code sent to external APIs needs contractual protection.
• Security: run untrusted generated code only in sandboxes (no network, resource limits); LLM-written native code can corrupt memory.
• Sustainability: search itself consumes energy; account for it against the savings.

Getting started

1. Profile first: pick one hotspot that accounts for a large share of runtime or GPU spend.
2. Write the evaluator: reference outputs, tolerances per dtype, representative shapes, locked clocks, repeated timing; include adversarial checks against reward hacking.
3. Compare against strong baselines (cuBLAS/cuDNN, torch.compile, existing Triton kernels).
4. Run OpenEvolve or a simple agent loop with a budget cap; log every variant and its metrics.
5. Review the winner by hand, run the full regression and numerical suite, then roll out with a rollback plan and continuous benchmarking on each new GPU or driver.

Sources

1. Google DeepMind - AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms - https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/
2. Novikov et al. - AlphaEvolve white paper (arXiv 2506.13131) - https://arxiv.org/abs/2506.13131
3. Ouyang et al. - KernelBench: Can LLMs Write Efficient GPU Kernels? (arXiv 2502.10517) - https://arxiv.org/abs/2502.10517
4. Sakana AI - The AI CUDA Engineer (including the later correction notice) - https://sakana.ai/ai-cuda-engineer/
5. OpenEvolve - open-source AlphaEvolve implementation - https://github.com/codelion/openevolve
6. Triton language and compiler - https://github.com/triton-lang/triton
7. Regulation (EU) 2024/1689 (AI Act) - https://eur-lex.europa.eu/eli/reg/2024/1689/oj