Yash Bishnoi
ML engineer, Glasgow
I work in PyTorch, Triton, CUDA and quantisation on an NVIDIA GB10. My fixes run upstream in PyTorch, Helion and bitsandbytes, and I've rebuilt two language models bit-exact from scratch.
I'm open to ML engineering roles in the UK or remote, and to relocating. I don't need UK sponsorship.
On fp32 and CPU, for this one prompt. See the result
Seven controlled changes to Qwen3-0.6B, each tested at matched compute across three seeds and scored on held-out text. A point to the right of zero means the model predicts that text better, in bits per byte. Filter the results, then open any row to see what happened. All of it ran on one NVIDIA GB10 box.
The largest effect in the study came from changing the data alone. Code gained 0.70 bits per byte and English did not move.
English −0.010, interval −0.026 to +0.006, seeds −0.006, −0.016, −0.007.
Code +0.703, interval +0.664 to +0.743, seeds +0.713, +0.699, +0.699.
The blend kept about 84% of the code gain and held English steady, so this mix went forward into mid-training.
English +0.016, interval −0.001 to +0.033, seeds +0.017, +0.004, +0.027.
Code +0.590, interval +0.553 to +0.628, seeds +0.590, +0.573, +0.608.
This edge fades with budget. Across the token ladder the English gap falls from 0.474 to 0.072, and at the top rung the fitted edge sits inside the noise floor. Code keeps a resolved edge.
English +0.474, interval +0.443 to +0.505, seeds +0.455, +0.478, +0.490.
Code +0.502, interval +0.456 to +0.547, seeds +0.516, +0.462, +0.527.
Value residual, layernorm scaling and head gating. Each one is significant on its own when tested separately.
English +0.118, interval +0.100 to +0.135, seeds +0.123, +0.107, +0.123.
Code +0.305, interval +0.259 to +0.351, seeds +0.295, +0.313, +0.306.
The control took the same learning-rate cooldown on FineWeb-Edu alone, so the data carries this gain and the schedule does not. I expected no change on English; it moved slightly.
English +0.016, interval +0.014 to +0.017, seeds +0.016, +0.015, +0.016.
Code +0.272, interval +0.264 to +0.279, seeds +0.272, +0.269, +0.274.
The interval crosses zero on both corpora, so the study claims nothing for this schedule.
English +0.024, interval −0.017 to +0.066, seeds +0.011, +0.030, +0.032.
Code +0.036, interval −0.065 to +0.137, seeds −0.003, +0.057, +0.054.
No measurable effect on either corpus.
English −0.003, interval −0.018 to +0.011, seeds −0.005, −0.006, +0.001.
Code −0.001, interval −0.052 to +0.051, seeds −0.019, +0.011, +0.006.
| Change | Budget | English | 95% interval | Code | 95% interval | Source |
|---|---|---|---|---|---|---|
| Swap the training data to dclm-edu | 131M tokens | −0.0097 | −0.0255 to +0.0061 | +0.7034 | +0.6639 to +0.7430 | Results file |
| Blend the data 50/50 | 131M tokens | +0.0161 | −0.0005 to +0.0327 | +0.5904 | +0.5532 to +0.6277 | Results file |
| Replace AdamW with NorMuon | 42M tokens, early training only | +0.4743 | +0.4435 to +0.5052 | +0.5016 | +0.4560 to +0.5471 | Results file |
| Add three architecture modules | 131M tokens | +0.1179 | +0.1005 to +0.1354 | +0.3049 | +0.2588 to +0.3511 | Results file |
| Anneal on the 50/50 mix | 150M tokens | +0.0157 | +0.0140 to +0.0174 | +0.2716 | +0.2641 to +0.2792 | Results file |
| Use a warmup-stable-decay schedule | 131M tokens | +0.0245 | −0.0173 to +0.0662 | +0.0361 | −0.0649 to +0.1372 | Results file |
| Add z-loss | 131M tokens | −0.0034 | −0.0175 to +0.0108 | −0.0007 | −0.0522 to +0.0508 | Results file |
Response-masked fine-tuning looked like a 0.68 perplexity win with a very tight interval. The two arms had been scored on different tokens. On one fixed held-out set the gap fell to 0.009, and a full-sequence check flipped its sign.
I pre-registered that GRPO wouldn't help at this size, because the base model solves about 1% of maths problems. After 300 steps it scored 1.0% on GSM8K against 1.12% for the fine-tuned floor, and it didn't beat a random-reward control. This run used a single seed, as registered.
Read the full study or browse the code. The repo also holds a SmolLM2-135M build that is bit-exact on CPU, and records how far it drifts on GPU.
New GPUs expose assumptions that older ones never tested. Here's one bug from my GB10, from failure to merged fix in PyTorch Helion.
test_scaled_mm failed on my GB10 before a single kernel could run.
OutOfResources: out of resource: shared memory,
Required: 147456, Hardware limit: 101376.GB10 had no heuristic file of its own, so the loader fell back to the sm90 file, whose configs were tuned on H100 with 227 KB of shared memory per block. The test checked for FP8 support but not for shared memory, so it ran a config that could never launch.
First, I guarded the test with the existing shared-memory check at 147,456 bytes, so it skips on smaller GPUs, GB10 included, and H100-class hardware still runs it.
Second, I generated an sm121 heuristic file on the GB10 for the two test shapes. One config covers both, at most 1.02× slower than the best per-shape config, and it fits the limit. This was a correctness fix, tuned quickly on two shapes, not a full performance pretune.
It merged on 20 August 2026. The wider problem is in Helion #4097, in review: on a GB10 with only the sm90 file, 19 of 24 decode shapes failed at first launch.
Fixed output stores splitting along the wrong dimension in epilogue_subtile. My fix from #3896 was included in this merged PR, with co-author credit.
Split OrderedSetVariable off SetVariable in TorchDynamo, so an OrderedSet keeps its order under torch.compile.
Split FrozensetVariable off SetVariable in TorchDynamo so torch.compile matches CPython.
Added sm121 to the wide_n_simt rule in 4-bit GEMM dispatch.
Added an sm121 tier to the 4-bit GEMM heuristic. To prove nothing else changed, I enumerated all 60,720 dispatch decisions in the test grid: 216 changed, and every one was on sm121.
See my landed pull requests and work in review, with what each one fixes. My ONNX MatMul documentation patch has also been accepted through the ONNX Slack, and the public pull request is still to come.
I reproduced the central result of Evan Duan's paper on steering side effects in sparse autoencoders. On GPT-2-small, decoder crowding predicted downstream collateral at a Spearman of 0.482 [0.381, 0.571], against the published 0.466. My first notebook scored 0.548 before I removed contexts ending in padding.
The paper hadn't controlled its collateral labels for firing frequency, so I ran that control across all four of its model and SAE settings. I'm now co-authoring the revision.
Open source contributor across PyTorch, Helion, bitsandbytes, HAMi and others
Independent ML research, alongside work as a Team Leader at Co-op with a team of 8+
MSc Computer Science, University of Dundee. Dissertation on deep learning for facial 3D reconstruction
Market Research Analyst at Nidhi Herbal, automating SQL and Python reporting
MBA, JECRC University
BSc Biotechnology, Amity University Jaipur
In August 2026 I placed 18th of 785 in the RSNA knee abnormality competition on Kaggle, with a macro ROC AUC of 0.903.