· 10 min read tooling case-study
Teaching an LLM a Language It Has Never Seen
Fine-tuning a 4B parameter model to write valid Hew code, using the compiler as the test suite and a 780M iGPU as the training rig.
4 posts
Fine-tuning a 4B parameter model to write valid Hew code, using the compiler as the test suite and a 780M iGPU as the training rig.
Most of the gaps microgpt found got fixed within the week. Here's the refactoring sequence — commit by commit — and how close the output matches Python.
The point of porting a real program isn't to have a GPT. It's to find out where your language breaks. MicroGPT found 13 compiler issues and revealed that 31% of the code was working around language gaps.
I needed a real program to stress-test the compiler. A 120-line GPT turned into 1,092 lines of Hew, and a third of those lines were working around compiler bugs.