💬 Homemade GPT • JS

Homemade GPT JS is a minimal re-implementation of Andrej Karpathy's minGPT in TypeScript and TensorFlow.js. The whole GPT model, the decoder part of the transformer described in the Attention Is All You Need paper, fits in a single file of under 300 lines.

A web playground lets you train the model, experiment with its parameters and generate text right in the browser, on your GPU through WebGPU, and there's a Node.js playground as well. A simplified copy of the model processes attention heads one at a time, which is slower but easier to read, and presets range from tiny models up to the size of GPT-2.

It's made for learning how GPT works from the inside, and it pairs well with Karpathy's lecture Let's build GPT: from scratch, in code, spelled out.