a vocabulary of its own.
Custom tokenizers and carefully prepared training data lay the groundwork for learning.
From the ground up
the model is only
part of the story.
built on apple silicon.
We train models from scratch on Apple Silicon, exploring what accessible hardware can make possible.
results you can inspect.
We share evaluations, training records and limitations so others can inspect the results and build on the work.
Inside the training setup
distributed training
Our current cluster brings five Apple Silicon Macs together over a Thunderbolt RDMA ring. Explore the hardware, how the workload is split, and how every worker stays in sync.
Inside distributed training