I have a newsletter to produce, and I wondered if recursive self-improvement might be useful for me as well. Could I use AI to train and continually improve a model that automates some of this newsletter’s busywork? After a week or so of experimenting, the answer appears to be a resounding and surprising hell yes. What is more, dabbling with self-improving models shows a different vision for how AI might unfold, one that does not center on a handful of companies that control the whole industry.
Starting with AI experimentation
I started by trying out a simple self-improving loop. To get my feet wet, I experimented with training a small language model from scratch, by which I mean I dumped all the hard work on Claude’s plate. I installed AutoResearch, which helps an off-the-shelf AI model build and improve a smaller model. AutoResearch is the brainchild of Andrej Karpathy, a superstar AI researcher who helped found OpenAI, led AI work at Tesla, and recently joined Anthropic.
I fired up Claude and gave it the recommended instruction: 'Hi, have a look at program.md and let us kick off a new experiment!' While Claude did the hard stuff, I provided the silicon (an Nvidia DGX, a desktop supercomputer designed for AI experimentation), the electricity (running hot for a few days straight), and a possibly ill-advised willingness to let the model skip all the usual permission checks in order to do its thing. I checked in on the AutoResearch project every few hours and marveled as Claude adjusted parameters and training regimes, looked at how this changed the smaller model’s output, and went on refining it further.
Here is what an early version of that smaller language model produced when I prompted it to complete the phrase 'In the beginning': 'In the beginning of the beginning of the end of the end of the end end of end end end end end end end end beginning end end end end.' Not so brilliant. But later models, improved autonomously by Claude, got more coherent and less prone to insane, endless repetition. It is hardly GPT-5, but it showed a promising path toward continual improvement.
Moving to complex and useful tasks
My journey continued with something more complex and useful. I already use an agent that relies on Claude to help me find noteworthy research papers, so I decided to see whether it was possible to build something that went beyond that. I turned to a tool from a startup called Prime Intellect, which uses AI to train a custom model for a specific task. I collected 100 or so previous 'Elsewhere on the frontier of AI' entries, the bits and bobs of research that follow the main essay in my newsletter. Then, I created a Prime Intellect training environment and asked Claude to help me build my own model, which it dubbed Frontier_Paper_Curator, to find and summarize interesting papers.
Claude found more papers and generated a bunch of synthetic data to help with training. It then tapped yet another model to assess Frontier_Paper_Curator’s output, while the training environment also improved the model with reinforcement learning. Vincent Weisser, CEO of Prime Intellect, which recently received $15 million in funding, tells me that his company aims to make recursive self-improvement accessible to everyone, not just frontier labs. The models made by frontier labs might be brilliant, but democratizing this kind of AI training could produce just as capable specialized models, he says.
'Give every company access to frontier training infrastructure, and the collective creativity of the market unlocks far more than any handful of labs can,' Weisser says. 'We do not want one centralized, almost godlike intelligence, we want a billion intelligences that go into all the niches that create beautiful things.'
Prime Intellect is not the only company that sees the future this way. Adaption, another startup, offers a tool called AutoScientist, which automates AI model training. CEO Sara Hooker says Adaption is working with several large companies that are burning through tokens and do not have in-house AI experts. When Anthropic decided to block certain requests to its latest model Fable 5, it exposed the risk of relying too heavily on one frontier model. And some executives, like Palantir’s Alex Karp, have warned that using frontier labs also means handing over your own data and control over your technology.
The ultimate goal for recursive self-improvement is for AI to apply novel ideas to a model and come up with its own insights. The tools available to the rest of us are more limited, but still impressive. After less than a day of cooking with Prime Intellect, I was able to create a surprisingly good model for finding and summarizing research. Here is one example entry it created for me: Researchers at iFLYTEK have developed iFLYTEK-Embodied-Omni, a unified multimodal AI model that integrates vision, language, and action generation into a single framework. Unlike prior embodied agents which treat visual understanding, future state prediction, and action generation separately, their model uses shared multimodal self-attention to enable close coordination between a vision-language high-level brain and an action-generating low-level cerebellum. This approach reduces error compounding and interface bottlenecks common in cascaded pipelines.
Not bad for a first try. The new model is still a bit overeager, choosing too many papers that I would skip, and its summaries are a tad generic. But it is a promising start. Here is hoping I can one day use it to free me from the chains of busywork.
