A programmer fixes a bug before lunch. By the afternoon, the same class of error has appeared somewhere else. She can fix that one too, or she can change the development process so the next version of the mistake is caught before it reaches a customer. The first repair makes the program better. The second leaves behind a better way of making programs.
That inward turn is the subject of this book. Recursive self-improvement begins when an effort to improve reaches back into the machinery that produces improvement. The system changes how later answers will be found, how later mistakes will be noticed, or how later changes will be chosen. Once that machinery can change, a second problem follows close behind: the method used to judge improvement may also be inside the loop.
The loop turns inward. Each circuit can change the stairs used on the next ascent. Photo by Andreas Kind on Unsplash
More than an AI story
The phrase is often attached to artificial intelligence, usually alongside visions of a machine rewriting its own code and rapidly becoming more capable. That is one possible case, but it is too narrow a place to begin. People revise their study methods. Companies change how they run experiments. Democracies amend the rules under which they make rules. Markets fill with competitors who adapt to one another. Evolution changes populations without any organism understanding the process.
The resemblance is real, though it only goes so far. A person has purposes and can reflect on them. A market has no single mind. Evolution has neither a plan nor a unified self. Each case isolates a different part of the pattern: changes are tried, consequences are judged, lessons are retained, and sometimes the machinery carrying out those steps is changed in turn.
The comparison is useful because engineered systems make some features easy to see while human and social systems expose the complications. Software has explicit tests, but the tests can be incomplete. A person can change a habit, but self-knowledge is noisy. An organization can conduct a postmortem, but its incentives may reward a tidy story instead of an honest one. A constitution can be amended, but only through procedures the constitution already authorizes.
Improving the result is one achievement. Improving the way future results are improved is another.
The three questions
Any claim of recursive self-improvement should survive three questions. What is changing? A better output isn’t the same as a better method. What evaluates the change? Improvement requires some standard, even when the standard is implicit or distributed. What carries forward? Unless the change affects later attempts, the loop hasn’t turned inward.
Those questions keep ordinary repetition from masquerading as recursion. A thermostat acts again and again, but a conventional thermostat doesn’t redesign its control rule. A runner repeating the same workout may become fitter, but the training process becomes recursive only when results change how future workouts are selected. Feedback supplies information; learning changes behavior; recursive improvement changes some part of learning or optimization itself.
A practical test: After an improvement, ask what the system can now do differently when it next tries to improve. If the honest answer is “nothing,” the process may be useful, adaptive, or repeated, but it isn’t recursive in the sense used here.
This is also why “better” cannot remain a vague compliment. A system can become faster while growing fragile, more productive while exhausting its people, or more engaging while making its users miserable. Capability and direction are separate. A stronger improvement mechanism can pursue a poor objective more efficiently.
Recursive self-improvement gives us a way to notice where lessons are stored, who may alter the rules, which signals count as success, and whether a mistake can be undone. The chapters ahead follow that loop as it grows harder to see and harder to govern. Software writes lessons into tests. People carry them in habits. Organizations bury them in incentives. Markets, politics, and evolution add other agents whose responses change the next round. By the time the loop reaches artificial intelligence, the question is no longer only whether a system can get better. It is who still has the standing and the power to say what better means.