Angus Forrest

The Autonomous Model

Alex Wang’s article “The model is the company” argues that the next generation of startup companies will have vastly reduced headcounts with the knowledge located within a ‘Model’. The specifics of the Model are left a little vague, so I will make my best attempt at describing what the Model is. The Model is a machine that takes in information and outputs either actions or directives. Wang describes the Model as superintelligent, implying the extraordinary claim that the Model would be capable of outcompeting a similar business11 I realise that this could devolve into a No True Scotsman fallacy. I will define the comparative company as one with similar financial capital to the company running the Model.. He argues the Model would be capable of using the entire history of inputs, actions, and results to make informed decisions about future actions. Another feature of the Model is that it can work, learn and execute at any time.

The article is exuberantly optimistic. Wang is arguing that the next generation of startup companies will each be a demigod in a bottle22 Wang uses phrases like “superintelligent” and “complete context of the business” in his article, so forgive my literary flair..

I consider myself a technology advocate however Wang’s illustration fails to take into the physical constraints of limited focus a data resolution. As any undergraduate science or literature student can tell you, authoring a piece of work in not a unbiased affair, nor is collecting data. There is inherent choices being made on what to measure and what to keep. This is important when the customer is reliniqusing control to the Model as the biases of the customer and the Model must align for a fruitful relationship.

To simplify the picture, let us assume that the Model showcases complete accuracy and capability to perform its function. In principle the Model has solved its chosen domain. Let us call the owner of the Model The Company. The Company is now liable for the Model and its operation. It requires sales, insurance and customer relations teams: someone to sell and promote the Model, someone to negotiate liability when the Model is misaligned with reality, and someone to maintain the relationship through hard times. If asked, a techno optimist may argue that these roles ought to be delegated to a large language model or incorporated into the Model. The key word, though, is that it is a delegation of responsibility from The Company. Don’t believe for a second that the Model itself can hold responsibility, à la a computer can never be held accountable33 Attributed to a 1979 IBM training slide, a photo of which was shared online in 2017..

A techno optimist may concede that The Company will need the founder, a lawyer, and an insurer. However, the argument becomes a little curly when you start thinking about what the insurer needs. Will the insurer trust the Model to certify itself? That would be a very poor financial move on the insurer’s side. The task for The Company, then is to satisfy the insurer. How will the insurer be satisfied? I believe that will only occur once the Model’s processes can be validated in a way that is comprehensible and standardised. But who determines that the embedded logic is standard? Who can comprehend the Model? This all starts to sound like it requires expertise and knowledge from outside observers.

The techno optimist may argue that these validation services could be contracted out, perhaps even to other models. That misses the forest for the trees. The insurer cares that someone has made representations about how the Model behaves and is willing to stand behind them. Most risks are priced from loss history, the record of past claims, which is why freight and motor vehicle insurance are routine. A new Model has no loss history, so it falls into a different camp, alongside autopilot in aviation: insurable only once a certification regime, run by experts, vouches for it.

Footnotes

  1. I realise that this could devolve into a No True Scotsman fallacy. I will define the comparative company as one with similar financial capital to the company running the Model. ↩

  2. Wang uses phrases like “superintelligent” and “complete context of the business” in his article, so forgive my literary flair. ↩

  3. Attributed to a 1979 IBM training slide, a photo of which was shared online in 2017. ↩