Making users pay more

As far as priming AI users to pay more -- the industry is already on it, in my opinion, as the title of this post suggests.

AI is offered today not (just) to make new users but mostly to make new addicts. This applies to both private and corporate users. Much is invested in making AI solutions as versatile as possible and as accessible as possible, regardless of how much they cost to operate. It is possible that AI today is not profitable to some of those who provide it, because those providers are investing rather than selling. The investment is towards creating consumer addiction to AI products. Once an engineering organization is fully invested in AI, and has structured its workforce accordingly, possibly reducing its own headcount, this organization is existentially dependent on this technology. Similarly, once individuals use AI across their devices for everything from search to running errands to doing all mental work for them, they will not be able to go back even if the price for those services increases tenfold. AI is going to cost a lot, a real lot, and the only customers who pay a real lot (for anything) are the addicts.

I believe that the current AI products are provided, to a large extent, to cause addiction to AI, not necessarily to be cost-efficient. They are undoubtedly doing pretty well at that.

Reducing computational cost

To understand how the compute cost of AI can be reduced, we need to first understand how it might be excessively spent.

Large Language Models (LLMs) everywhere

AI is not the abstract function we often treat it to be. AI is not just a large language model that understands everything and does anything. This is probably the more expensive type of AI, but also for the intentions I discussed above, this is the “AI” that is most aggressively promoted. This type of generic AI is the AI that is the closest to the person and the farthest from the machine. You can say or type anything you want in your natural language, and much compute is put into making the machine able to understand and operate on your prompt. It is easier for the person, but less so for the machine, and that “less so” implies more compute, which costs.

Programmers will realize that this is not different than other languages used in computer science. Some languages, like Python, are “closer” to the person, where programs are written intuitively using commands and constructs that are more natural for the programmer, but take extra CPU cycles to convert into the machine language equivalent that the computer runs. On the other hand, there are languages like Assembly, or even C, which are closer to machine language and hence create more efficient machine language representations, but are somewhere between “more difficult” (for C) and “nightmarish” (for Assembly) for the person to code in.

Some AI uses require the machine to speak human language. For example, a virtual personal assistant is worth nothing if you have to read a manual to be able to use it. Also, consultancy chat-bots, like those in support centers, are what they are precisely because they understand humans naturally. More profession-centric uses for AI, however, like those that assist in coding, do not necessarily need to rely on the same type of LLMs used for a call-center chat-bots to be effective. True, this might imply that your grandma will no longer be able to use AI to produce Python code, but is this our primary objective of AI? For those software engineering practitioners who need to produce code, the starting point at natural language is the cause of both higher cost and lower accuracy.

This applies not just to coding. When all uses of AI start by feeding natural language prompts into general-purpose LLMs, we pay an added computation price. Natural language is neither efficient nor accurate, and reaching complex reliable results requires very long (and often iterative) prompts that increase the overall computation cost, particularly in situations that blur the boundary between design-time and run-time (see below.)

The “one AI for everything” that is less efficient but gives worthwhile results to everyone in everything is what got us here, but may not necessarily be what takes us forward into sustainable efficiency and cost for AI when AI is truly everywhere.

Design at runtime

Most AI applications today are not written in an AI-native way. Software engineering, with its languages and development tools, has not yet entirely transformed into writing “real” AI programs. Of course, there are many programs written to utilize AI, and dozens of startups that focus on introducing AI into just about every field or task that humanity spends any time on, but most of those are still not AI-native applications, in my opinion. As I see it, an AI-native application is such that interacts directly with the LLM or the neural network to accomplish its tasks where those tasks are not only defined but are also designed and implemented outside the AI function that is called at runtime. This is not how many AI applications are still built today. Most AI applications today do not contain the design and implementation logic entirely in their code; they delegate much of that work to an LLM at runtime. In the more extreme cases, applications simply wrap their input with boilerplate instructions, and throw the whole bundle at the LLM to first figure out what needs to be done and then to also do it. Aside of the inefficiency and possible inaccuracy that this method brings, it essentially means that the implementation of the task is (at least partially) coded at runtime, over and over again each time the program runs; barely our most efficient use of energy.

Use of a single model

Whereas there are several LLMs being offered, with different strengths and different costs, many current AI-enabled applications do not use more than one of them, particularly not in the course of carrying out a single task. This again is a result of not really having to pay the full cost for using any of those models at scale. Application creators just take the model that is best and usable and write their application to use it. Going forward, it may make more sense for the application to decide at runtime what LLM to use. It may, for example, start with a relatively simple (and cheap) one, and if the task cannot be accomplished to an acceptable level (which itself needs to be defined in an efficiently-measurable way), gradually escalate to more expensive models. This is similar to how support centers gradually escalate support calls to higher-payed personnel if lower tiers fail to produce satisfactory solutions.

AI is at a transition state. It is growing up from having to focus on wide proliferation at the cost of efficiency, into improving its efficiency to make it affordable, at least by the AI addicts we become.