ChatGPT and generative AI are booming, but the costs can be extraordinary

• The cost to develop and maintain the software can be extraordinarily high.
• Nvidia makes most of the GPUs for the AI industry, and its primary data center workhorse chip costs $10,000.
• Analysts and technologists estimate that the critical process of training a large language model such as GPT-3 could cost over $4 million.

Before OpenAI’s ChatGPT emerged and captured the world’s attention for its ability to create compelling sentences, a small startup called Latitude was wowing consumers with its AI Dungeon game that let them use artificial intelligence to create fantastical tales based on their prompts.

But as AI Dungeon became more popular, Latitude CEO Nick Walton recalled that the cost to maintain the text-based role-playing game began to skyrocket. AI Dungeon’s text-generation software was powered by the GPT language technology offered by the Microsoft-backed AI research lab OpenAI. The more people played AI Dungeon, the bigger the bill Latitude had to pay OpenAI.