Happy Sunday and welcome to InsideAI! I'm Rob May, CTO at Dianthus. My goal here in the weekend commentary is to share with you some of what I see in the market as an investor and operator in the AI space. If you like the newsletter, please forward it to a friend.
This week I want to briefly discuss this article about how AI workloads are moving out of the cloud and back on prem. Those of us that have been around tech a long time often joke that compute infrastructure just keeps changing from local and decentralized to remote and centralized. Mainframes -> Workstations -> Client-Server -> Desktops -> Cloud... it just keeps cycling back and forth where we run our computations. But over the last 10-15 years as stuff has moved to the cloud, the tech industry has grown dramatically and so cloud has become more tech gospel than these cycles would have been in previous eras.
But to the point in the article, there are cost reasons around both training and inference to move compute local, or in some cases to the edge.
“The high cost of training is giving people some challenges,” said Danny Lange, vice president of AI and machine learning at gaming and automotive AI company Unity Technologies. The cost of training can run into millions of dollars, Lange said.
“It’s a cost that a lot of companies are now looking at saying, can I bring my training in-house so that I have more control on the cost of training, because if you let engineers train on a bank of GPUs in a public cloud service, it can get very expensive, very quickly.”
This is an underappreciated trend, and very investable against. When infrastructure becomes more local and hybrid again, companies need new tools to manage it. New chips, new software, and new cases are going to keep this in flux for a long time. Foundational models influence this trend as well. I believe we are still 7+ years away from standardizing on common AI tech stacks. This creates lots of space for innovation and investment, so if you are working at this intersection, I'd love to hear from you.
Thanks for reading.
@robmay
Comments
Post a Comment