![\"ai-brains-efficiency\"\/](\"https:\/\/imageio.forbes.com\/specials-images\/imageserve\/63e6a4164b4576e4ebe5ccbf\/ai-brains-efficiency\/960x0.jpg?height=46&width=71&fit=bounds\")
<\/figure>\n\n\n\n\u201cWe have to give up immortality,\u201d the CEO of Rain AI, Gordon Wilson, told me in a recent TechFirst podcast<\/a>. \u201cWe have to give up the idea that, you know, we can save software, we can save the memory of the system after the hardware dies.\u201d<\/p>\n\n\n\n Wilson is quoting Geoff Hinton, a cognitive psychologist and computer scientist, author or co-author of over 200 peer-reviewed publications, current Google employee working on Google Brain, and one of the \u201cgodfathers\u201d of deep learning. At a recent NeurIPS machine learning conference, he talked about the need for a different kind of hardware substrate to form the foundation of AI that is both smarter and more efficient. It\u2019s analog and neuromorphic \u2014 built with artificial neurons<\/a> in a very human style \u2014 and it\u2019s co-designed with software to form a tight blend of hardware and software that is massively more efficient than current AI hardware.<\/p>\n\n\n\n Achieving this is not just a nice-to-have, or a vague theoretical dream.<\/p>\n\n\n\n Building a next-generation foundation for artificial intelligence is literally a multi-billion-dollar concern in the coming age of generative AI and search. One reason is that when training large language models (LLM) in the real world, there are two sets of costs to consider.<\/p>\n\n\n\n Training a large language model like that used by ChatGPT is expensive \u2014 likely in the tens of millions of dollars \u2014 but running it is the true expense. Running the model, responding to people\u2019s questions and queries, uses what AI experts call \u201cinference.\u201d<\/p>\n\n\n\n That\u2019s precisely what runs ChatGPT compute costs into the millions regularly. But it will cost Microsoft\u2019s AI-enhanced Bing much more.<\/p>\n\n\n\n And the costs for Google to respond to the competitive threat and duplicate this capability could be literally astronomical.<\/p>\n\n\n\n \u201cInference costs far exceed training costs when deploying a model at any reasonable scale,\u201d say<\/a> Dylan Patel and Afzal Ahmad in SemiAnalysis. \u201cIn fact, the costs to inference ChatGPT exceed the training costs on a weekly basis. If ChatGPT-like LLMs are deployed into search, that represents a direct transfer of $30 billion of Google\u2019s profit into the hands of the picks and shovels of the computing industry.\u201d<\/p>\n\n\n\n If you run the numbers like they have, the implications are staggering.<\/p>\n\n\n\n \u201cDeploying current ChatGPT into every search done by Google would require 512,820 A100 HGX servers with a total of 4,102,568 A100 GPUs,\u201d they write. \u201cThe total cost of these servers and networking exceeds $100 billion of Capex alone, of which Nvidia would receive a large portion.\u201d<\/p>\n\n\n\n Assuming that\u2019s not going to happen (likely a good assumption), Google has to find another way to approach similar capability. In fact, Microsoft, which has only released its new ChatGPT-enhanced Bing in very limited availability for very good reasons probably including hardware and cost, needs another way.<\/p>\n\n\n\n Perhaps that other way is analogous to something we already have a lot of familiarity with.<\/p>\n\n\n\n According to Rain AI\u2019s Wilson, we have to learn from the most efficient computing platform we currently know of: the human brain. Our brain is \u201ca million times\u201d more efficient than the AI technology that ChatGPT and large language models use, Wilson says. And it happens to come in a very flexible, convenient, and portable package.<\/p>\n\n\n\n \u201cI always like to talk about scale and efficiency, right? The brain has achieved both,\u201d Wilson says. \u201cTypically, when we\u2019re looking at compute platforms, we have to choose.\u201d<\/p>\n\n\n\n That means you can get the creativity that is obvious in ChatGPT or Stable Diffusion, which relies on data center compute to build AI-generated answers or art (trained, yes, on copyrighted images), or you can get something small and efficient enough to deploy and run on a mobile phone, but doesn\u2019t have much intelligence.<\/p>\n\n\n\n That, Wilson says, is a trade-off that we don\u2019t want to keep having to make.<\/p>\n\n\n\n Which is why, he says, an artificial brain built with memristors that can \u201cultimately enable 100 billion-parameter models in a chip the size of a thumbnail,\u201d is critical.<\/p>\n\n\n\n For reference, ChatGPT\u2019s large language model is built on 175 billion parameters, and it\u2019s one of the largest and most powerful yet built. ChatGPT 4, which rumors say is as big a leap from ChatGPT 3 as the third version was from its predecessors \u2014 will likely be much larger. But even the current version used 10,000 Nvidia GPUs<\/a> just for training, with likely more to support actual queries, and costs about a penny an answer<\/a>.<\/p>\n\n\n\n Running something of roughly similar scale on your finger is going to be multiple orders of magnitude cheaper.<\/p>\n\n\n\n And if we can do that, it unlocks much smarter machines that generate that intelligence in much more local ways.<\/p>\n\n\n\n \u201cHow can we make training so cheap and so efficient that you can push that all the way to the edge?\u201d Wilson asks. \u201cBecause if you can do that, then I think that\u2019s what really encapsulates an artificial brain. It\u2019s a device. It\u2019s a piece of hardware and software that can exist, untethered, perhaps in a cell phone, or AirPods, or a robot, or a drone. And it importantly has the ability to learn on the fly. To adapt to a changing environment or a changing self.\u201d<\/p>\n\n\n\n That\u2019s a critical evolution in the development of artificial intelligence. Doing so enables smarts in machines we own and not just rent, which means intelligence that is not dependent on full-time access to the cloud. Also: intelligence that doesn\u2019t upload everything known about us to systems owned by corporations we end up having no choice but to trust.<\/p>\n\n\n\n It also, potentially, enables machines that differentiate. Learn. Adapt. Maybe even grow.<\/p>\n\n\n\n My car should know me and my area better than a distant colleagues\u2019 car. Your personal robot should know you and your routines, your likes and dislikes, better than mine. And those likes and dislikes, with your personal data, should stay local on that local machine.<\/p>\n\n\n\n There\u2019s a lot more development, however, to be done on analog systems and neuromorphic computing: at least several years. Rain has been working on the problem for six years, and Wilson thinks shipping product in quantity \u2014 10,000 units for Open AI, 100,000 units for Google \u2014 is at least \u201ca few years away.\u201d Other companies like chip giant Intel are also working on neuromorphic computing with the Loihi chip<\/a>, but we haven\u2019t seen that come to the market in scale yet.<\/p>\n\n\n\n If and when we do, however, the brain-emulation approach shows great promise. And the potential for great disruption.<\/p>\n\n\n\n \u201cA brain is a platform that sports intelligence,\u201d says Wilson. \u201cAnd a brain, a biological brain, is hardware and software and algorithms all blended together in a very deeply intertwined way. An artificial brain, like what we\u2019re building at Rain, is also hardware plus algorithms plus software, co-designed, intertwined, in a way that is really … inseparable.\u201d<\/p>\n\n\n\n Even, possibly, at shutdown.<\/p>\n\n\n\n Get a <\/em>full transcript<\/em><\/a> of our conversation, or subscribe to <\/em>TechFirst<\/em><\/a>.<\/em><\/p>\n\n\n\n Follow me on Twitter<\/a> or LinkedIn<\/a>. Check out my website<\/a> or some of my other work here<\/a>. <\/p>\n\n\n\n I forecast and analyze trends affecting the mobile ecosystem. I’ve been a journalist, analyst, and corporate executive, and have chronicled the rise of the mobile economy. I built<\/p>\n","protected":false},"excerpt":{"rendered":" Running ChatGPT costs millions of dollars a day, which is why OpenAI, the company behind the viral natural-language processing artificial intelligence has started ChatGPT Plus, a $20\/month subscription plan. But our brains are a million times more efficient than the GPUs, CPUs, and memory that make up ChatGPT\u2019s cloud hardware. And neuromorphic computing researchers are […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7,67],"tags":[],"class_list":["post-1108","post","type-post","status-publish","format-standard","hentry","category-ai","category-machine-learning"],"_links":{"self":[{"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1108","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1108"}],"version-history":[{"count":1,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1108\/revisions"}],"predecessor-version":[{"id":1109,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1108\/revisions\/1109"}],"wp:attachment":[{"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1108"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1108"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.carloswsmith.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1108"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}