Two collaborative videos by @ClimateAdam and @AnkurShah on the relationship of Artificial Intelligence and Climate education, adaptation, and action.
Over the last ten years, AI, specifically deep learning, has yielded remarkable results. When Siri understands what you say, when Facebook identifies your cousin, when Google Maps reroutes you, chances are that a deep learning system is involved.
What is less noticed is that these models are churning away at a staggering cost, not just in terms of dollars and cents, but also in terms of energy consumed. On its current trajectory, AI will only accelerate the climate crisis. In contrast, our brains are incredibly efficient, consuming less than 40 watts of power. If we can apply neuroscience-based techniques to AI, there is enormous potential to dramatically decrease the amount of energy used for computation and thus cut down on greenhouse gas emissions. This blog post aims to explain what causes this outsized energy consumption, and how brain-based techniques can address AI’s incredibly high energy cost.
Why does AI consume so much energy?
First, it is worth understanding how a deep learning model works in simple terms. Deep learning models are not intelligent the way your brain is intelligent. They don’t learn information in a structured way. Unlike you, they don’t understand cause-and-effect, context, or analogies. Deep learning models are “brute force” statistical techniques. For example, if you want to train a deep learning model to identify a photo of a cat, you show it thousands of images of cats that have been labeled by humans. The model does not understand that a cat is more likely than a dog to climb up a tree or play with a feather, so unless it is trained with images of cats that include trees and feathers, it is unaware that the presence of these objects would aid in identifying a cat. To make these inferences, it needs to be trained in a brute force way with all possible combinations.
The enormous energy requirement of these brute force statistical models is due to the following attributes:
- Requires millions or billions of training examples. In the cat example, pictures are needed from the front, back, and side. Pictures are needed of different breeds. Pictures are needed with different colors and shadings, and in different poses. There are an infinite number of possible cats. To succeed at identifying a novel cat, the model must be trained on many versions of cats.
- Requires many training cycles. The process of training the model involves learning from errors. If the model has incorrectly labeled a cat as a raccoon, the model readjusts its parameters and classifies the image as a raccoon, then retrains. It learns slowly from its mistakes, which requires more and more training passes.
- Requires retraining when presented with new information. If the model is now required to identify cartoon cats, which it has never seen before, it will need to be retrained from the start. It will need to have blue cartoon cats and red cartoon cats added to the training set and be retrained from scratch. The model cannot learn incrementally.
- Requires many weights and lots of multiplication. A typical neural network has many connections, or weights, that are represented by matrices. For the network to compute an output, it needs to perform numerous matrix multiplications through subsequent layers until a pattern emerges on top. In fact, it often takes millions of steps to compute the output of a single layer! A typical network might contain dozens to hundreds of layers, making the computations incredibly energy intensive.
Seriously ?
We are going to spend our time decrying computer use instead of keeping our eye on stopping the burning of fossil fuels and how to speed up deployment of RE?
Computers get used whether they are assigned to AI modelling or not. In the grand scheme of things, how much would US emissions decrease if AI was banned? I’m thinking about zero.
It’s amazing how many articles and time are spent by so many on finger-pointing at the carbon footprints of individuals or demographic groups. Evidently the ‘enemy’ is:
Developed countries
Western Civilization
People who fly in airplanes
The Rich
People who have cars – even EV’s! – that are not small enough
People who have houses that are not small enough
People with too many children
People who eat meat
People who don’t use public transportation enough
People who don’t walk enough
People who don’t ride bikes enough
People who don’t use Zoom enough
Meanwhile, if we kept our eyes on the ball, we could have all the computing we want with essentially zero emissions. Why, we would have to use AI to figure out who to point our fingers at in future.
It should be both – energy replacement plus demand reduction. Not either/or. See indy222’s comment as to why. If we have continued high levels of growth, humans will just act like humans and continue to gobble up everything in sight. That has both great effects (and other kinds of effects than just CC) of environmental destruction while also making full energy replacement much more difficult to achieve. Growth keeps eating into the gains we have seen in renewables. Every headline about them seems to tout some new record of them being built, and yet we keep surpassing the yearly record in carbon emissions as well.
I don’t rule out the possibility that there will be some future where we don’t use fossil fuels anymore, but with continued growth we’re talking decades, at best. Energy replacement combined with lowered demand could slash that timing to a fraction.
This should be patently obvious.
All that said, though, I place zero faith in humans actually taking that path. So, growth it is. AI it is. Progress!
I don’t relish you your new role as The Crank Who Hates Computers.
We sit bathed in a billion times more free energy than we could possibly use, and you think we should be preaching energy austerity? That doesn’t smell like Enlightenment to me, it reminds of Stockholm Syndrome.
Our RE future could and should be one of low-cost energy abundancy and improved quality of life for a stable, not growing, population.
I don’t hate computers. I deeply distrust humans who use computers, seriously doubt human wisdom regarding technological advancements (we build new technologies because we can and because it enlarges our economy, yet we get more and more miserable as a human society, we create more and more powerful ways to destroy ourselves as a species, and there are always other destructive and unintended consequences to these advances), and I consider technological utopianism to be a pipe dream leading us off a cliff. I’ll live with being called a crank.
My lament at all these worthy efforts is that they neglect the Natural Selection elephant in the room. All these rapture-inducing improvements to energy grid balancing efficiency, better weather predictions, etc etcetc, only will only generate savings, including savings in the fraying of the networks of Civilization. What will humans do with savings? They’ll spend them, of course. Even more, they use them as an asset base in the money center banks from which to leverage yet more spending, impoverishing future generations. Every $ spent is a $ of local-entropy reducing “civilizing”, which will then require yet new energy consumption rates to support those ‘civilizings’ against the 2nd Law of Thermodynamics. And the end result is yet faster growth, or previously unattainable growth now attainable.
I get no goosebumps of optimism from any of these lost-in-the-weeds improvements in technology. Not until we solve the REAL problem, which is us, and worse yet, is in our Natural Selection determined motivational mechanisms which compel us, and also compel us to make the very economic / political systems which have brought us here. Without a fundamental change in human nature (CRISPr??), we will tip ourselves into Hot House Earth long before we could succeed in attaining true zero-growth sustainable Earth on a planet abundant with life as it once was.
The Progressives want every man woman and child in deepest darkest Africa and Southeast Asia to have the abundance of a modern European. The Republicans don’t give a rat’s behind about any damages, and just want mo’ money to fill their empty souls (or try to). No one wants what we actually require.
On my grandfather’s cabin wall, built in 1929 in northern California, is a bit of homey philosophizing with a German flavor…. “Vie are too soon Oldt, und too late Schmart”. Maybe this tragic observation is the solution to the Fermi Paradox, for all carbon based Galactic life.
Darwin called. He’d like his theory back, if you are done strangling it.
A welcome addition to YT! 🙂
Data centres can be powered by #CleanEnergy
Don’t stay stuck in “checkers thinking” . Graduate to “Chess thinking” and look 5 moves ahead to see what you’re so enraptured with. Short term gratifications have consequences.
Am I advocating for FFs? NO! Of course not. What’s the solution I’m advocating? Graceful de-growth to pull back to a truly sustainable Earth. I don’t see how focus exclusively on mo’ money for mo’ entrepreneurs is going to get us there. Human nature itself needs to change – and no one wants to talk about it. They’re all pumping for mo’ mo’ mo’. On a finite planet with finite rate of energy input, and 97% of all mammalian body mass already in humans and their pets and livestock, and the Wild Kingdom now crushed to only 3%. Where do you suppose this will end?