I don’t think energy use is a serious problem, that just seems to get thrown around just because it’s trendy. Does it even matter compared to gaming or crypto? It’s also an easily solved problem, just install more solar. Training the initial model isn’t time critical or depended on location, so there is a lot of flexibility here that you wouldn’t have in other applications. Meanwhile running the already trained model is very cheap, it’s literally the most efficient way to solve the problem. Trying to replicate what StableDiffusion is doing with a 3D renderer and you’d need to burn a heck of a lot more cycles, as well as hire a truckload of artists, which would all use substantially more energy.
Basically, people are going to use AI when it makes better use of time/money/energy than the competition. Nobody is going to use AI to burn energy just for the fun of it, it has to improve on what we already have.
As for the concentration of power and wealth, that can certainly happen to some degree, but I could also easily see that get balanced out by the amount of freedom that local models give. Right now I can generate subtitles for video with Whisper, generate voices with tortoise-tts, generate images with StableDiffusion as well as play around with LLMs on my local machine with OpenSource’ish models. Nobody controls what I do and I am not paying for anything. There are obviously still aspects that those models can’t do, local LLMs aren’t up to GPT-4, but already quite close to ChatGPT for some tasks, StableDiffusion isn’t quite as good as Midjourney for plain txt2img, but state-of-the-art in a lot of other aspects (custom training, ControlNet, LORA, etc.). But for a lot of tasks those models are already “good enough” and they are constantly getting better. Meanwhile ChatGPT or BingChat are so heavily censored that they flat out just don’t work for a lot of task, even seemingly simple things like summarizing movies (too much violence). Nobody even talks about DALL-E2 anymore, due to being surpassed by everything else out there.
Now centralization can still happen, Google is sitting on more data than everybody and if they make some multi-modal model that is trained on it all, that could be a very potent offering. But for the time being at least, everything that was released was outclassed by another thing within a few months. Nothing in the AI space so far lasts very long and the fact that AI models can use other AI models to improve themselves, hopefully makes that continue for a while. With the censorship going on I also have a hard time seeing local models disappearing anytime soon, as so far none of the commercial offerings had the balls to just build a model that knows everything.
I don’t think energy use is a serious problem, that just seems to get thrown around just because it’s trendy. Does it even matter
Yes, since it’s a rapidly growing field.
compared to gaming or crypto?
Proof-of-work based tokens are the enemy and not what we should be comparing things to.
It’s also an easily solved problem, just install more solar.
It’s a little trickier than that. Renewable doesn’t mean infinite; we still need to limit consumtion to sustainable rates. Also, there is the hardware in the rigs themselves. Solvents, wiring, metals, plastic…
Training the initial model isn’t time critical or depended on location, so there is a lot of flexibility here that you wouldn’t have in other applications.
That’s a good point. It’s less vulnerable to wind or light conditions.
Meanwhile running the already trained model is very cheap, it’s literally the most efficient way to solve the problem.
Yep. I never argued against that part. That’s great, as long as we can hold it together and not make new models every fifteen minutes just to keep up with the joneses, but there’s also a drawback to the “expensive to train, cheap to run” model: that’s the very thing that is driving the wealth concentration of big capital like Google.
Basically, people are going to use AI when it makes better use of time/money/energy than the competition. Nobody is going to use AI to burn energy just for the fun of it, it has to improve on what we already have.
That would be a perfect argument if we had accounted-for environmental transaction externalities, but we don’t. Using energy is cheaper than it “should” be to account for the environmental impact of that energy use. The old “if I sell you a can of gas, the price of the forest that got wrecked by that gas isn’t factored in” problem. Even otherwise laissez-faire stalwarts like Hayek acknowledged this.
As for the concentration of power and wealth, that can certainly happen to some degree, but I could also easily see that get balanced out by the amount of freedom that local models give.
Right; once it does get truly democratized with open source model we can have a post-scarcity pay-it-forward future where the step from dream to reality is smaller than ever before.
We’ve been through backs and forths of this. The big data mainframe era was replaced by PC. Then that got centralized again in the age of big dialup. But then with broadband everyone could run a server. And then the web 2.0 debacle happened and we got a silo era where people voluntarily started using Google Search and Facebook Messenger and stuff like that to give big capital ownership of our platforms.
You seem like you have your head on your shoulders (you’re on feddit, after all) but among the general population there’s a lack of awareness around these power&wealth-concentration issues.
Now centralization can still happen, Google is sitting on more data than everybody and if they make some multi-modal model that is trained on it all, that could be a very potent offering.
Yes, and I want a plan for that.
Nothing in the AI space so far lasts very long
Which is why we’re risking runaway energy use and climate impact.
Yes, the fossil economy has enabled society as a whole to create temporary wealth; the past has borrowed from the present. It’s going to be a rough comedown.
We haven’t been, and still aren’t, commensurately accounting for our environmental externalities.
I don’t think energy use is a serious problem, that just seems to get thrown around just because it’s trendy. Does it even matter compared to gaming or crypto? It’s also an easily solved problem, just install more solar. Training the initial model isn’t time critical or depended on location, so there is a lot of flexibility here that you wouldn’t have in other applications. Meanwhile running the already trained model is very cheap, it’s literally the most efficient way to solve the problem. Trying to replicate what StableDiffusion is doing with a 3D renderer and you’d need to burn a heck of a lot more cycles, as well as hire a truckload of artists, which would all use substantially more energy.
Basically, people are going to use AI when it makes better use of time/money/energy than the competition. Nobody is going to use AI to burn energy just for the fun of it, it has to improve on what we already have.
As for the concentration of power and wealth, that can certainly happen to some degree, but I could also easily see that get balanced out by the amount of freedom that local models give. Right now I can generate subtitles for video with Whisper, generate voices with tortoise-tts, generate images with StableDiffusion as well as play around with LLMs on my local machine with OpenSource’ish models. Nobody controls what I do and I am not paying for anything. There are obviously still aspects that those models can’t do, local LLMs aren’t up to GPT-4, but already quite close to ChatGPT for some tasks, StableDiffusion isn’t quite as good as Midjourney for plain txt2img, but state-of-the-art in a lot of other aspects (custom training, ControlNet, LORA, etc.). But for a lot of tasks those models are already “good enough” and they are constantly getting better. Meanwhile ChatGPT or BingChat are so heavily censored that they flat out just don’t work for a lot of task, even seemingly simple things like summarizing movies (too much violence). Nobody even talks about DALL-E2 anymore, due to being surpassed by everything else out there.
Now centralization can still happen, Google is sitting on more data than everybody and if they make some multi-modal model that is trained on it all, that could be a very potent offering. But for the time being at least, everything that was released was outclassed by another thing within a few months. Nothing in the AI space so far lasts very long and the fact that AI models can use other AI models to improve themselves, hopefully makes that continue for a while. With the censorship going on I also have a hard time seeing local models disappearing anytime soon, as so far none of the commercial offerings had the balls to just build a model that knows everything.
Yes, since it’s a rapidly growing field.
Proof-of-work based tokens are the enemy and not what we should be comparing things to.
It’s a little trickier than that. Renewable doesn’t mean infinite; we still need to limit consumtion to sustainable rates. Also, there is the hardware in the rigs themselves. Solvents, wiring, metals, plastic…
That’s a good point. It’s less vulnerable to wind or light conditions.
Yep. I never argued against that part. That’s great, as long as we can hold it together and not make new models every fifteen minutes just to keep up with the joneses, but there’s also a drawback to the “expensive to train, cheap to run” model: that’s the very thing that is driving the wealth concentration of big capital like Google.
That would be a perfect argument if we had accounted-for environmental transaction externalities, but we don’t. Using energy is cheaper than it “should” be to account for the environmental impact of that energy use. The old “if I sell you a can of gas, the price of the forest that got wrecked by that gas isn’t factored in” problem. Even otherwise laissez-faire stalwarts like Hayek acknowledged this.
Right; once it does get truly democratized with open source model we can have a post-scarcity pay-it-forward future where the step from dream to reality is smaller than ever before.
We’ve been through backs and forths of this. The big data mainframe era was replaced by PC. Then that got centralized again in the age of big dialup. But then with broadband everyone could run a server. And then the web 2.0 debacle happened and we got a silo era where people voluntarily started using Google Search and Facebook Messenger and stuff like that to give big capital ownership of our platforms.
You seem like you have your head on your shoulders (you’re on feddit, after all) but among the general population there’s a lack of awareness around these power&wealth-concentration issues.
Yes, and I want a plan for that.
Which is why we’re risking runaway energy use and climate impact.
If you were right about markets only using energy when it made sense, we wouldn’t have this problem:
World population and our standard of living have improved drastically over those years too, we aren’t burning that additional energy for nothing.
Yes, the fossil economy has enabled society as a whole to create temporary wealth; the past has borrowed from the present. It’s going to be a rough comedown.
We haven’t been, and still aren’t, commensurately accounting for our environmental externalities.