(http://www.autoadmit.com/thread.php?thread_id=5656162&forum_id=2:#48500142)

Apple silicon doesn't doesn't have this problem. Neither does Qualcomm now. Enjoy paying $20k for less VRAM than you can get in a Macbook, TSINAH.

(http://www.autoadmit.com/thread.php?thread_id=5656162&forum_id=2:#48501577)

Regular people will sit shivering in the dark, munching on cold bugs in their micro-studio apartments, while the massive AI data centers glow and hum with energy off in the distance.

(http://www.autoadmit.com/thread.php?thread_id=5656162&forum_id=2:#48501623)

Key Factors Affecting Electricity Usage:

1. Model Size: Larger models, like GPT-4, consume more energy to process queries than smaller ones. The deeper the neural network, the more computation is needed.

2. Server Load: The number of users and queries being processed at any given time affects the overall power consumption. High demand can lead to more servers being activated.

3. Data Centers: AI models like mine run on clusters of high-performance GPUs or specialized hardware (e.g., TPUs) located in data centers. The energy consumption of these data centers includes cooling, power for servers, and other infrastructure.

4. Query Complexity: Simpler queries likely require less computation, while more complex or multi-turn interactions consume more energy.

General Estimation:

It’s difficult to pinpoint the exact energy cost for each query, but research into AI model energy consumption can provide a rough idea.

1. Energy per Inference:

• A single query or inference (answer) from a model like GPT-3 or GPT-4 could range from 0.1 kWh to 1 kWh depending on the size of the model, the hardware used, and the complexity of the query.

• For comparison, 1 kWh is about the energy needed to power a standard light bulb for 10-12 hours or run a microwave for 15-20 minutes.

2. Total Data Center Energy:

• Large data centers that host AI models can consume tens of megawatt-hours (MWh) per day, depending on the scale. However, the actual energy used for each query is a fraction of that total.

Conclusion:

While it’s hard to pinpoint the exact electricity consumption per question, each query likely consumes a very small amount of energy, possibly in the range of 0.1 to 1 watt-hour (Wh). Given the rapid advancements in more energy-efficient hardware and optimizations in data centers, the energy footprint per query is continuously decreasing, but it still adds up at scale across millions or billions of interactions.

The energy consumption is largely driven by the infrastructure behind the AI, including both hardware and cooling needs in the data centers, and not just the direct computational work of answering a single question.

(http://www.autoadmit.com/thread.php?thread_id=5656162&forum_id=2:#48501725)

https://epoch.ai/blog/can-ai-scaling-continue-through-2030

"We find that training runs of 2e29 FLOP will likely be feasible by the end of this decade. In other words, by 2030 it will be very likely possible to train models that exceed GPT-4 in scale to the same degree that GPT-4 exceeds GPT-2 in scale."

we are likely to see large increases in training compute availability in a few years, even with power being limited. the other issue is that the focus on training costs is the result of the poor generalization of transformers and current training techniques. they have to train them on every bit of data, because generalization is not great. the news stories are all focused on finding as much data as possible, or creating synthetic data to produce more robust models. humans don't train on 10 trillion words. if they could find algorithms with human level data efficiency, the compute costs would plummet. considering humans are able to learn to be reasonably competent in a few years (and a lot of that time spent processing irrelevant/redundant information) with brains that are clearly not optimized (with even the smartest people having substantial mutational load that impairs brain function), I would not be optimistic about theoretical costs. it wouldn't surprise me if AGI training costs could be <$100K even at current prices.

(http://www.autoadmit.com/thread.php?thread_id=5656162&forum_id=2:#48506410)

There is certainly a lot more they could be trained on. Videos are an obvious one. I think there are much more sensible ways of training these models

(http://www.autoadmit.com/thread.php?thread_id=5656162&forum_id=2:#48506577)