I remember reading that from 2021-2023, LLMs generated more text than all humans had published combined - so arguably, actually human generated text is going to be a rarity

Will you sign my petition?

I’m an AI Developer.

TLDR: CUDA.

Getting ROCM to work properly is like herding cats.

You need a custom implementation for the specific operating system, the driver version must be locked and compatible, especially with a Workstation / WRX card, the Pro drivers are especially prone to breaking, you need the specific dependencies to be compiled for your variant of HIPBlas, or zLUDA, if that doesn’t work, you need ONNX transition graphs, but then find out PyTorch doesn’t support ONNX unless it’s 1.2.0 which breaks another dependency of X-Transformers, which then breaks because the version of HIPBlas is incompatible with that older version of Python and …

Inhales

And THEN MAYBE it’ll work at 85% of the speed of CUDA. If it doesn’t crash first due to an arbitrary error such as CUDA_UNIMPEMENTED_FUNCTION_HALF

You get the picture. On Nvidia, it’s click, open, CUDA working? Yes?, done. You don’t spend 120 hours fucking around and recompiling for your specific usecase.

Try using a 1-bit LLM to test the article’s claim.

The perplexity loss is staggering. It’s like 75% accuracy lost or more. It turns a 30 billion parameter model into a 7 billion parameter model.

Highly recommended that you try to replicate their results.

Hello, expert solarpunk here.

TLDR: Car battery is 350Wh. Fridge uses 143W idle, so it’ll run a fridge for 2-3 hours.

Explanation below:

Car batteries are lead-acid (sulphuric acid and lead plates).

They discharge according to Peukert’s Law as the negatively charged plate gets covered in lead via the acid (electrolyte).

As the battery depletes, the negative plate can begin to take permanent damage, and so you can’t discharge a lead-acid deeper than 10-20%, or about 10.8V, with the safe limit being ~50% discharge.

Most 12V, 60Ah batteries therefore only safely store and nominally discharge 350 Wh @ 350W.

You can discharge that as fast as you want but the faster you discharge, the lower the capacity is (with 1000-1500W bringing you way down to like 65 Wh). Fridges have a surge when they start up to fire up the compressor. Starter batteries can take that, but once the refrigerant is cold, the fridge just maintains the temperature which uses a lot less energy - about 143W on average.