Deep Thoughts with Tiki

At least I had fun taking the pictures

POV: mister Devon Price, PhD, telling me that I am right about everything

Lasers have a lot of uses! Measuring things, heating things, cutting things, entertaining cats, particle physics. Lasers are pretty cool. Very versatile, very useful, potential to be very powerful.

Someone shows up one day and says "I have developed a never before seen technology! I call it a Death Star."

And it's a 3.4mW laser. Well no, we haven't seen this exact size of laser much since that's not really standard, but that's a bit of a misnomer, and I wouldn't call it new -

"HOLY SHIT GUYS! This Death Star is so entertaining! My cat loves it and it has such a nice color!" The Death Star becomes a viral novelty, and is mildly entertaining, as laser pointers often are.

Somehow, seemingly overnight, this leads to mania. "Lets stick lasers in EVERYTHING! The public loves them!"

More companies make 3.4mW lasers to jump on the bandwagon. Everyone that makes anything vaguely mechanical starts sticking lasers into their designs.

Everyone is calling them Death Stars. Any time there is a "Death Star innovation", it is just that they made a bigger laser.

Ford's next truck comes out and it has "Death Star integrated headlights", where they have just stuck giant lasers in place of their previously functional headlights.

An electric toothbrush is now "Powered by Death Stars" and shoots a laser at the tooth its cleaning. You think that maybe this could have actual applications as a sanitizing device if you're being generous, but when you actually look at the product, its laser has no purpose but to point at the tooth and drain the battery.

Mechanical products across the board get noticeably worse as everyone starts stuffing lasers in places where lasers have no right to be.

The lamp business gets in on it. "Here's a Death Star powered lamp!" These guys haven't even tried to stick a laser in their damn lamps. They've just started calling their light bulbs Death Stars and hoped you bought it before you could tell the difference. You at least appreciate that they haven't ruined their lamp about it.

Death Stars are lauded as the solution to all the world's problems. If it's not working, you should stick a laser in it! That'll fix it, everyone says. Once in a blue moon, it's even true! Weather prediction is really good now. But most things are garbage. Like "Death Star powered washing machines". What the fuck does that even mean?

Meanwhile, since all functioning mechanisms are being replaced with lasers, problems start showing up. All mirrors now cost $1000+ dollars, because the whole supply is being used up to make more lasers. The earth heats up, because everyone's blasting lasers at everything. People keep going blind, on account of all the lasers.

You, in fact, study optical mechanics. You know what a laser is, and how it works, and that it was invented many years before any of this nonsense actually started. People keep asking you about Death Stars, since surely you must know so much about them.

You explain that this is not really what lasers are for, except you have to call them Death Stars now, and that they're causing a lot of harm, so you don't like them much.

"Oh, but they're still such new tech!" they reply. "They'll figure out how to make Death Stars that don't burn your eyes out soon, and then it won't be an issue anymore!"

Somewhere, deep and buried, you remember lasers being used in particle accelerators, or in telescopes, or in laser cutters, or funny cat videos. They are, in fact, still interesting. Still cool.

But by this point they have replaced roads with "Death Star Powered Pathways", which are just laser pointers propped up on tooth picks pointing vaguely through the forests.

And you think you are going mad.

And they are still just FUCKING LASERS.

NNs are fundamentally power hungry [read: inefficient] because they require so many calculations to train properly. There are exactly 3 advancements that have occurred in that space to bring them where we are currently from where we were 20+ years ago.

Computer hardware has gotten advanced enough to solve the differential calculus problems needed to tune in the model weights during backpropagation. [This still requires an insane amount of calculations, which goes to why it gets more expensive - not less - to train new models as they work on improvements.] We're already running into limitations here only one decade into this work. Each successive generation of hardware has diminishing returns on the capabilities over prior generations.

The sheer amount of data collected to train these models has ballooned since the early 2000's. This likely needs no further explanation, but there are a couple caveats to be aware of. One is that we're very close to "peak data" - the amount of data that's being consumed to train these models has outpaced the amount of data being added to the world every year. These are both astronomical figures - alas, one is larger than the other, and we will run out of fresh, usable data in the next couple years. [This aside from the fact that AI- generated data is now mucking up the landscape of potential data sources. This is akin to shitting upstream of where you draw water to drink.] The other caveat follows from the first. As the data itself becomes stale and less novel, efforts to train new models on it risk "overfitting" - the models start getting tunnel vision and less flexible on what their outputs should be. This is akin to when you overtrain for something and start making more mistakes because you're tired and everything looks the same.

After we got better hardware and more data, the renewed interest in neural networks [because it is a technology that goes back to the 60's and has already gone through several cycles of plateuing, waning interest, and eventual stagnation before some other new "trick" comes along], there have been legitimate advances in the algorithms used to train them. This is probably the most exciting and useful of the advancements but tends to not get discussed because NERD ALERT RESEARCH BORING WHO CARES yadda yadda.

Ultimately, large-language models and other neural networks can be powerful solutions to certain problems, but the domain of said problems isn't nearly as wide as the CEOs would have anyone believe. Also, they are fundamentally less efficient solutions compared to other machine learning solutions that often approach the capabilities of NNs without nearly the same resource load.

They have their place, but things like ChatGPT or Gemini or Stable Diffusion, Mid journey or whatever are of extremely marginal utility [one could argue negative utility] for what they require in order to exist in the first place. This is neither economically or ecologically sustainable, to say nothing of the ethics around data collection or the ripple effects these tools have on our psychology and society.

pride 🌈 collab part 2 with @rookfeathers!!! 🧡💛💚💙💜 lineart: me

we're looking to create an AI policy, and when we were talking about this, my boss (older millennial) was genuinely shocked to hear that younger people do not (seem) to view AI positively (a la the recent commencement speakers being booed)

please rb for larger sample size!

Question 1/3

Thank you for helping with this data collection. Please rb for as big a sample as possible!