I think the main thing that’s happening is analogous to what’s happened with a lot of electronics over the past couple of decades. It seems like every electronic device runs off of a way more powerful computer than is necessary because it’s easier/cheaper to buy a million little computers and do a little programming than it is to have someone design a bespoke circuit, even if the bespoke circuits would be more resource efficient, robust, and repairable. Our dishwashers don’t need wifi, but if you are running them off a single board computer with wifi built in, why wouldn’t you figure out a way to advertise it?

Similarly, you have all sorts of tasks that can be done with way more computational efficiency (and trust and tweakability) if you have the know-how to set something bespoke up, but it’s easier to throw everything at an overpowered black box and call it a day.

The difference is that manufacturing costs for tiny computers can come down to be cheaper in price relative to a bespoke circuit, but anything that decreases the cost of computing will apply equally to an LLM and a less complex model. I just hope industry/government pushing isn’t enough to overcome what the “free market” should do. After all, car centric design (suburbia, etc) is way less efficient than train centric, but we still went there.

My work would be improved by the dumbest of dumb retrieval augmented models: a monkey with a thesaurus, ctrl+f, and a pile of my documents. Unfortunately, the best they can offer is a service where I send my personal documents into the ether and a new wetland is dried in my honor (or insert your ecological disaster metaphor of choice).

If there is a demand for a forensic capability, there’s someone willing to sell it to a police department (and a jury).