I've seen similar arguments for a while, and even if it's true now, I'm not sure why it would stay at that level.
Even if you assume LLMs fully stop improving, heavy re-training for many customized models (acting as modules) attached to more traditional statistical systems should clear the "cat" level without too much trouble, but I don't see that happening. I don't think LLMs/other massive single training runs are out of steam yet.
Something among the lines of : "No matter how many resources you put into an LLM, it will always hallucinate". And researchers proven that : https://arxiv.org/abs/2401.11817
LLMs are just statistical models and just "regurgitate" what they've been trained by. They have no will, no desires, don't feel anger, hunger, joy, nothing. How can you you say LLM can match the cat intelligence "without too much trouble" with "heavy-retraining" ? This is a just some "alignement". Please don't overestimate LLMs as they just mimic just a few of visible human behaviour with some statistical tricks. You just need to show a dog once to a cat so it understands what a dog is, how many times do you need to show it to an LLM to understand what is a dog ?
Yes, well, it's a cat. They don't have the best mental stability anyway. For the "quick learning" stuff, that's why you would add the more traditional statistical systems. They would be "informed" by the large transformer models, but could function way faster and have "memory" of a sort.
Anger, hunger, and joy are just counters in a persistent program.
On the transformer models, there would be a bunch of them attached together in a mesh, with duplication and staged swapping out to fix the context length problem. Yeah, you'd probably need a supercomputer to run it, but that wasn't the question.
We don't have the same definition of "without too much trouble".
It doesn't rely too much on further AI progress. Sure, you'd need a really big training cluster, and an inference cluster, and a supercomputer to put the results together and shuffle the large transformer model I/O, and a good team, and a huge budget...
[dead]