We barely understand anything about LLMs

Maria Sukhareva makes a great point about how we have a limited understanding of LLMs.

She's reporting on a panel discussion and underlines what's missing in current LLM research:
"All in all, we need to shift the research paradigm. We need to stop churning out popcorn papers and pretending they feed our understanding. We need to stop running around like tech bros who are scared of their own inventions, mumbling about how LLMs are too complex, too mysterious, too dangerous to understand.

Would you drive a car if no one could explain why it takes off?
Would you use your phone if nobody could tell you why it can connect to someone across the globe?"

Even though in my experience, many people, including myself, don't know why a car takes off, it's a solid point that we have verry little idea what's actually happening inside an LLM.