Tech is heavily invested in giant language models.
This is because, with transfer learning, language models suddenly started being able to do a LOT more. And a lot better.
A lot of high citation publications have also been an outcome of this exploration of language models
For most parts in NLP engineering, LMs went from being cute to being the standard model from which you build solutions in the last three years. And it’s not just in NLP, the language model approach (use large amounts of data to train a network to do specific tasks, and some approximation of the underlying first principles will be approximately encoded into the model) seems to work well for images as well (stable diffusion).
And given that all of deep learning and the engineering approach to AI seem to be approximate/mimic till you make it, this is a powerful learning.
For academics, in a sense, iterative improvement, criticism, existential threats etc. are part of the publish-perish cycle. For the engineer, iterative improvement is, but it is driven by entirely different incentives.
When the ethics people started talking about bias, the engineering world was not very happy about it, well no one likes the police. More so because bias is not easy to tackle, its not easily quantified, its not something you can fix with a software update. And because there doesn’t seem to be any consensus yet about how to deal with it.
I think there are distinct parallels here from the early days of medical ethics and evidence based medicine. No one likes to hear that their life’s work is actually harming people despite having honorable intentions. We’d all like to be told that well, this is OK, you were doing the right thing, and we can fix it, easy.
But as the decades of literature in medical ethics and the history of clinical medical practice shows, ethics isn’t easy but is also not optional. The medical world was brought kicking and screaming into getting its shit together and behaving.