One concern that has been raised about LLMs, both in Bajohr’s piece and in a different way, in Parks’ piece about translation software, is the idea that LLMs “infer the future from the past.” In other words, ChatGPT and the like cannot create anything new unless they are fed new training data. Parks, referring to translation, writes that, “phrases which have been frequently used and translated before will be translated in the same way; everything that is new or unusual will present a problem.” ChatGPT cannot have its mind changed, so to speak, and if we imagine that future training corpuses will involve less human writing and more AI text, the idea of “value lock” seems likely (a term used by Bajohr). Weatherby (the writer of “ChatGPT Is an Ideology Machine”) formulates this idea aptly in reference to the inability to imagine a world outside of what he terms digital global capitalism: “what if the most average words, packaged in a “pre-digested form,” constitute the very horizon of that reality?”

Oh No, Not Another Essay on ChatGPT - 3 Quarks Daily
https://3quarksdaily.com/3quarksdaily/2023/05/oh-no-not-another-essay-on-chatgpt.html