a16z基础设施团队的合伙人Jennifer Li在Big Ideas报告里说了一句让很多人印象深刻的话：企业AI现在最大的瓶颈，不是模型不够聪明，而是自己的数据太乱。她用了一个词——"数据熵"。每家公司都淹没在PDF、截图、邮件、操作日志里，80%的企业知识以非结构化的形式散落在各个角落，从来没有被系统整理过。你买了最好的模型，搭了最贵的系统，但喂进去的是一团乱麻，出来的自然是错误和幻觉。
Returning back to the Anthropic compiler attempt: one of the steps that the agent failed was the one that was more strongly related to the idea of memorization of what is in the pretraining set: the assembler. With extensive documentation, I can’t see any way Claude Code (and, even more, GPT5.3-codex, which is in my experience, for complex stuff, more capable) could fail at producing a working assembler, since it is quite a mechanical process. This is, I think, in contradiction with the idea that LLMs are memorizing the whole training set and uncompress what they have seen. LLMs can memorize certain over-represented documents and code, but while they can extract such verbatim parts of the code if prompted to do so, they don’t have a copy of everything they saw during the training set, nor they spontaneously emit copies of already seen code, in their normal operation. We mostly ask LLMs to create work that requires assembling different knowledge they possess, and the result is normally something that uses known techniques and patterns, but that is new code, not constituting a copy of some pre-existing code.。Line官方版本下载是该领域的重要参考
。关于这个话题，WPS官方版本下载提供了深入分析
1L decoder, d=7, 1h, ff=14。业内人士推荐91视频作为进阶阅读
Author(s): Jun Chai, Javier LLorca