The mentioned but unsupported link to “general intelligence” reeks of bullshit to me. I don’t doubt a modified LLM (maybe an unmodified one as well) can beat lossless compression algorithms, but I doubt that’s very useful or impressive when you account for the model size and speed.

If you allow the model to be really huge in comparison to the input data it’s hard to prove you haven’t just memorized the training set.