If your benchmark relies on a static dataset or sampling from a static distribution densely known at training time， then it is fundamentally measuring memorization/retrieval. Which might be fine if you're looking for a retrieval benchmark！ But don't confuse it with intelligence.