We define the distance between two information structures as the largest possible difference in value across all zero-sum games. We provide a tractable characterization of distance and use it to discuss the relation between the value of information in games versus single-agent problems, the value of additional information, informational substitutes, complements, or joint information. The convergence to a countable information structure under value-based distance is equivalent to the weak convergence of belief hierarchies, implying, among other things, that for zero-sum games, approximate knowledge is equivalent to common knowledge. At the same time, the space of information structures under the value-based distance is large: there exists a sequence of information structures where players acquire increasingly more information, and ε > 0 such that any two elements of the sequence have distance of at least ε. This result answers by the negative the second (and last unsolved) of the three problems posed by J.F. Mertens in his paper Repeated Games , ICM 1986.