I wouldn't know for sure, might depend on century and source. In Aeschines I (4th century B.C.E.), there is only the mention of olive oil and not sand in relation to the gymnasium.
Edit; Apparently according to Pausanias it was olive oil or sand in the 8th century B.C.E., I gotta look up his...