According to Marc Snir, William Gropp, and Peter Kogge
Most importantly, we need to reduce the amount of communication used by computations. One can expect improved algorithms to be the major source of communication reduction. There has been so far little work on communication-efficient algorithms (see [4,6], for recent exceptions): Reducing communication is often seen as a tuning issue, to be handled when the algorithm is coded, but not part of the algorithm design. This must change. We need to better understand the inherent communication requirements of various computations and trade-offs between computation and communication in time (storing to and loading from memory) and in space (communication across cores and nodes). Languages must enable locality control; compilers, runtimes and architecture must reduce the gap between the minimum amount of energy needed to move a chunk of data and the actual amount of energy spent by current memory and communication systems to move such a chunk.