A beautiful essay by Emrys Westacott concludes
Ultimately, the ideal college is one in which every student is genuinely interested in learning and needs neither extrinsic motivators to encourage study, nor surveillance to deter cheating. Ultimately, the ideal society is one in which, if taxes are necessary, everyone pays them as freely and cheerfully as they pay their dues to some club of which they are devoted members – where citizen and state can trust each other perfectly. We know our present society is a long way from such ideals, yet we should be wary of practices that take us ever further from them. One of the goals of moral education is to cultivate a conscience – the little voice inside telling us that we should do what is right because it is right. As surveillance becomes increasingly ubiquitous, however, the chances are reduced that conscience will ever be anything more than the little voice inside telling us that someone, somewhere, may be watching.
I’ve often thought about how heavily I lean on the crutch of vanity, simply to exercise and eat right. And I’ve wondered, “How would it change me if everyone could read my mind?” Not for the better.
Soon some big organizations will be able to read our minds. According to an old USA radio drama series, “Who knows what evil lurks in the hearts of men? The Shadow knows!” Soon we will really have such a Shadow.
The anti-moral, anti-creative force of groupthink will get more intense.
According to George Dyson
The ultimate goal of signals intelligence and analysis is to learn not only what is being said, and what is being done, but what is being thought. With the proliferation of search engines that directly track the links between individual human minds and the words, images, and ideas that both characterize and increasingly constitute their thoughts, this goal appears within reach at last. “But, how can the machine know what I think?” you ask. It does not need to know what you think — no more than one person ever really knows what another person thinks. A reasonable guess at what you are thinking is good enough.
The United States has established a coordinated system that links suspect individuals (only foreigners, of course, but that definition becomes fuzzy at times) to dangerous ideas, and, if the links and suspicions are strong enough, our drone fleet, deployed ever more widely, is authorized to execute a strike. This is only a primitive first step toward something else. Why kill possibly dangerous individuals (and the inevitable innocent bystanders) when it will soon become technically irresistible to exterminate the dangerous ideas themselves?
There is one problem — and it is the Decision Problem once again. It will never be entirely possible to systematically distinguish truly dangerous ideas from good ones that appear suspicious, without trying them out. Any formal system that is granted (or assumes) the absolute power to protect itself against dangerous ideas will of necessity also be defensive against original and creative thoughts. And, for both human beings individually and for human society collectively, that will be our loss. This is the fatal flaw in the ideal of a security state.
According to Tim Harford in Adapt: Why Success Always Starts with Failure
It isn’t right to expect a Mario Capecchi to risk his career on a life-saving idea because the rest of don’t want to take a chance.
The moral of the story is not that we should admire stubborn geniuses, although we should. It is that we shouldn’t require stubbornness as a quality in our geniuses. How many vital scientific or technological advances have foundered, not because their developers lacked insight, but because they simply didn’t have Mario Capecchi’s extraordinarily defiant character?
Jack Galvin also taught Petraeus that it is not enough to tolerate dissent: sometimes you have to demand it.