Slow Fourier Transform

Having wasted a lot of today and yesterday reading historical documents to let my background thought processes swap out and then back in — the Unix Haters Handbook, a few Wikipedia pages and some Jargon file — I’ve come to an unexpected conclusion that the category of “relevance” needs a serious reevaluation… Not the word, the notion itself, as applied to world at large, rather than to individual objects.

First, here’s the definition for the purpose of this discussion: Something is “relevant”, when it’s importance for the whole world¹ cannot be reasonably denied — therefore it must be considered seriously when making any decisions that might be affected by it’s existance. Obviously, varying degrees of relevance may be defined, even if they can’t be objectively measured. USSR was “relevant” when it existed, and it’s component countries quickly much reduced in relevance, no longer being taken seriously.

Previously I mentioned here that quite a lot of what we see on TV — most things related to politics, most things related to so-called “mainstream” art, and even many things related to economics, as it is presented in the news, are in fact irrelevant. I’ve also commented that the real power will be in the hands of those who find what is actually relevant in the current situation.

Reading historical documents I’ve noticed that a whole geological layer of culture — people who used command line shells not because they knew them well and liked them, but because they had nothing else, as well as true hackers which liked command line for what it was — is now lost completely. Twenty and thirty years ago these people also thought what they were doing — yearning for better-designed operating systems of a previous era, or touting failed examples of the next one — was relevant. What was actually relevant and changed the industry were six people in jeans who created NeXT. But NeXT never had a huge commercial success, right? What changed more was the open source movement. And yet in 1992 pretty much nobody knew that Linux was out there already.

In this gargantuan mess of computing, it appears that everyone is basing their opinions of relevance of other people and ideas on the experiences which end some five years previous, at most. It still exists completely parallel the world on TV, with which they struggle to be mutually irrelevant. The human reality fractures along the lines of mutual irrelevance, wherever the fragments actually clash, strange things happen.

It is pretty obvious from that, that there can’t be a general purpose “relevance” category. Social groups will fold their perceptual reality along the lines of what they consider irrelevant. It’s also pretty obvious that the so-called “real world” doesn’t matter to any of these groups as long as they can afford to ignore it completely, but I’ve been saying that for as long as I’ve been teaching… Now the million dollar questions for today are:

1. It can be readily observed that shifts in relevance can be inevitable — for example, UNIX Wars allowed Windows NT to enter corporate market, which allowed a turnaround when Linux matured just a few years later, which is what we’re seeing now. Is there a way in which they could be predicted back in 1992?
2. Is it possible to study the “reality fractures” and determine how large a “reality shard” is, and what’s it’s growth potential? How is it possible, when close study implies intimate involvement with it?
3. …most importantly, has someone else made sense of this mess yet, and does anyone actually consider thinking about culture above the “age-gender-income-education-nationality” level? Are there serious quantitative approaches to studying multiple intermixed cultures on a global scale?

1. …of humans, mind you — on the galactic scale most of non-cosmic objects are pretty irrelevant by definition until we go FTL, and even then, probably not for at least a few thousands of years — and even that’d be optimistic.↩

I’ve been reading up on 9/11 attacks and a quote crossed my search: “…and FBI didn’t know what it did know.” — that was in a treatise on social epistemology. That led to an interesting observation.

Let’s assume that “FBI knows that X is a terrorist.” is true. What does this expression actually mean? Most likely, it means that a record that says that “X is a terrorist”, presumably giving some reasons why X can be classed is such, exists somewhere in FBI files.

But “exists in the files” does not necessarily entail “is used in all practical decisions” right? Databases exist specifically because the amount of data stored is too large to track with any single person’s mind. Therefore we must assume, that the statement “FBI knows that X is a terrorist” must entail that “some people in FBI know that X is a terrorist”, but not all. In practice, it might turn out that none of the people in FBI actually know that, because, for example, the people who entered this fact into records have since left office and are no longer members of FBI, while nobody had the time or reason to look up that particular file since.

This results in one more interesting conclusion — that at any given time, a very large set of data is completely absent from humanity’s “scope”, so to say, and is only present in written records, but not actually considered in making decisions. This is somewhat of a stretch, since we might assume that with 6.5*10⁹ people on Earth, some information from the collected store of human knowledge would definitely end up in someone’s head, but that is most likely not true, since to function as an organized society, humans have to share (and that means duplicate) a large set of knowledge just to ensure compatibility, and the size of readily accessible human memory is limited. And let’s not forget that fairly often, people will ignore a fact that is indeed stored in their own memory in making a specific decision — because no associative track between it and the decision exists.

Which means that a piece of information may be discovered multiple times in many different fashions, and will, each time, be effectively “new”. There’s probably a set of interesting consequences to the mechanism of “news” generation in the media that result from that conclusion.

Gotta be a propagation model out there somewhere already, but finding it would be “new” for me. :)