How many variables can one person understand?
If you think you have a good grasp of all the data you read throughout the day, perhaps you should revise that belief. Neuroscience shows us how difficult it is for our brain to process information and what tricks it has developed to improve that ability.
The widespread use of computers, something that many of us have seen grow exponentially, has brought about a true revolution in the art of quantifying and recording data. In a world where so many events take place simultaneously, we have discovered since years the importance of collecting an infinite amount of data for analysis.
Organizations of all kinds devote a great deal of effort to monitoring their activity, to adjust decisions and processes. And public administration has the resources to collect and publish information about our society, which citizens can consult.
As a result of this obsession with quantitative data, any news or analysis leans on data, comparisons, graphs, and infographics. The question is: How much of this do we understand?
For more than fourteen years, I have belonged to a very peculiar tribe. Much of my work consists of immersing myself in hundreds of thousands of records on the state of a telecommunications network capable of serving more than 20 million customers. I look for patterns, shifts in trends, unforeseen relationships, and exceptions to the rule. From all this, you can learn and get clues on how to adjust designs and operations. The job of “data scientist” (which I am not) has been called the sexiest profession of the 21st century. Don’t believe all you hear. That is a profession for the brave. You have to kiss lots and lots of frogs to find a prince.
The first time I was faced with the challenge of making sense of several Megabytes of data on a new feature of our mobile network I asked myself a question: How many variables can a human being really handle? And reading around I found a statement that stuck with me. “The maximum number of variables a human being can handle is 7, plus-minus 2”. This comes to say that a human being can handle between 5 and 9 variables.
I am afraid that this statement, which can be easily found on the internet, is far from my experience. After years of studying and presenting information, I can assure you that most humans suffer horrible torment if you expose them to more than three variables at the same time. So I set out to investigate the origin of this phrase, to see what that magical value of seven referred to.
The magical Number 7
It didn’t take me long to find the source of almost all the studies on the human limitation to processing information. A magnificent article by cognitive psychologist George A. Miller compiling the results of multiple experiments on the subject, entitled “The magical number seven, plus or minus two. Some limits on our capacity for processing information”. Let’s see what we know about our capacity to process information.
Before we begin to see the results, we must consider the difficulty of studying something as rich and complex as our brain. The different experiments seek to analyze a particular characteristic or faculty in an entity (our brain) defined by its capacity to integrate all kinds of stimuli in parallel. As a result, many experiments are striking or very far from what would be a “normal” activity. That is the price of trying to study only one capability at a time.
In this case, we will start by studying the human capacity for “absolute judgment” about a feature. Put another way: how many levels we distinguish in a given particular stimulus or phenomenon. Don’t be scared by the title of the next section. It’s all about discerning levels of sound loudness or different pitches of sound, among other experiments.
Absolute judgments on one-dimensional stimuli.
As I have already mentioned, these experiments consisted of observing how many different levels human beings can distinguish when faced with a series of stimuli of the same type.
In some cases, the experiment was based on sound comprehension. After listening to a series of sounds of different tones (up to 14), on a successive scale, the listeners had to indicate how many different tones they had heard. The result was that people were able to differentiate 6 tones, no more.
Tests of this type were also done where the challenge was to distinguish levels of sound loudness. What we usually identify with the volume of a sound. Tests were done with 4, 5, 6, 6, 7, 10, and 20 different levels. The result was very similar to the previous experiment. Humans are not able to distinguish more than 5 volume levels in sound.
Other experiments focused on different senses. For example, taste sensitivity was tested using salt solutions in water, with different concentrations. Using 3,5,9, and 17 different concentration levels, it was evaluated how many levels the testers could distinguish. Only 4 levels.
The results with visual stimuli gave better values. For a species that receives 80% of its information from the environment through sight, this is reassuring. When the challenge is to mark areas on a line segment, humans are able to identify between 10 and 15 different positions.
Many more experiments have been carried on visual abilities, such as detecting levels of curvature, length, direction of lines, or estimation of areas. These experiments show an ability to discriminate categories between 3 (curvature) and 8 (length) levels.
Our brain cannot differentiate beyond 6 or 7 levels when categorizing data regarding a single dimension.
In any case, except for its skill in distinguishing segments in a line, our brain cannot differentiate beyond 6 or 7 levels when categorizing data concerning a single dimension. The next time you look at any classification, such as those offered by bar graphs or the levels to score a survey, remember this fact.
Absolute judgments about multidimensional stimuli.
There is major weakness in the test results regarding our ability to categorize from a single characteristic. If we can only distinguish 6 or 7 “types” in a set, how can we perfectly identify one person or any everyday object among hundreds of thousands?
The key lies in how artificial the experiments on a single characteristic seem, in contrast to our real world. We perceive multiple features of an object, and our judgment is formed by combining all of them. For this reason, we studied how categorization capacity improved when several characteristics were combined.
For sound, experimenters designed tests to combine variations in both the tone and the intensity. This way up to 9 different levels could be identified, instead of only 6. In the case of taste, samples with different salt and sugar content were tested simultaneously, and the result was to go from 4 levels to 5. As for positions in space, when going from 1 dimension to two, it improved again, differentiating 24 levels.
Below we summarize the results of using one or two features in those cases that are (roughly) comparable.
As can be seen from the results, the classification ability of our brain improves when using more features. Nevertheless, this improvement is not proportional to the increase in the number of properties we handle.
On the other hand, having more features improved the overall ability to distinguish categories but worsened the accuracy with which we can classify stimuli concerning a single variable.
It seems clear that our brains are better adapted to classify events or objects based on information from several features versus the challenge of doing so with the help of a single property. In the next section, I will try to explain why.
Our brains are better adapted to classify events or objects based on information from several features versus the challenge of doing so with the help of a single property.
A digression: the brain as an organ of adaptation to the environment.
Two preconceived (and erroneous) ideas have made it difficult to understand the brain as an organ subject to the same evolutionary pressures as the rest of the organism. In doing so, we have forgotten the importance of nature in those abilities in which our brain has specialized.
The first is the “computational theory of mind,” prevalent for many years in the 20th century. It describes our brain as a mere information processor that receives stimuli from the environment and elaborates responses according to algorithms.
Throughout the history of science, the brain has always been described in terms of the latest innovations in technology. In the time of Descartes, the brain was described as a machine. It was the time of the first automatons, and these rudimentary inventions astonished everybody. The 20th century insisted on seeing it as a computer. But this is not the case. The brain is fully embedded into a body that has to survive in an environment. Our brain’s decisive mission is to help us optimize our adaptation to that environment, as happens with any other organ in our body.
The second idea that blocks us is that the world has always been as we know it. In the history of humanity as a species, this era of abundance of food and scarcity of direct threats, of extended longevity and enjoyment of an environment modified to our convenience is very recent. And evolution has acted over millennia to prime the ability to survive in a very different environment.
When we ask ourselves what skills our brain has developed in preference, we must always bet on those that improve the ability to survive in a hostile environment, scarce of food, with more extreme climates, and that it was impossible to face without the help of our fellows.
The skills in which our brain excels will be those that improve the ability to survive in a hostile environment, scarce of food, with more extreme climates, and that was impossible to face without the help of our peers.
Natural selection has operated on us, as with any other species, giving priority to the traits that helped us to survive in these conditions. In the figure following, I try to represent how short the period of “civilization” has been compared to our entire evolutionary history. I hope this image will make it easier to understand that we are “cavemen in suits and shoes”.
And for a defenseless creature in a dangerous environment, it is more critical to quickly identify potential threats from little information about a handful of variables than to elucidate with high accuracy gradations within a single characteristic.
To give an example, identifying the roar of a saber-toothed tiger in the distance was a survival skill.
Distinguishing whether our tiger roared with a lower or higher pitch, whether it was a bass, a baritone, or a tenor… was not very helpful.
So our brain is good at categorizing from many traits, with a fair bit of information from each. But not so adept at distinguishing levels on a single characteristic. Just what you are looking for in most data analysis tasks.
Let’s talk about working Memory
But let’s not forget that, in addition to surviving in an environment full of threats, humans developed a complex social environment and unmatched manual skills. Facing these challenges required language and the mastery of categorization far beyond distinguishing items through a handful of characteristics.
In these more sophisticated tasks, our brain needed to develop additional techniques to categorization based on an isolated feature, such as:
- Increasing the number of variables through which to characterize an item (which we have already seen).
- Making relative judgments, and not only absolute ones. Humans are much more sensitive to changes than to absolute values. Marketing experts take this into account when pricing different products within a range.
- Organize the analysis so that we can chain successive judgments. To execute data analysis in several rounds, storing the result of each phase to be used as input information for the next one.
This last technique is very remarkable, as it introduces Memory as a crucial element in the ability to discriminate. Despite its bad reputation, Memory is an essential part of our intelligence, and this example is just one of the facets in which its use is fundamental.
Working Memory and Recoding
We can define working memory as the memory we use to “keep in mind” the data we need for immediate use. Some authors consider it equivalent to short-term memory while others think they are different, attributing to working memory a greater degree of awareness on our part.
Experiments to measure working memory capacity are somewhat similar to those that measure categorization capacity. The people participating in the trial received a series of stimuli in succession. Instead of answering how they would categorize them immediately after perceiving the event, they should report all the stimuli received (sounds, spoken words, or letters, for example) at the end.
With all this, scientists have estimated the capacity of our working memory in about 7 “chunks” of Information. That is what our brain is able to discriminate in a number of 7 plus-minus 2. The “blocks” of Information that our working memory can in its center of attention at any given time.
The capacity of our working memory is about 7 “chunks” of Information. That is where the magic number of 7 comes from, plus-minus 2.
There is much to be said about these chunks of information. Since Miller’s time, there have been multiple experiments to elucidate how many there are, and how much information they can hold. The basic idea is that the limit is in the total amount of information we can store. We can retain more chunks in memory if they contain uncomplicated data, and fewer if they are more complex pieces of information. However, this statement is not easy to prove.
Somehow, our brain can train itself to add more and more information in those blocks, associating more characteristics. What in psychology is known as “recoding” is the key to the learning process and to what we usually call experience: grouping relevant information about an element in such a way that it is consolidated as a whole that our brain handles and combines.
As a simple example of this Recoding we can cite the memorization of sequences of numbers: In experiments on this ability, subjects developed tricks, such as grouping pairs or trios of numbers to create a new mental object that only had meaning within their mind. That allowed them to memorize many more numbers, tactually managing the same number of memory blocks in their working memory.
As we become familiar with a set of data, we can mentally construct more complex aggregates of information. Those richer aggregates will constitute the blocks of our memory, and we will be able to “play” with them to reach conclusions. As the chunks become more enhanced, we can readily judge results, detect significant behaviors or merely understand them. Training over a defined dataset makes us able to handle and understand more information. To master and understand (this very time) more variables.
In Essence: What is our capacity to understand information?
For more than a century the capacity of the human brain to process information has been studied to identify its limits and determine its underlying cause.
According to these experiments, it seems clear that we can discriminate between 5 and 7 levels when dealing with a single characteristic of an object.
On the other hand, our working memory limits the number of blocks of information we can handle at once to about 7. The magic number 7 appears everywhere in these measurements of the limits of our information handling capacity.
However, we know that we can go far beyond these limits thanks to experience. Our brain was adapted to make us survive in a hostile environment where we had to categorize threats and opportunities by combining little information from many parameters, which makes us a bit bad at discriminating levels of a single variable.
But it is also true that the brain has remarkable plasticity, which allows us to extend our abilities through training. No doubt, a professional musician is able to distinguish many more tones of sounds than 5 or 6 levels. Our brain re-train itself to go far beyond our general, innate ability.
In addition, the development of hunting and manual skills, which are critical to our intelligence, required great gifts for discriminating distances at different scales. So when it comes to establishing stretches in a linear segment, our discrimination ability is much better. We can distinguish between 10 and 15 different levels, and in 2 dimensions this improves to identifying 24 positions.
Finally, the mastery of language and other complex skills demanded a much more expanded capacity for comprehending information, increasingly more elaborate. Here memory and recoding were essential.
A journey into the process of understanding complex information.
Faced with the novel challenge of understanding a set of data that we know nothing about and have no direct experience with, we somehow go back to the origins: we will try to group them into sets with 6–7 different levels. That is our elementary ability to categorize.
If we need to go deeper, we will have to gradually get used to the data to create new blocks of information in our working memory. These blocks will be increasingly more rich in content and more adapted to the purpose of our analysis.
The blocks of information generated by our brain allow us to integrate various features and examine them as a whole, and the result of these analyses generates new, richer, and more complex blocks of information to work over. As you can see, the process is iterative and progressive. Its result can be called “Experience” or “Knowledge” depending on what we want to emphasize.
As we become accustomed to working with data sets relating to a given phenomenon, we are able to quickly identify patterns and exceptions where other people literally “see” nothing. Sort of like when any of us, not being radiologists, try to analyze an ultrasound or another imaging test.
In short: obtaining relevant information from piles of data is only possible after having mentally “amassed” it for many hours. And by this, I do not mean magic algorithms or supercomputers. I mean understanding and internalizing the relevant features of the phenomenon as the caveman recognized the roar of the tiger. That requires “touching” the data. Drawing tables of numbers, aggregating them in different ways, painting them with different types of diagrams. That is how our brain learns to find the most noticeable features.
As you can see, it is not a simple process. So no, regarding the data about which we read every day, we don’t really understand much. Because to understand them, we would need several lifetimes.
Our civilized world poses a challenge of assimilation of information infinitely more complicated than the environment in which our species has developed. The amount of information we need to understand our world has grown as fast as our means to assimilate it, so we are like our ancestors. We try to survive by obtaining conclusions from a few clues. Just enough to detect when the tiger is approaching… and act accordingly.
