L'Abstraction, c'est la Cognition

I’ve long wanted to write down some of the thoughts that have come to me over these years of study, and I’ve finally found the time. Let me begin with the first wisps of thought that drifted into my head after I started university. In the first semester of my freshman year, because the digital logic professor at my own university taught so terribly, I went on Bilibili and found a digital circuits course taught by Prof. Wang Hong of Tsinghua University. To this day, I still consider it the most logically structured teaching I’ve ever come across, and the best at keeping hold of the main thread. The course’s central logic is this: a computer’s CPU is an extremely complex system, and understanding a complex system like that requires mastering the skill of “abstraction.” As another excellent textbook, which I later read nearly cover to cover, puts it:

The critical technique for managing complexity is abstraction: hiding details when they are not important.
— Chapter 1, Digital Design and Computer Architecture

Abstraction in Computer Science

What the course teaches is that computer architecture can be divided into many levels of abstraction. The lowest is the physical level: electrons and whatnot. The physical properties of these microscopic particles give us transistors, and from there analog and digital circuits, such as all the various AND, OR, and NOT gates. That already puts us at another level of abstraction. When we draw these logic gates on paper, what we’re always thinking is “How should these gates be used?” and not “Why do these gates work?” That’s because, at a higher level of abstraction, a complicated thing from the level below is treated as a single whole and “encapsulated,” after which we stop worrying about any of its details. And so on up the ladder: logic gates are used to build combinational and sequential circuits (such as the D flip-flop) that can express more abstract logical concepts, and those circuits are then used to build arithmetic units that can add, subtract, multiply, and divide, and so forth. Finally, at the highest logical level, we end up with a CPU capable of executing a whole set of abstract instructions. What made Prof. Wang Hong’s course so good is that she spelled out this way of thinking right at the start, and came back to it again and again as she went through each level of abstraction: “Whenever we get something complex, the first thing we think of is to encapsulate it and use it as a whole.”

By a happy coincidence, that same semester I was also taking another course: procedural programming (in Python). Luckily, the instructor, Marc Feeley, a veteran professor whose research is in programming languages, didn’t do what the dime-a-dozen Python cram courses out there do and rattle off one Python feature after another. Instead, he focused on functions and on the ideas of functional programming. At the heart of these ideas is function decomposition: when we want to implement some functionality, we should break the requirement down bit by bit. This actually corresponds to the division into abstraction levels in digital circuits. Concretely, to borrow the vocabulary of abstraction, the function we want sits at a very high level of abstraction, and what we need to do is design functions at lower levels of abstraction that implement this high-level function. By the same logic, those lower-level functions can in turn be implemented with functions at an even lower level. Eventually we reach the lowest level of abstraction, where all we have to implement are some very simple functions, like basic addition, subtraction, multiplication, and division, which we can call auxiliary functions. (Of course, a function at any level can serve as an auxiliary function for the functions at a higher level.) The functions at each level form a neatly encapsulated toolbox, just like the logic circuits at each level. When I use the level below from the level above, all I care about is the input and output of whatever the lower level implements; its internal implementation doesn’t matter. Likewise, when programmers who mostly just glue libraries together read the documentation, most of them only look at the inputs and outputs, and trust that the library has no bugs.

Abstraction/Grasping the Principal Contradiction vs. Cognition/Learning

So we see that, whether in digital circuits or in programming in general, the concept of abstraction takes center stage. Going further, this principle isn’t limited to computer science, nor even to the sciences. We should elevate it into a way of thinking that guides how we learn every discipline: a “science of sciences,” in other words, a philosophical idea. At the level of philosophy, this great idea is nothing new. Materialist dialectics has a similar formulation, namely: “grasp the principal contradiction.”

… in studying any complex process in which there are two or more contradictions, we must devote every effort to finding its principal contradiction. Once this principal contradiction is grasped, all problems can be readily solved. — Mao Zedong, “On Contradiction”

In “On Practice,” good old Mao goes further, pointing out that people come to understand and know the world gradually, by analyzing the contradictions within and between all sorts of things and by grasping the principal contradiction. I first came across these ideas, only half understanding them, in my high-school philosophy class. Back then their subtlety was lost on me, and I simply engraved that sentence into my mind by rote. But truth never fades. In the years that followed, the world I saw, lived through, and felt was like a primary-school teacher brandishing a pointer, tapping on those few words over and over, until I began to think about them and understand what they mean in real life.

The first step toward knowing and understanding the world is to abstract out some concepts. Picture a newborn in the first few years of life: every day, every one of their senses is bombarded with an ocean of information. For a baby with no ability to filter information (or, put differently, no cognitive ability), such saturation is undoubtedly an overload. Because, however amazing the brain may be, it can’t possibly record every detail of life exactly as it is; details are endless, just as a photo could have an arbitrarily high number of pixels. If babies couldn’t summarize and simplify these details before storing them away, they would inevitably end up utterly bewildered, like a student for whom the math teacher might as well be speaking Greek. I remember some hypotheses suggesting that the reason we have no memories from the very beginning of our lives is precisely that we haven’t yet acquired a language system. That’s because language is itself an abstraction of reality, and the ability to abstract is precisely what cognitive ability is. It’s an ability we humans have to work very hard to acquire as children. For example, as they grow, children only gradually come to understand that the concept “chair” covers anything with a backrest that’s meant for sitting on, whatever its shape, material, or color, while “stool” refers to those chairs without a backrest. Once they’ve mastered such concepts, children’s memories are no longer an animation made up of a jumble of pixel images, but concise little stories strung together from concepts, which are much easier to remember. It’s like how storing an image as a bitmap takes a lot of memory, while storing it as a vector graphic takes far less. Here’s another example of using concepts to remember a great deal of detail. Over the past two years of keeping a dream journal, I’ve found that although the dreams I have at night are easily forgotten during the day, if I jot down a few keywords from them, then even if I’ve forgotten the dream completely by daytime, seeing those keywords lets me recall its rich plot in full. All of the examples above reduce the amount of information the brain has to process, and thereby strengthen one’s cognitive ability, by the same method: summarizing what things have in common, grasping the principal contradiction. These reflections on abstraction vs. human cognition also offer another angle on certain methods in deep learning. In deep learning, for example, the attention mechanism can help a model learn to “grasp the principal contradiction” (credit to ke chen), that is, to focus more on the parts of the input relevant to the task and ignore the unimportant ones. In CNNs, the deeper the convolutional layer, the more the parts of the image that its feature maps show the model focusing on shift from fine details toward abstract semantic concepts. Representation learning, too, aims to build concise vector representations of complex inputs. (This one is debatable, since embeddings don’t seem to be any more concise than the raw input. I need to think about this some more. From this angle, existing embedding models should still have room for improvement.) The reason classification tasks can be used to train feature extractors is precisely that classification requires the model to be able to grasp the principal contradiction.

Let’s Abstract a Little Further

In the second semester of my freshman year, I took a course on object-oriented programming and learned about the concept of a “class.” I realized then that classes actually correspond to the various concepts we have in everyday life. For instance, I can create a class called “bird.” Its attributes then represent the principal contradiction(s) we care about. If, say, the scenario in which we use the “bird” class requires knowing the color of a bird’s feathers, we can define an attribute called “feather_color.” The beauty of object-oriented programming is that, by defining different attributes and methods, it can model how the principal contradiction shifts from one context to another. Going further, thanks to mechanisms such as superclasses, interfaces, and support for generics, object-oriented programming also provides different levels of abstraction, which makes it a powerful tool for “modeling” the world.

Much like the philosophy of object-oriented programming, the second step in “modeling” the world is to abstract, from the basic concepts we pick up early in life, concepts at higher levels of abstraction, and eventually to build up an abstraction hierarchy. This requires further analysis of the contradictions within these concepts. Say a toddler who’s just learning to talk spots a “pretty upperclassman with long, straight black hair” by the roadside, and Mom tells them it’s a “magpie.” The next day, the little one is captivated by a crow cawing on a pine branch. On the third day they meet a blackbird, on the fourth an azure-winged magpie, on the fifth a sparrow, and so on. At this point, Mom will tell them that these are all “birdies.” Perhaps in later conversations she’ll say to the child, “Look, there’s a birdie on that branch over there!” This completes one act of abstraction, from the concepts of specific kinds of birds to the broad concept of birds. Eventually, watching a documentary, when the child sees an albatross they’ve never seen before, they’ll exclaim without thinking, “Wow, that bird is huge!” At that point, they’re already able to use the abstracted concept of “bird” to make sense of things they don’t know.

Further abstraction can come through study. Middle-school biology, for example, teaches us that living things are classified by “kingdom, phylum, class, order, family, genus, species,” and birds are just a few of the leaves on this great logical tree. I have a friend (olgd) who is obsessed with biological classification; after learning this system, he could name the exact place in it of every plant around him. I now completely understand his obsession: it’s the enjoyment of a higher cognitive process. Another example is history. Once we’ve learned language, we can write the history of China down in a book. But can we abstract some further patterns on top of that? That is presumably the mission of historiography. For instance, the rise and fall of China’s successive dynasties show that most of them were destroyed because land annexation starved the central treasury of revenue, because local powers grew too big to rein in, or because the scholar-official class sold out the country for personal gain, and so on. The laws of physics are another example (thanks to the handsomest man in mtl for this one). That Newton could abstract F=ma from the way things move shows that mechanics, too, is a kind of abstraction. And mathematics is the abstraction of abstractions; I trust everyone will agree with that. In college, my school’s linear algebra professor wasn’t a great lecturer either, so I learned advanced algebra from Prof. Qiu Weisheng on Bilibili. What left a deep impression on me was his method for understanding linear algebra, which he called “dissecting the sparrow.” His basic framework of thought was to start from Euclidean space or simple examples from everyday life and guide students to abstract, on their own, the various concepts of more general linear spaces. As the saying goes, “the sparrow may be small, but it has all its vital organs”: analyzing one instance of a class can likewise help us understand the characteristics of that class. In deep learning, CV is harder to scale than NLP because a CV model first has to abstract semantic information out of the complex world of images; then, if we want the model to understand more complex semantic concepts, we presumably need more data to train its ability to abstract further on top of (the semantic concepts extracted from images). That’s a two-stage learning process, and clearly harder to train.

On the other hand, knowing and understanding the world does still require mental activities other than abstraction, such as logical reasoning, association, and so on. These activities can’t be entirely subsumed under the framework of “abstraction.” Association perhaps can: if two things are connected, it’s quite likely because they’re close to each other in the abstraction hierarchy they belong to. The magpie and the azure-winged magpie, for example, are connected in that they’re both birds. (I’m too lazy to look up their full kingdom-phylum-class-order-family-genus-species, haha.) Logical reasoning seems to be an ability entirely different from abstraction, but we should acknowledge that learning it first requires being able to abstract some concepts, so that we can then look for more relationships between them. Going further, reasoning between more abstract concepts tends to be more concise (debatable?).

The Secondary Contradictions We Can’t Ignore

Those are the insights I’ve gained into “grasping the principal contradiction” over the past few years. Readers who still remember their philosophy textbooks will surely not have forgotten its inseparable companion: “do not neglect the secondary contradictions.” The most typical example is actually the problem of modeling in research. We want to abstract a real-world problem into a mathematical model, then solve the problem using results about that model. But in the process of abstraction, we often make certain “assumptions.” For example, the most classic language model in NLP, the bag-of-words model, assumes $p (w_{1},w_{2},w_{3}) = p(w_{1})p(w_{2})p(w_{3})$, i.e., that each word occurs independently. Such an assumption simplifies the model, but it doesn’t reflect reality. In other words, it ignores a secondary contradiction that shouldn’t be ignored. In fact, language modeling should also take into account the context to the left and right of each word, as well as the order of the whole sequence. Masked language modeling and autoregressive language modeling, which came about once these issues were taken into account, are much more reasonable. And yet GPT, trained with autoregressive language modeling, isn’t perfect either, because language itself is an abstraction arrived at by ignoring many details of the real world (secondary contradictions). So perhaps multimodal language models with visual grounding could do a better job of filling in those secondary contradictions and modeling the world. I think that in research, constantly asking yourself questions like “What assumptions does this method make? Could it be improved if those assumptions were relaxed?” helps you think. Other examples: when we use a Taylor expansion, we decide how many orders to expand to based on the precision we need, and the neglected remainder term is then the secondary contradiction. When the precision requirement goes up, that remainder can no longer be ignored. When a programmer calls a function and runs into a bug inside it, they can’t just look at the input and output information the documentation provides; they have to dig into the function and debug it. At that point, the function’s implementation becomes a secondary contradiction that can’t be ignored.

Another example of secondary contradictions that can’t be ignored is the relationship between different levels of government. A higher-level government assigns tasks to a lower-level one; the lower-level government holds meetings to discuss them, carries them out, and then reports the results back up. The higher-level government can’t take care of everything itself, so it puts its energy into decision-making. In that situation, it has no choice but to assume that the lower level’s reports are trustworthy, and the details of implementation become, for the moment, a secondary contradiction in its eyes. (This suddenly reminds me of that meme: “You lot go handle these matters. I’ve got too much on my plate—I need to focus my energy on military affairs.” XP) If a lower-level government deceives a higher-level one, the higher-level government will have a hard time finding out. It can set up supervisory bodies to make sure the secondary contradictions are also carried out as perfectly as it imagines, but the supervisory apparatus is itself a hierarchy, and it too is prone to deceiving those above and hoodwinking those below. It’s a bit like how hard it is to trust the functions in a mountain of crappy legacy code. All this goes to show that abstraction really is an art, because attending to both the principal and the secondary contradictions at once is often impossible for lack of resources. As the saying goes, you can’t have both the fish and the bear’s paw. In such situations, bringing our subjective initiative into play to make a concrete analysis of concrete problems, and correctly and courageously identifying the principal and secondary contradictions of things under different conditions, is a precious ability—or perhaps, a strength of character.

The Justice of Abstraction in Everyday Life

Life itself is a complex system too. In this information age, with the technological frontier changing by the day, there always seems to be more news than you can read, more opportunities than you can count, more things to do than you can ever finish. No one can be perfect in every respect; if you insist on having it all, you usually end up stretched thin on every front. Perhaps we need to keep listening, at every moment, to our own inner voice: “What matters most to me right now?” If, after careful analysis, you arrive at an answer you’re sure of deep down, then go for it, no holding back!

I dedicate this essay to my Ph.D. life, which is about to begin.