Intellectual Exoskeletons — Andy Matuschak (#141)

From language and writing to the Hindu-Arabic numeral system, computers and Adobe Photoshop, our species has a history of inventing tools for augmenting our own intelligence. But what comes next?

Andy Matuschak is a developer and designer. He helped build iOS at Apple, founded and led Khan Academy's R&D lab, and now works as an independent researcher investigating tools for thought — that is, technologies that can transform human cognition and creativity.

Transcript

JOE WALKER: Andy Matuschak, welcome to The Jolly Swagman Podcast.

ANDY MATUSCHAK: Hey, thanks so much for having me.

WALKER: Andy, it's so great to speak with you. I found your work last year through a friend. In fact, I think a mutual friend of ours, Peter Hartree. And discovering you was kind of like being unplugged from the matrix. It was one of the most important intellectual events of 2021 for me.

MATUSCHAK: Wow. That's very flattering.

WALKER: Well I think you're doing some really interesting and important work, and I'm so thrilled to have the opportunity to chat with you and share your insights and your ideas with my audience. So thank you so much for joining me. I want to cover a bunch of different topics: some first principles related to the kind of problems that you are solving; learn a little bit more about you and your background; and then talk about tools for thought, and your own note taking system and other tools that you use on a daily basis. I thought perhaps we could start with your background, and then go into first principles. So, as a kid, you spent much of your time making video games. And then as an early teenager, you worked on an app for making art for video games. And then you joined Apple and, again, you were making tools which could help make apps for making art for video games. So what is your favourite video game of all time and why?

MATUSCHAK: I think if you were to ask me today, my answer would be The Witness by Jonathan Blow, which is a very beautiful game that's basically about discovery. It's about insight, it's about epiphany. And one of the things that's so striking about it to me, and just really inspiring as a designer, is that the game includes basically no written or spoken language. It is a lengthy game, maybe 70 or 80 hours, and you're learning all these very complex mechanisms in this very unusual environment. And yet you're doing this without anyone really telling you anything explicitly. And so this game is really inspirational for me as I think about human learning, because such complex things are being learned by people here without language. And so maybe things like that are possible outside of the game context.

WALKER: You studied at Caltech. You studied computer science. And while you were there, I understand you were involved in updating Caltech computer science curriculum prior to it are coming as popular as it is today. What did you change about the curriculum and why, and how did you find yourself in that position to begin with?

MATUSCHAK: Well, I should couch that by saying that I doubt that what I did contributed to it being popular today. I think that's part of just a broad trend. I was in the right place at the right time. I really focused on the first year year. I was a senior at the time and there were two problems that we wanted to solve. One of them was that Caltech is very focused on theory. And so its students spent a lot of time studying math and proving things. And this was really wonderful. So when people left, they could think about problem solving and software in a really principled way. But often they couldn't actually build software. So there was an introductory course that was sort of designed to help people do that.

But it needed a lot of help. So with a couple of a couple of colleagues, we sort of retooled this around the way that people were really building software today. And I felt like this was a lovely opportunity for me to apply some of what I'd learned doing a lot of building to this very like theoretical context at Caltech, very math-centric. And the other thing we were trying to do was: Caltech has this very unusual setup where even if you're a geologist or something like that, you're gonna study quantum physics, you're gonna study chemistry and you're gonna study biology. Everybody for the first two years—at least this was the case when I was there—studies roughly the same thing. And for most of the majors that includes computer science.

And so that created a practical problem that you have a bunch of people who are going to make computer science their field, in the same class with somebody who's gonna be like a chemist, who studied computer science because everyone is kind of expected to know how to think computationally. And so we kind of broke out the first subjects for that very first introductory class, so that there was something that would be more suitable for all scientists to take. And then something else: that they'll be a little more specialised to computer scientists. There's a very classic computer science text called, Structure and Interpretation of Computer Programs. That's where we moved that text to for the computer scientists.

WALKER: When did you discover the work of David Deutsch and how did it influence your world view and the trajectory of your life?

MATUSCHAK: David Deutsch substantially changed the trajectory of my life actually. I discovered him through a friend of mine Mills Baker, who has a really lovely blog full of wonderful essays online. And he sent me this email that said, "Andy, this is incredibly urgent. I need to send you a book today, because I think this book solves basically all of the problems that you and I have been discussing—politically, artistically, philosophically." We were obviously very enthusiastic at first. Of course that enthusiasm is tempered a little bit by distance, but at the time this book was really the right place at the right time. For me, it contained a lot of fundamental answers to questions of purpose, meaning, power, the role of the self, that were very novel to me, having not very rigorously studied a certain branches of philosophy previously.

There are a number of ways in which it really substantially changed my thinking, but probably the most powerful are these messages that are carved on stone tablets introduced early in the book. The message is that if a problem's solution is not forbidden by the laws of physics, then there is a solution; we just don't know it yet. And the second tablet says: and there will always be problems. So we don't get to get out of it. Stasis is not a solution. And yet, all these things, which seem so intractable or which seem like the status quo we were born into and therefore they will always be that way: they don't have to be that way. And it sounds very trivial when stated so baldly like that, but then the book goes on for another 800 pages, you know, talking through the consequences of this.

At the time, I was working at Apple and having a very rewarding time, building things with a great deal of craft and trying to make the back of the cabinet extremely polished and beautiful even though no one would see it. This really expanded my sense of what I should consider for my career. And I started interrogating: "Well, if all problems are either forbidden by the laws of physics to be solved, or in fact are solvable, then what problems should I be working on?" And that made working at Apple basically feel impossible. It made me temporarily quite unhappy. I woke up and I got on the bus to go to Apple and I felt like, "What am I doing? What am I doing here? This is of no cosmic significance whatsoever." Now, in retrospect, I think that reaction was unhealthy. I think insisting on that kind of cosmic, eternal purpose is misguided. But at the time it really shoved me onto a different track.

WALKER: I assume the book you're talking about here is The Beginning of Infinity.

MATUSCHAK: It is.

WALKER: Have you read The Fabric of Reality as well?

MATUSCHAK: Yes, absolutely.

WALKER: Do you have an opinion as to which of the two is the better book?

MATUSCHAK: Well, they address related but fairly different topics. The Fabric of Reality was for me somewhat more abstract. It made these really fascinating claims about the nature of the universe that, again, just based on my prior reading, were very unusual and novel. And so I found that book really edifying. And to this day, it has changed the way that I think about questions like free will, determinism, destiny, but The Beginning of Infinity felt much more personally relevant. It really spoke to me at a teleological level. "What is the good? What can I, should I, be doing?"

WALKER: So I'm going to glide over the Apple years and jump to Khan Academy. At Khan Academy you founded and led the R&D lab. And I'm curious, what were the most important things you learned about how to lead teams successfully during your time at Khan Academy?

MATUSCHAK: Right. Well I should begin by saying that I think I wish I'd internalised these things a lot sooner. Certainly I could have been a better leader at the time. Let me try to give some answers that aren't so cliched — a lot of the things that I learned are I think the things that everybody learns when they're thrown into a really difficult leadership situation for the first time. An unusual thing that I learned really has to do with the creation, and governance, of a culture of taste. I think this is a really challenging thing in organizations. Taste is subtle and there isn't one right answer either. If you're doing research, there's a certain degree of, say, depthiness and polish that one should insist on, and there's places where you wanna be scrappy. And then if you're doing production work, there are different answers to that question.

And so when we look at companies like Apple or Stripe or Peloton, and we compare their work to many other companies, one reaction many people have is like, "Wow, this is a company that maybe ships really consistently highly polished stuff." And it's very difficult on the ground to make that happen. There are so many people contributing to the product, and there's so much contingency. Each individual has so much of the product's destiny in their hand. In a small way, often it'll just be the polish at a particular corner, or the thoughtfulness of a particular interaction. Command and control can't make it all work. And so I learned how to govern this through some trial and error and also through some of my colleagues at Khan Academy: Ben Kamens, who led the engineering wing, and May-Li Khoe, who came over with me from Apple and led design.

One thing that really worked was being quite conspicuous in setting an example, and this requires being a bit of a player-coach. It also requires some delicacy because, taken too far, this can turn into like, "Well, you know, the CEO or the director or whatever, they're the only ones who can do it the right way, and no one else is allowed to do it the right way." So I think there's a negative interpretation of this. But a positive interpretation is something where when there's a particular methodology or practice that you want to see appreciated, you conspicuously demonstrate it in a way that does meaningfully contribute. And you highlight very consistently and overtly and genuinely instances in which others on the team manage to achieve that particular practice or that facet of taste.

And this isn't just like good versus bad. It's often things like for teaching and students. It's very tempting to basically have an authoritarian relationship with students. Much of the school system is structured around this. And so when thinking about designing instructional material or tools that can help students, it's very tempting to talk about doing things to students or making students learn a thing. This is really misguided. And so even in our speech and in our way of thinking, we want to be thinking about student as the agents, and we want to be thinking about their goals. So it was very important that I and other leaders conspicuously modelled that as much as possible and highlight and congratulate others who are thinking in this very student-centered way, rather than in kind of an authoritarian way. That's just one example. I don't think I've gotten that completely right, but I think this question of how to sustain a particular kind of taste in an institution is just so interesting.

WALKER: I think what you're referring to there is while you're at Khan Academy you implemented this cultural hack which was to always make students the most important subject of your sentences. So instead of saying "We're teaching students X", you would change the sentence to "We're helping students to learn X". And that little grammatical tweak was a really profound kind of shift in perspective that helped your team to constantly be thinking about how they could enable students to be their best rather than dictating knowledge to them in an authoritarian manner. What was the outcome there? Did that little grammatical tweak turn out to have important consequences?

MATUSCHAK: Sure. Thank you. Thank you for highlighting that. That's a very specific story, and it's one that works worked very well. It's basically just a reaction. May-Li and I noticed that we were often talking at cross-aims to other designers or engineers who had somewhat of a more authoritarian relationship, not with negative intent, it was just kind of automatic. And we had trouble putting our finger on, like, what is this difference? It wasn't that we always talked in one way, with the students as the subjects of the sentence, and they always talked in the other way, but we realised that at the heart of it, that was the difference: thinking about who was the subject and who was the object. And we found that, by speaking in this other way where the students were the subject, it would influence people.

And when repeating people's ideas back, we would often rephrase them in this way. So someone would say, "Yeah, we're gonna make the students understand this particular law by having them do this exercise." And we would say, "Okay, so your plan is to make it possible for students to realise that there's this relationship by you creating this particular context." And repeating that back actually really did change people's at least speech patterns, and I think thought pattern too. Over the course of the time we were there, we saw really substantial differences in the culture moving from a fairly instructionalist perspective to, somewhat more, we might say, constructivist perspective, where the learner's doing more of the constructing of the ideas. And of course this particular hack was one of activities that pushed things this way. But I think these kinds of very intentional modeling behaviors really do help.

WALKER: I'd love to talk about some first principles now just to lay some foundations for the rest of the conversation. This next question I actually take from the book Understanding By Design, by Wiggins and McTighe, which I discovered through reading your work. And the question is this, what is understanding and how does it differ from knowledge?

MATUSCHAK: Right. Well, if you've read this book, you've encountered the fact that there are many different definitions of understanding, but the working one that I like to use is adapted from Dewey and that's that a person understands something when they can flexibly and fluidly apply what they know in a variety of contexts, through synthesis, procedural means, creating something new, judging something, making connections. It's really that flexibility and the fluidity that's important, as well as the variety of contexts that's important. So, we have transfer to a number of domains. I think that's what characterises the difference between understanding and knowing, at least for me.

WALKER: And when you say "transferring it to a number of different domains", how would you in practice actually know when you've genuinely understood something?

MATUSCHAK: Right. Well, I don't think there's a binary line. I think there's never a moment when you say "Aha, now I understand it. And before I didn't, and there's no more understanding to be done." I think instead it's a little bit more like you're on a hike through craggy terrain and you pass over a ridge and you see this kind of frontier that you couldn't see before, and then of course you see a higher ridge in front of you. So when and how does that happen? You need to be able to recognise the applicability of knowledge in this other domain. That tends to require abstracting the knowledge to a greater degree. So it's not just that, you know, this particular bird has this particular colour of feathers.

But rather you start to understand that, like, birds which have these feeding behaviours have feathers which work in this way, and that allows you to notice that "Ah, because the trees here are shaped in this way, we should expect to see birds with these properties". So abstraction helps. Lowering other kinds of metacognitive load helps. So for instance, being in a situation where you aren't being really taxed by what's going on in this new domain will make it easier for you to apply stuff that you know from the old domain. And there's kind of a number of properties like that. And your ability to understand and apply things in these new domains will vary based on the situation. So if you go to a new domain, and you're having a really difficult interaction with a colleague in this new domain, then you may find yourself unable to apply the things that you knew from some other domain. You might not see a connection that you otherwise might see because, yes, some part of your metacognition is occupied with governing this relationship.

WALKER: Got it. What do we currently know about how human beings store long-term memories?

MATUSCHAK: Sure. There's a couple of ways to approach this. There's a functional way we can describe it: we can say "Well, when we sit people in rooms and we have them do tasks and talk to them, we notice these patterns. So that's like one kind of thing I can describe." And then there's the physiology, there's how actually chemically and biologically is it encoded. And we've made some progress in both of those things. So in terms of long-term memory specifically, we understand for instance that there's a difference between what we call semantic memory — and that's like, you know, knowing that a toucans feathers are a particular colour — and episodic memory, which is more like a personal history.

It's a little bit like a movie in your mind. It's what allows you to play back experiences vividly. So those are stored in different places, for instance, and damage to different spots in the brain will cause different effects on those things. We also know that memories can reinforce and harm each other in the long term — or let's say inhibit each other. So for instance, if you go to the same place and do exactly the same thing many, many times, then details of what happened the second time you went to that place will actually be eroded by the subsequent memories that you formed in that place. These are kind of these inhibitory things that happen.

So there's sort of like an averaging out that happens over time. But there's also a reinforcement that's possible. For instance, if you learn a particular thing and then you encounter it in another context and in yet another context, and then it connects to something very deeply, personally meaningful, then now this same thing is encoded in a lot of different different ways, and you'll find it easier to retrieve. There's a whole bunch of a phenomena like this we can describe functionally. So for instance, if we ask you to relate things that you're learning to yourself very personally, like give you a word list and ask for each of the words, "Does this word apply to you? Does it describe you?", then you'll remember that word better, because we're narcissistic in this interesting way.

If I ask you, for each word I'm having you learn, to form an image, a really vivid visual image, then you'll remember it better. So there are these kind of long lists of micro effects that improve memory. It's very difficult to add all these up into some kind of very simplistic unifying theory, like a theory of gravitation. And this brings us to the physiology: how are things actually physically encoded? This is not a topic that I've studied in a great deal of detail. My understanding is that things are encoded both locally and in a highly distributed fashions. So for instance, sometimes we'll find that there are like very specific neurons which activate very precisely and in very specific settings.

And then other times we'll find that your representation of a bus is distributed over some massive swath of your brain. There are some durable chemical changes; sometimes these things are actually observable, either by fMRI or other means. There are consistent regions of the brain that participate in these things. And so if damage is to these regions of the brain, then we have difficulty making memory. We also know that there's physical growth involved, plasticity involved. There's a classic study of London cabby drivers that discovers that they end up with a great deal more white matter in their brain than the regular population has. And actually a related study followed these cab drivers after retirement and found that many years after retirement the retired cab drivers have like more than the general population but less than the active cab drivers.

And so this tells us something about the time dynamics of these things. And there are similar things found, for instance, for pianists, by number of hours spent in childhood practicing. There's ones for mathematicians — the number of hours spent doing math problems when they're kids correlate with the amount of grey matter in another region. And so we have these very like coarse associations. I think what one wants often is something a little bit more like a theory of gravitation. And we don't really have that. We have some very broad rules of thumb that I can describe — and that the systems that maybe that we'll talk about take advantage of — but these feel much higher-level than everything I'm describing so far.

WALKER: If you had the choice between your current memory and having perfect memory — so every detail that ever occurred to you, you stored in high fidelity and could retrieve it, it was always at your fingertips —, would you choose perfect memory, or would that just be an absolute curse?

MATUSCHAK: Well, it's so hard to imagine, isn't it? It's difficult to even get my head around. One thing I feel is a kind of humility. I can't imagine what that would be like. And so it makes me a little scared. There are all of these stories of savants who do have these kinds of characteristics, and it's difficult to find stories of this kind where the savants have a happy life and a happy ending. So that's a little discouraging; just empirically I think we should probably be wary of such a thing. If it's a permanent change and I can't go back, I think I'd probably not take it actually. More just fear of the unknown. It feels a little bit like someone offering some kind of dose-unknown, effects-unknown psychedelic and saying, "Would you like to take this?" "Geez, I don't know."

WALKER: In a way it's kind of like Thomas Nagle's question, "What is it like to be a bat?" It's almost inconceivable.

MATUSCHAK: Would you take it?

WALKER: I have a similar reaction to you. I think, probably not. Although I'd be pretty curious.

MATUSCHAK: Yeah, absolutely. So one thing that's interesting — we'll probably discuss this more — is that I've done a bunch of things to make my memory a lot stronger over the last few years. And that has just been, as far as I can tell, an unalloyed good. I had feared for instance, that like, "Oh, maybe this will make my memory for other things worse," or, "I will regret remembering all this stuff in great detail," and nope, as far as I can tell there's no downside. So clearly you can crank up memory a great deal and probably way, way more than I've done before reaching some of these negative effects.

WALKER: How much better do you think your life is now as a result of having cranked up your memory? Is it three 3x better, 10x better, less than 1x better?

MATUSCHAK: You know, it's so subtle. It's so hard to talk about. There are these things that you could do quantitatively. You could have me read a book and give me a test on the contents of the book and compare that to someone else. And it wouldn't be a 10x higher score. I would know this stuff more reliably, that's true. But what is the impact on my life? I tend to think it has the most impact at these interesting threshold moments where actually a few percentage points better performance leads to a really non-linear difference. When one is already at the top of some field, some competition bracket or something, maybe you're already in the 99th percent of weird designers working in this particular domain, like I am, small amounts of additional benefit can lead to really enormous consequences. One interesting application is just creativity and the role of memory and creativity. In order to notice a connection or notice a coincidence — notice a contradiction is another common kind of creative recipe: notice something that you observe that doesn't add up with something that you'd previously learned — in order to notice all of these things, you can't have to go and look the thing up. To some degree, you have to have the thing in mind already. So expanding the repertoire of things that I can make creative connections to has been really helpful for me. And at least relative to my previous life, it's done a lot of good.

WALKER: I think that's a really profound point because I'm conscious that, that some people may have a visceral reaction to this notion of improving memory. They might think that it's just not desirable, because why does that matter? I can just go and Google stuff. Surely deeply understanding something, having it sink into your bones, being creative, being able to find unique connections between things, is more important. But the point you just made is that actually having all of these disparate facts and ideas at your fingertips enables you to make those unique connections.

MATUSCHAK: Yeah, that's right. I don't want to argue that memories of panacea here, but I think it is really important to interrogate when we talk about knowing something or understanding something or creating something, what actually do we think is happening mechanically? To some extent, if you learn a very complex piano piece or something thing, what's happening physiologically is a change in memory. And it may not be a change that you can just snap your fingers and cause to happen. To some extent, if you learn some complex math idea, that is a kind of change in memory. Maybe this is just a pathological use of the word, but I find it helpful to interrogate what's happening creatively.

What do my creative insights actually depend on? And one of the things I notice is that it usually kind of does depend on my working set, so to speak. There's a related, really important phenomenon here that's often called chunking — an unpleasant sounding name. But the observation is applied in a number of domains. One really classic example is chess. There's this wonderful classic experiment where chess grandmasters and novices are given a chess board that's all set up and they're given another blank chess board and a set of pieces. And they're asked to make a copy of the set-up chess board on the blank chess board. And the really interesting thing is that the grand masters are able to just do it in a glance.

They look at the fully setup chess board and they don't have to look again. It's set up so that glancing is expensive. You have to like look back and forth across this partition. So they look once and then they can just go and set up the chess board on the other side, whereas the novices have to go back and forth several times. Their working memory can't hold all the pieces. There's a lot of different pieces on the chessboard — 32 pieces maximally. What the grandmas are doing, they figured out through a whole lot of careful experimentation, is they're not storing: "Ah, this pawn is here and that rook is there." They're storing these higher order phenomena like: "This part of the board looks like this very famous game in a way that I remember, and there's this line of force over here, and the king is threatened in this way by this structure." And so they're remembering in some sense the same number of details as the novice is remembering in their working memory but those details are pointing to these much richer representations. Those representations exist in memory. So building memory is also about building these kinds of rich abstractions.

WALKER: Is there anything else you'd say about how you think about the relationship between memory and creativity?

MATUSCHAK: Yeah. I could talk about this for two hours, but I'll just share one of the other thing that's interesting. The specific mechanism that I mostly use to reinforce my memory is called retrieval practice. This involves basically challenging myself to remember something on some kind of a regular basis. This impacts my memory, but it also does something else, which is that it allows me to maintain my attention on a particular topic in some kind of consistent way over time. Previously, if I read a paper that strikes me as interesting in that moment then I put it down and I go away, my relationship to that paper is over. But by doing this interesting kind of memory practice, there's a non-memory effect, which is that I stay in contact with that paper, emotionally, creatively. And it kind of percolates into my life over this longer timeframe. It has more chances to connect with stuff and it changes my relationship to it.

WALKER: I think Brad DeLong called this secular catechism, right?

MATUSCHAK: Yes, exactly.

WALKER: Which is a really neat description. To make this more concrete through an example, say you are studying a topic that is slightly outside your field of expertise. Maybe you're learning about quantum mechanics or economics or something. By continually having contact with that topic over time through retrieval practice, you start to subtly influence your identity. Maybe you start to think, "I am someone who's interested in economics or quantum mechanics." And then maybe, "I am an amateur economist or an amateur physicist." Is that the idea?

MATUSCHAK: Yes. With some of the research that I've been doing, people have said as much. These participants have said that instead of just feeling that "I read a book about X", it's now a feeling like, "Oh, I am a student of X". There's this interesting translation. There's another interesting and related effect, which is that there are these kinds of things that you learn that you want to remember in some sense, but it's more like you want to carry them with you like a little charm or a jewel, like somebody tells you something really profound that changes the way you relate to something. There's a quote from a researcher of addiction that really struck me. He said, "It's hard to get enough of something that almost works."

I was really struck by this. Thankfully I don't suffer from any kind of the addictions that he was researching, but I was really struck by it in the context of needing to achieve or perform for others and having a very strong drive to that and feeling bad if I didn't. And it's hard to get enough of that, because it only almost works. You impress people, get their attention and it kind of fritters away. It's not the kind of acceptance or belonging one might want. So that's a very profound insight. In this kind of catechism style, I can tuck that into my pocket and bring myself back to that insight as well as allow it to apply to whatever moment I'm in now by engaging something that's kind of like retrieval practice. It's kind of like reinforcing my memory, but here the point isn't really to make sure I remember that phrase. I'm going to remember it. It's more to have repeated exposure and lengthen its time exposure.

WALKER: So how's that for first principles? Are there are any big things we haven't covered off yet before we move on to tools of thought?

MATUSCHAK: I mean there's so many things we could discuss, it's hard to say. In order to discuss some of the specific tools for thought I've been developing, we'll probably have to talk about the role of retrieval practice and reinforcement or something like that at some point here. But we can also do that like in-line.

WALKER: Let's do it in-line. So tools for thought. This is like an amazing new, I'm not sure: field of science?

MATUSCHAK: The funny thing is it's not even that new. It's 60, 70 years old, arguably millennia old.

WALKER: Yeah, true. And tools for thought are inventions, which can change the thought patterns of an entire civilisation, as you and Michael Nielsen define them. And the definition is necessarily quite loose, but I thought we could make it more concrete just by sharing some examples. So one of the examples that you and Michael discuss in an absolutely in incredible and thorough essay called 'How can we develop transformative tools for thought?' — which I think took you guys about six months to write, came out in late 2019 — is the Hindu-Arabic numeral system. So why were Hindu-Arabic numerals a great leap forward in the history of tools for thought?

MATUSCHAK: Oh my gosh. The thing you said earlier about changing the thought patterns of civilisation: that's a wonderful Alan Kay quote. I'm always collecting other ways to think about tools for thought. Another one that I like and that I'll apply in this instance is the notion of alien minds. There are things that you can learn that make your patterns of thought actually alien to a previous you who didn't know that thing. That can happen in subtle or small ways. You know, you read David Deutsch, it changes the way that you think about problems. But it can happen in really grandiose ways. It makes your thinking really very alien. And I think that's really true of these Hindu-Arabic numerals. If you were to somehow watch what's happening in the mind of a person who's multiplying two numbers with Hindu-Arabic numerals — and to clarify for listeners, that's just the normal numerals that you've learned and that you use. You're multiplying two two-digit numbers. What's happening in that person's mind is just so alien if all you know are Roman numerals.

Just think about just trying to multiply two two-digit numbers with Roman numerals. You have XL. (Is L 500 or is D 50? I don't know.) You have XXIII times XXXIV, and you're going to multiply these things together. How do you do it? So there are certain operations that were very, very difficult to perform with prior numbers systems. But Hindu-Arabic numerals bringing together the notion of place value — that is, the horizontal position of a number can determine its represented value — as well as the concept of zero, and using it both to represent literal zero but also to represent shifting the place values to the tens or the hundreds when necessary, when the ones and tens are empty, that made operations like multiplication possible and easy to think about in a way that they weren't before.

WALKER: Amazing. Compared to the grandiosity, to borrow your term, of the Hindu-Arabic numerical system, how is something like Adobe's Photoshop also an example of a tool for thought?

MATUSCHAK: Photoshop is astonishing. There's a wonderful institution that used to exist called Layer Tennis. In Layer Tennis, there would be two artists who would get together and there would be a referee as if it were a real tennis match. And one artist would make some little Photoshop illustration and they would volley that to their peer. And the peer would have a couple of minutes — I think it was five minutes — to do something very interesting in response, and they'd volley it back. They had to kind of build on each other and say something in each volley, and of course, ideally not say it with words. They're like puffing out their chest and kind of circling each other almost like fighting on the school playground, but all just with imagery and visuals.

Photoshop is the perfect medium to do this because layers allow you to think in this very different and unusual away. It's an extension of previous forms of collage, which are now very fluid and non-destructive. We're probably all familiar with the idea that you can just layer stuff on top of each other. You can draw on one layer and then draw on another layer and then like move the top layer around without damaging the layer underneath. And that's cool. And that already allows you to do multiples of an idea in a much easier way than you could on paper. But layers are actually these somewhat more abstract objects. They can blend in arbitrary ways.

A layer can distort a layer underneath in these interesting procedural and non-destructive ways. As a visual artist, the way that you think about constructing visual art is actually just different and alien to someone who doesn't have these tools at their disposal. It's not to say that that the previous person couldn't make those things, but it might be the case that they wouldn't. And this is one of the things that that's so interesting about tools for thought: often that the way in which they're they're most powerful is not just that it makes something possible that wasn't possible before, but rather that it changes the set of what's salient or what's tractable. You can multiply together multi-digit Roman numerals, but maybe it's not something you're going do on the spot or on the fly, or you're going need an Abacus and that's a whole other representation. And likewise, there's kinds of art which you can make without Photoshop, but maybe you wouldn't. And so you change what art gets made by changing the fundamental nouns and verbs.

WALKER: One of the key ideas in the essay by you and Michael I referred to is that good tools for thought mostly arise as a byproduct of actually doing original work on some other serious problem. Could you share an example of that?

MATUSCHAK: Yeah, that does seem to be the case. One really classic, lovely example is Mathematica by Stephen Wolfram. This is a little nerdy but it's a very important piece of software that includes a number of very exciting and original ideas about manipulating math symbolically. So in the same way that you might use a pocket calculator to manipulate digits, Wolfram allows you to manipulate equations and expressions and graphs and higher-order mathematical things. What's so interesting about Mathematica's history is that Wolfram didn't really set out to create this. He was working on a bunch of very difficult, original research in mathematics studying these cellular automata and symbolic systems. He needed an environment to help him and do this kind of symbol manipulation that he was doing, to run these simulations. And there were people who were already working on tools of this kind; he built on prior work. But I think it's telling that this particular instantiation that Mathematica which was created in this very serious context, is the one that gave rise to the primitives that we use today.

WALKER: Another classic example would be Alan Turing trying to solve a particular mathematical problem or problems, and then eventually that births computers.

MATUSCHAK: Yeah, it's really interesting. Turing, when he is thinking about the limits of computability, he's really interested in it from a mathematical perspective. It's a theoretical problem for him. It's not about physical computers. I'm not totally sure how to draw the analog to the serious context of use. Maybe the main thing is really just that Turing really wanted to understand those limits. I think that's the main thing. An issue for a lot of tool-makers is a risk that I run into myself: we're really just very interested in the tools, like "Oh, what can we do with this?" But we often lose track of the reason for the tools or some kind of underlying motivation — in the case of Photoshop the art, in the case of Mathematica the research one is trying to do.

WALKER: So one kind of tool for thought that you and Michael have been advancing is the mnemonic medium. What is the mnemonic medium and how can it radically change the way we think?

MATUSCHAK: So there's this kind of silly problem that might be familiar, which is that you spend a bunch of time reading a book, and then you find a couple of months later that you can remember like two or three sentences from this book. It seems very frustrating. If you were reading the book for entertainment, maybe this is fine, and it's true that the book has probably subtly influenced your thought patterns in various ways, and that's fine and good also. And yet, probably at least for some of the books you read, you would like to be able to remember the details.This is especially the case if you're trying to learn something fairly difficult, like quantum computation, which is the context in which Michael and I have been researching the mnemonic medium. Quantum computation contains all of these new ideas and terms and notations, and they come at you as a reader just fast and furious, one after another.

By the time you're on page 20, you're making use of a dozen different things that are totally original and new and unfamiliar, and without support for your memory it may be really very difficult for you to get these ideas into your mind. So the monic medium is an attempt to solve this problem by introducing some of the memory support techniques that we alluded to earlier, directly into books. The high-level prompt is: what if you could design a book that does the job of a normal book but just has the property that when you read it in the normal way that it suggests reading it, you end up remembering stuff, and ideally remembering stuff really quite reliably and ideally for a fairly low additional time cost.

So the mnemonic medium is an attempt to achieve that using this technique called retrieval practice, where basically after you finish reading a section, you just try to remember the key stuff from the section using some supports that the author provides and then you repeatedly do that in the weeks and the months that follow, for a total time cost of something like 30, 40, 50% extra time over the original reading time. The readers, at least of our kind of prototype book, end up really very reliably internalising all the key details of this textbook.

WALKER: So there's a sense in which retrieval practice and the mnemonic medium are like "just" flashcards, right? Although I have "just" in scare quotes there because they're actually incredibly impactful and useful.

MATUSCHAK: Yeah, there's a sense in which they're just flashcards. They're just flashcards, except they're integrated directly into the text and there's this kind of adaptive scheduling system that schedules them at the right time. But even that wouldn't be that interesting on its own. I think the thing that makes it much more interesting is that almost everybody's exposure to flashcards is of really trivial flashcards, like vocabulary words, places like country capitals, stuff like this — really trivial data. But a really profound idea that Michael and I have been exploring, and that people like Piotr Wozniak have explored before us, is that it's possible to use something like the structure of a flashcard — the challenge to recall, the challenge to answer — to support not these trivial things like the definition of a term, but rather these very complex conceptual ideas. It's possible to write flashcards which reinforce deep ideas about quantum mechanics and about quantum computation and which collectively cover all of those fundamentals.

So that if you can remember all the answers to these flashcards, it's not just that you can speak in this other language, these vocabulary words, it's that you actually understand a whole lot about like what qubits are and how they relate to classical computing bits and how to manipulate them. Ideally you understand, you don't just know.

WALKER: So just how effective is retrieval practice for halting forgetting?

MATUSCHAK: It varies a lot by domain and by person. But it is apparently more or less possible to find parameters of retrieval practice which will essentially halt forgetting for pretty much anything you could care to learn. It might take prohibitive long for some things, but I'm probably just hedging too much here. Let me say instead that, on a practical basis, for everything I've tried, which includes some just arbitrary sequences of digits, it's possible to use retrieval practice to reliably remember it with relatively low cost. Some things do require more practice than other things, and often it's possible to refactor the cards in order to make it cheaper.

WALKER: It's quite magical. So what do you make of the critique that spaced repetition is often just narrow pattern-matching. So people are recognising a question that they've written for themselves as a prompt — maybe it's long and awkwardly phrased — and then remembering that "Ah, that answer goes with that question", but not really deeply internalizing the concept embedded in that question and answer.

MATUSCHAK: I think it's a risk, and it's a fairly common thing that happens when people write the prompts poorly. Unfortunately one of the challenges of this space is that it's fairly difficult to write these prompts well. There aren't a lot of good training materials around it, and it's difficult to evaluate whether a prompt is good as a novice. You can write prompts that are more susceptible to that problem than others. It's also true that even if you do a really good job writing prompts, spaced repetition alone I think is not sufficient to deeply internalise some new subject. You have to actually do stuff. But it can really accelerate your way into the subject because you'll find that as you try to do stuff that you all all the prerequisites, you have all the tools ready at hand. I think the criticism is legitimate but can be mitigated to a large degree.

WALKER: So what are some tips for writing good prompts in a spaced repetition memory system?

MATUSCHAK: Let me just begin with the overall principle that, for me, really helps understand it. There are two principles that the rest more or less follows from. The first is that you're basically giving yourself a task in the future. So you are trying to write a task that you will do repeatedly in the future. So think about what task you want yourself to do. And the second thing is that there's a specific thing that you can cause yourself to do called retrieval practice, which will reinforce memory of what you're retrieving — that is, if you cause yourself need to remember something, you will reinforce that thing that you're remembering. So if you think about this carefully, you'll realise that you want to write the prompt in a way that specifies pretty precisely what it is that you're supposed to remember, because otherwise maybe you'll remember different things each time and it'll be indistinct, and then you won't reliably reinforce the thing.

Sometimes you may write a prompt that has you do a task that's really unpleasant or onerous. Basically you write a flashcard that has you recite five sentences or something. Often that's an unpleasant thing, and so this is a task that you don't want do in the future — and so that's not a good thing to do. So from these principles, some useful practical things follow. You tend to want to break the knowledge up into very fine pieces, and to write prompts which support memory of each of those very fine pieces. It's helpful to approach conceptual knowledge from many different angles. So if you want to understand the ways in which a qubit is unusual, you want to understand a qubit, you need to understand it in comparison to a classical bit.

You need to understand how it differs. You need to understand how practically to manipulate them, how they connect to other structures, like, "Okay, they connect to complex numbers in this particular way." So there's a list of ways you can connect things to other things that I often run through. One other thing I'd say — this is another useful meta-skill — is to reflect and be critical about these prompts. As you're reviewing them, if you notice "This one just really isn't working for me now," just ditch it or rewrite it. You should have very little patience for these things. They should feel cheap. You should be happy to accumulate thousands of them. These systems are very efficient, and you can accumulate thousands with very little cost.

WALKER: How important is it that you write your own prompts?

MATUSCHAK: We don't know. It's a very interesting question. There's trade-offs here. It's important in the sense that the more processing that you do on a thing that you're trying to learn, the more you're going to learn it. So if we remove cost from the equation, then yeah, you're almost certainly better off writing your own prompts, unless you're so novice at either the domain or at prompt-writing that you'll do a bad job, but I find that's relatively unusual. So I think it's more about like finding the right spot on the efficient frontier. One thing that's been interesting about Quantum Country — Quantum Country is the name of the textbook that Michael Nielsen and I wrote; it introduces quantum computation using the mnemonic medium — is that Quantum Country provides all the prompts for readers.

So you do this retrieval practice thing and you don't have to write the prompts using these tips that I just alluded to; some experts did it for you. And so you're missing out on this work, this extra processing that you might do. If you were to have done it, then you would probably understand the material better. And yet, right now it's the case that about 20% of the people who sign up will answer every question in the first chapter of the book. And that's maybe a decent conversion rate. It's probably kind of okay. If we were to wonder what percentage of people who start this thing would actually write a comprehensive set of spaced repetition prompts for the entire first chapter, I think it's probably two orders of magnitude less. So I feel pretty comfortable about the trade-off, given what we're seeing from interviews and from people's retrieval. Their knowledge is not as flexible as I think it would be if they had written all these prompts on their own. But they do seem to understand a bunch of stuff. And they seem to have an easier time making their way into more difficult material as a consequence.

WALKER: I think we've already discussed implicitly some of the common failure modes in memory systems, but are there any others that people should be wary of?

MATUSCHAK: I think the main thing to worry about with memory systems and with all kinds of other augmentations for one work is a sense of dutifulness. Memory systems are particularly susceptible to this because people who find them interesting are often optimisers of a kind, but it's very easy to find yourself feeling like you should learn X or Y or Z, and that you should practice A or B or C now that you have this system that can make that happen, but what this will do often is turn this thing into a chore. When these systems are at their best, they're a kind of communion with ideas that you find inspiring and exciting. It's a way to underline and practice the kind of the kind of learner you want to be, the kinds of things you want to be thinking about. And so if you find this joyful ritual filled with burden and obligation, it's a problem, and it should be rooted out. It's very difficult to reconcile that advice with the context in which spaced repetition is most often used, namely that of students in classrooms. I think that's like a fairly fundamental problem that I can't solve. But at least for professional knowledge workers who are using these systems, I think that's the main thing to look for. Keep it joyful.

WALKER: So you mentioned Quantum Country, the online quantum computing and quantum mechanical textbook that you and Michael have created which is an example of the mnemonic medium, because it uses these spaced repetition prompts which are interspersed throughout the text. It's a fascinating project. I really encourage people to check it out. That's an example of where experts have created the prompts for you. But if people were seeking to create their own prompts for their own particular projects, what are some of the tools they could use? Like Anki, Orbit. Could you talk about some of those?

MATUSCHAK: Sure. The classic that I got started on, and Michael did too, is Anki. It's an open-source spaced repetition tool. It's very un-opinionated, and so it's easy to shoot yourself in the foot with this thing, because it's not really going to help you kind of use it well. But it's a good place to start. There are some other new tools that are kind of interesting too. I can recommend taking a look at a tool called RemNote, which is a hybrid writing and spaced repetition tool. If you want to take notes and also do some spaced repetition at the same time, that's pretty interesting. There's another tool will called Mochi that works similarly.

I haven't used either of those seriously, so I can't really speak to them in detail. All of these tools operate under more or less the same principles. If you're a Windows user, the original tool along these lines is called SuperMemo. Again, I haven't used it seriously, but it has a whole lot of much more complex functionality. I tend to be wary of adding more features. I think really the core thing to master is that there's this kind of virtuosic skill of giving yourself a task to do in the future. That's the thing you need to figure out. Unfortunately really all of these tools I find fairly unpleasant to actually use. They fall afoul of various design challenges.

And so that's a limitation of the space. You alluded to a tool I have been working on called Orbit. It's not something I can recommend to your listeners yet. It's a research environment. It, like Quantum Country, is really trying to explore this part of the design space: what if these things are integrated into books that you're reading? Eventually I hope Orbit can be used standalone like these other tools and perhaps it'll solve some of the design issues, but it's not at that place yet.

WALKER: I've got one more question before moving onto your note-taking system, and that is, can you think of any ways in which a startup in the tools for thought space could strongly leverage the network effect? For example, could a platform that uses the mnemonic medium to help people remember ideas from books create a network effect by enabling its users to see the prompts that other users have written about the same book? I guess I'm just trying to think of ways around the public goods problem that you and Michael have identified that holds back progress in this space.

MATUSCHAK: Right. First off I should unpack that for folks listening in. The public goods problem is basically that if you invent an unusual novel interface, that is often fairly costly for you but very cheap for someone else to steal. So it's difficult to capture the value that you create when doing this. And we theorise that this is part of what keeps there from being more very unusual, new interface ideas. To your question, there are some exemplars in this space. Quizlet is a successful flashcard tool that has made good use of network effects in the classroom setting. It hasquite different goals from the systems we've been talking about; in some respects, it's not really about personal edification and growth and catechism, joyful communion with things you want to learn.

It is this very practical tool for students. They've had luck with their network effects. I've been currently exploring a system for Orbit, which is this prototype research platform I've been using, whereby if there's a book that already exists online, you could go and write a bunch of questions for it and then I could come along and see those questions in line as I'm reading the book; so not a separate thing you add on, as in Quizlet or Anki, but rather part of the reading experience, like in Quantum Country. I think something like that could be interesting, could work. There's another direction here that I find exciting. I did a little bit of an experiment this past year with Balaji: a memory-as-a-social-signal, proof of memory kind of idea.

There's this idea that pressing the 'like' button on Facebook or Twitter, these platforms, it doesn't actually mean that much. It's an inflationary currency. I can 'like' as much as I want, there's no skin in the game — that's just a way to say it's a poor signal. So I see that you 'like' something — that doesn't mean that much. But if I see that you read this essay and that you have been diligently reviewing the key ideas from this essay for months, that means a lot to me, like, "Wow, this is an essay that you really care about and you found meaningful,". I can have a conversation with you about that essay and I'll know that those ideas are fresh in your mind.

It might make me more likely to want to read it. If I don't know you personally, but I admire you, I might want to go look at your reading list of things that you're practicing in this way and that might mean a lot more than just something that you've bookmarked, saved to your pocket or whatever. It could also be used as a kind of certification. I really hesitate here, but the experiment that Balaji and I did was this idea of proof of memory. What if I want to incentivise people to learn about this particular new technology? So I'm going to say that if you read this textbook and you prove that you have remembered all of the key details from it, then I have a gig for you, or I have a bounty for you, or this is a badge that you can use for employment or something like that.

We did an experiment on these lines. It's clear that cheating is just an enormous problem in this space. It's obvious there are things one can do to overcome it. I find that whole space of problems really unpleasant, so I'm running away from it.

WALKER: That's super interesting. And 'Balaji' obviously being Balaji Srinivasan.

MATUSCHAK: Yes.

WALKER: I told you by email several weeks ago that I started thinking about similar problems about five years ago, although not thinking about them with the same level rigour and thoughtfulness that you have. A friend and I started working on a tool where you would input a couple of actionable insights from nonfiction books, and then it would message them back to you periodically but with decreasing frequency, as with spaced repetition.

MATUSCHAK: Product aside, did that practice work for you personally?

WALKER: It did. I haven't been consistent with the practice over the last five years. I've read many more books now for which I just haven't properly reviewed or distilled their insights. And it's scary how well I remember the older books relative to the more recent ones, because for the older ones I was deliberately trying to write down and distill the core insights.

MATUSCHAK: It's still a wide open problem space. I think something that you're talking about here that I haven't explored as much as I would like, but I have one paper about called 'Timeful Texts', that I'd like to take a lot further is this idea of actionable next things. There's a tool called Readwise, that some of your viewers might use or might find interesting, that does a piece of what you're saying, where it's like, "Okay, well, I'm reading through this book and I'm highlighting stuff that I particularly like, and then it'll resurface it to me." A lot of people really like that. I find it not all that helpful, just because there's nothing to do with the highlights. So I get this email that has the highlights and my eyes kind of bounce, skid off the surface of the highlights.

It's not retrieval practice, but it's also not something-else practice. So something that you're alluding to that I think could be really powerful is if the things that are resurfaced are actionable, then now that can change my behaviour in some durable way and we can create this kind of longer relationship with the book. I've been interested in the possibility of kind of taking some of the ideas I've explored with Quantum Country and applying them to books like Atomic Habits, books about behaviour change, to see if some of what you're talking about could be solved.

WALKER: You reminded me of something interesting. Just then when you said your eyes would bounce off the highlights. When I read books today, I don't put too much pressure on myself to over-analyse the book as I'm reading it. I might put an asterisk beside something important and then dog-ear the page so I can come back to it. Or maybe I'll draw a line beside an important section in the margin, maybe leave a note to myself or note how it connects to something else. But nothing much crazier than that. But in the past, what I used to do was almost like underline or highlight like key sections, and I don't know why, maybe this is like totally idiosyncratic, but I found that that highlighting of a passage of text somehow interfered with my recall, as opposed to leaving like a dot or a note in the margin. Have you heard of anything like that before? Is there anything to that?

MATUSCHAK: You know, there's a bunch of studies that empirically compare things like highlighting versus writing summary notes versus bookmarking and things like this. I'm not aware of a specific empirical result regarding the difference that you just described. Underlining is certainly a much more effortful way to accomplish the same thing, and so it's possible. One result that that has been replicated a couple of times is this observation that if somebody is taking really diligent notes in a lecture in real-time, they will often have poorer recall of the lecture's contents afterwards, because their attention was only half on the lecture. They're not really processing what's being said, they're just acting like a delay tape.

So it's possible that if you're underlining this passage that that's interfering with your comprehension of, or processing of, the actual words. I sometimes find, just thinking about this a little further, that I'll get to a passage in a book that's like, "Ooh, this is the juicy conclusion, this is where they're gonna draw out the tah-dah." And I'll realise that. And so I'll start highlighting. I haven't even finished reading the paragraph and now I'm highlighting. And so what I'm doing when I'm highlighting is kind of like searching for the end of the tah-dah sentence. That is not the same as reading and reflecting on what is being read. So maybe that's what's happening.

WALKER: Yeah. And then maybe a little bit later in the book you find that tah-dah part was actually articulated in a much better way and that actually you'd prefer to take that passage.

MATUSCHAK: Yeah, right.

WALKER: I'd love to talk about your note-taking system, Andy. There are some YouTube videos of you live-streaming yourself taking notes. I just find it fascinating. But the first question I wanted to ask you about it was: is "notes" the right label for the things you're writing, and if not, what's a more apt name for the units that you are producing?

MATUSCHAK: Right. Sometimes I'm writing things that you might call notes. I think those things probably align most with what other people think of when they think of notes: more or less summarising things that other people said, or other people thought, or summarising a thing that happened. It's kind of like a record. That's what I think of as a note. For a lot of people, that's what notes are for. Like, "I read this book and I want to write notes about the book, and what I'm trying to do is get a record of what was in the book." But for me that's actually like a small part of the practice, that's not really what I care about, despite all this stuff about memory systems and trying to remember what's in the book. We put all that aside for a second and say that when I'm writing, when I'm writing prose, usually what I'm trying to do is develop my own ideas.

Sometimes that involves deepening my understanding of others' ideas, but usually to deepen it, I have to get further away from what that author said, get further away from their terms, their representations. There's a great book called How to Read a Book, by Mortimer and Van Doren, and they make this distinction that I think is really helpful. They say that there's an analytical level of reading where your goal is to come to terms with the author: when the author uses the term synchrony, or mimesis, Gerard means that in a really specific way, it's not just like "meme" like we use it on the internet. It's loaded, it's freighted with all of this meaning. So you're trying to come to terms with that phrase.

And with those words, you understand very deeply what they mean, and that is distinct from this kind of later stage where you try to bring the author to your terms. You have a line of inquiry, you have a series of theories that you're developing, and you're reading in order to support it. And then you're trying to see what gems you can extract and how that informs your own writing. I don't think of a lot of what I do as "note-taking", I think of it as computer-supported thinking or writing-supported thinking.

WALKER: I love that. And I love that book. I purchased that book because of you. How to Read a Book, by Mortimer J. Adler and Charles Van Doren. There are actually some more things we could talk about in that book around things like inspectional reading and some of the other interesting tips they have. But I love what you just said. I think it's such a mature insight, and actually it took me many years to master this: just to, as the first step, actually work out what is the author's nomenclature, right? How are they actually using their particular words? Because otherwise you can just run off and start drawing erroneous connections between things.

Could you describe your note-taking system for people who are entirely unfamiliar with it? It's no doubt better for people to actually watch it happening in real-time, which is why I'll link to the YouTube videos where you live-streamed yourself taking your notes. But just for now, are you able to take us through step by step?

MATUSCHAK: Yeah, let me try. Those videos are difficult to watch, they're very long. You're you're watching me being confused, so I'm not sure I can recommend them exactly. But let me try. So let me try to characterise the differences between what I do and what many people do with note-taking systems. The main thing that I'm interested in is accretion. I've noticed that so many of the day-to-day activities of knowledge work seem to just bubble away. And there's an amount of that that seems acceptable, the angels' share kind of getting lost to evaporation. But when it's like, "I spent the entire day answering emails writing notes inside of a meeting note that I'm never gonna look again..." At that at the end of that day, I've written maybe 5,000 words, but they haven't added up to anything durable.

Hopefully I understand some stuff better, and so as I write tomorrow's 5,000 words, they'll be wiser. But it would be better if I could somehow do some of that writing in a way where it would build day-to-day. And so like one metaphor that that's helpful for thinking about this is it's like a personal Wiki, like you have a Wikipedia for your own beliefs, your own crazy ideas that you're developing. Wikipedia doesn't allow original research, but your personal Wiki can. Very concretely for me, that's this big folder of note files. I found that a number of practices really help to write notes — and again, I'm using this word "note" here in an unusual personal way — in a way that accretes day-to-day.

We've talked about a bunch of ideas over the course of this podcast about spaced repetition. And one of those ideas, for instance, was that when you repeatedly review ideas from a book, apart from the impact on memory, this has kind of like an identity impact on you. And that insight, that claim, is represented by like a single file in my system, there's a page and its name is basically a short version of what I just said. It started when I was in a couple of user interviews and I noticed that a couple of readers of Quantum Country had said something along these lines. And so when I noticed that, I summarised that effect as I saw it, and I extracted several quotes to that page.

And then when more readers said things along those lines, I extracted those there too. And then I was thinking very generally about, "Well, what are all of the ways in which spaced repetition affects me in a way that isn't about memory?" And actually there's a whole bunch of different effects, some of which we've discussed in this conversation — there's this interesting creativity effect, and so on. And so now, that one thing, that one insight, about changing your identity and giving you this connection over time, that can be related to these insights about creativity or about salience, through this question of what are the non-memory impacts of the spaced repetition system. So there's this kind of organic growth that's mediated by having a bunch of little notes.

You can think of them as index cards or as files or as pages, and those pages, critically, are concept-oriented. Most people take notes in a way that is event-oriented, where an event might be a meeting. Like, these are my notes on my conversation with Joe. Or it might be an event in the sense that I just read this book and so here are my momentary reflections on this book. These are kind of "write once". Like, at the end of this meeting I'm going write my notes on this meeting and then I'm gonna use this as a reference in the future. This is a record of the meeting, but it's not something to be expanded over time. By contrast, concept-oriented notes are structured so that, because they're organised around a concept which is open rather than shut and which isn't associated with a particular event, they can grow over time. So that insight that reviewing ideas from a book over time can change your identity, that is a concept rather than an event associated with those interviews, and it's something that I will add to and probably whose whose shape will change over time. There's a lot of other things I could say, but I'll pause there as a reasonable introduction.

WALKER: That's really useful. Thank you. And the basic unit of production for you is ultimately what you refer to as evergreen notes. So what are the properties of a good evergreen note?

MATUSCHAK: So they're concept oriented, as I described. That allows them to kind of expand over time. It tends to be better if small. I have evergreen notes which are large. So for instance, "mnemonic medium" is an evergreen note that I have, but mostly it's just links to other smaller notes. The smaller notes tend to be where the action is, and that's because when you're linking to things, linking to a very vague general concept is less useful than linking to something quite precise.

Another thing that I find quite useful is trying to be very thoughtful about the title of each of these evergreen notes. For programmers listening, the titles are like an application programming interface. They're a handle, they're a way to refer to that idea. And so often the hardest part is coming up with the title for that particular insight that I can refer to repeatedly as it grows. It's also good to make them densely-linked. So a thing that happens as you develop your personal Wiki or your Zettelkasten, whatever you want to call it, is that you wander and you find yourself surprised, and it's very good to find yourself surprised. In some sense, the system's only working if it surprises you. So one of the benefits you get, if you make these notes very, very small, very atomic, and also concept-oriented, is that you find they're full of links to each other.

WALKER: When you say atomic, what does that actually mean? Because presumably you could split and divide ideas ad infinitum.

MATUSCHAK: Yeah. There's judgment here. I'll try to make the following point in a minimally technical way. In programming, there's a constant tension between trying to make a maximally general little tool that can always be used in all these little places, on the one hand, and to make a like black box system that is very convenient that you can always deploy in a particular situation, on the other hand. The former is more flexible, but maybe you have to stack a whole bunch of those tools together to solve a problem. And then the latter is less flexible but it's more convenient. So there is that tension here also; the more fine-grained you make the notes, you lose cohesion, you start to have to gesture to a set of three or four notes in order to convey a particular idea space that you want to link to, instead of linking to one thing specifically that connotes the space. Sometimes I'll do a little bit of both. I'll factor a note into three or four sub-ideas, so that I can link to those sub-ideas specifically where it makes sense or develop those sub ideas separately, give them space, room to breathe, but then I'll make a parent that points to all the sub-ideas. And then in places where I want to gesture to that whole space, I'll refer to the parent.

WALKER: Makes sense. And so what is the process of turning inputs into evergreen notes? What does that process look like?

MATUSCHAK: So I should clarify that I have evergreen notes, which are accrete, and then I have other notes which don't decree, and those are usually either event-oriented notes, like about a conversation, just a record of what happened, normal kinds of notes, this is my journal, what I'm doing today, what I'm thinking about. Or, I read this book, here's roughly what this book says. None of these things accrete. So often things start that way. I'm just writing about what I'm thinking about. And then I notice that something can grow and outlive that context. There's an insight from this conversation. There's an insight from this book. There's something I'm developing in my thinking right now that deserves to live beyond this moment. And so that's when I'll try to extract that. Often it's connecting to something else. Often, when I will write, it'll be the second or third time that I've alluded to something in one of these throwaway contexts, that I'll realise, "Oh, okay, I've written about this a few times now. Let me go pull the times I've written about this and summarise and synthesise that and feed it into everything else I'm thinking about."

WALKER: Before I ask my next question, I'm conscious we're coming up on time. Are you okay if we go slightly over?

MATUSCHAK: Yeah, I actually have until half-past.

WALKER: Amazing. Thank you so much.

MATUSCHAK: Yeah. This is fun.

WALKER: I feel like it would be a tragedy to stop early, because I'm really enjoying this. So the next question. We've spoken about spaced repetition memory systems and we've just been introduced to your evergreen notes and your personal note-taking system. How do you integrate a spaced repetition memory system with an evergreen note system?

MATUSCHAK: This is something I've been experimenting with and which I find really captivating. A problem that you'll notice if you the reader try to use most spaced repetition systems, is that they feel a little bit like a shoebox, like you write on a card and you put it in the shoebox and then the card's gone. It's somewhere in the shoebox, with 10,000 other identical looking cards. And if you open the shoebox, you can find it again probably, but it's next to all of these other cards that look identical and it's completely lost its context. So something I found really interesting is trying to make these things really contextual. One way to do that is by embedding them in a book.

So we've talked about that with Quantum Country, but another way of doing that is by embedding them in my notes. I was just learning about how RSA encryption works in detail yesterday, and I was writing some notes about that and alongside the prose notes I was writing (the textual paragraphs), I wanted to reinforce 15 or 20 details about that encryption scheme and why it was constructed the way it was. And so I wrote a bunch of questions in-line in my notes and I have this way of doing that in my notes where they get turned into spaced repetition questions. There's some other software that you can find that's been in inspired by this practice, and in the case of RemNote, what was just independently developed, that will let you do this kind of thing too.

I think that the norms and practices for this are yet to be developed. So there's still some problems, there's overlaps. I'll find that I'm writing the same detail twice sometimes. Sometimes I write it in prose and then I have to write it again in spaced repetition form, and there's ways to get around this by using this thing called closed deletions. I'm not going to go into that in detail. Let me just say it's a little bit unsolved still. But there's something tantalising about it. And there's a correspondence where in the same sense that we talked about trying to find the really precise atomic representation of some insight, writing a good prompt is often the same way.

You want to get to the heart of this thing you want to remember, so that you can reinforce that. The process feels similar and if the similarity makes it feel duplicative and like wasted work, often when I find the really distilled insight, I I'll write the prose representation of the distilled insight, and then I'll turn that into some questions. And note also that I'm turning it into questions about my own insight. This is a thing that I think is not intuitive to do with spaced repetition questions, but I think it's great. "I just had this idea. I noticed this connection. Great. I'm going to reinforce it even though it's my idea. It's not something from a book, it's not something I learned elsewhere. It's mine." But practicing that will give it more chances to connect to things, give more chances to grow. I will do it in-line in the note, but there's a sense in which it still does feel kind of duplicative, even though now it's contextual so we've solved the problem with the shoebox.

WALKER: Why is the number of evergreen notes written per day the best metric for a knowledge worker?

MATUSCHAK: This is a provocative claim. I don't quite believe it as stated. But it's a really interesting heuristic. What is an evergreen note? For me, it represents a durable insight which is sufficiently distinct from other insights as to have its own identity, which is going to survive the day and possibly accrete over time and connect to other ideas. And in many cases it's original, or at least my framing of it is original.

And so in some sense, if I can extract 15 of those in a given day — and that's pretty rare by the way — then that's like a very, very intellectually productive day. It's like, "Wow, 15 like distinct, durable, independent insights that are well-articulated? Wow, that's great." Often, what a less successful day, creatively, looks like is a whole lot of typing into those ephemeral scratch files. What is less successful about that is not that I'm typing into the scratch file instead of into evergreen notes — that's a symptom. What's really happening is that I'm running in circles and I haven't distilled out any single sentence that I can articulate like, "Aha, I have figured out this sentence, this is a thing that I know now." Instead, I'm just kind of like writing in circles. So days in which I manage to extract a lot of these insights tend to have gone pretty well.

WALKER: Do you have a sense of the proportion of evergreen notes you've written to date that you've had to kill or substantially revise because you just got something wrong?

MATUSCHAK: Sure. Much more common than getting something wrong is something I changed my opinion on or revised. A very common thing that happens is I have to weaken something. So the titles will often be claims like "X is Y", "X causes Y", and a very common change, probably 10 to 20% of such notes, end up having to get to change to, "X sometimes causes Y", "X is associated with Y". Sometimes, I just completely change my view. That's a fun one. I would say that happens probably 5% of the time, maybe no more than that. And when that happens, there's a fun propagation where now I walk through all of the notes that cited that evergreen note that I no longer believe, and I have to go rethink them, and often I'll realise that it has some consequences that I hadn't expected.

WALKER: Right. So that node in the network has faltered.

MATUSCHAK: It was weight-bearing

WALKER: It was weight-bearing, right. It remains somewhat of a mystery that knowledge workers do appear to be so unserious about deliberate practice and honing fundamental skills. And I apply this to myself. Knowledge workers don't work on their reading ability or their note-taking in the same way that, for example, an athlete would be practicing by shooting hoops or a musician would be practicing scales on a daily basis. Why do you think that is?

MATUSCHAK: There's pretty good reasons for why it is, thankfully, so at least we can take a little bit of solace in our failures here as knowledge workers. I think it's mostly because we don't know how. So the deliberate practice academic research discipline has identified a whole bunch of things that must be true in order for deliberate practice to be possible. And there are things like there's a set of well understood skills which are necessary to the discipline, and there are known exercises which improve skills. There are ways of soliciting feedback reliably regarding these exercises. There are coaching techniques for communicating these exercises and teaching, and for many things in knowledge work, this is just not true.

We might agree that rapidly distilling insights from a text is a core knowledge work skill. We probably don't have a lot of consistent agreement on the exercises, the drills, that we could do in order to durably improve this skill. I mean, there's things that people propose. But there isn't a lot of consensus. I think it's possible to make some progress here. I think it's probably possible to formalise some of these practices more than has been done. And I think it's probably also possible to make progress through this other route of more effectively communicating tacit knowledge amongst ourselves. That's so much of what knowledge work relies on: these very subtle things that we do that we don't know how to formalise into exercises and practices. So, if we can figure out how to formalise them, that's great.

And if we can't, let's figure out how to communicate them anyway. And you actually alluded to one method that I'm pretty excited about earlier, which is that I did a screencast of me writing some notes about some obscure academic idea in design. And a bunch of people watched that, and it's kind of mystifying to me but I think some 10,000 people have watched this video now — it's like an hour and a half of me being confused. And I think part of the reason for that is that it's a source of tacit knowledge. You watching someone who has some skills maybe that you don't have — certainly still very confused in a variety of ways — but who has some certain skills you don't have, and by watching them, even if you don't have a, a deliberate practice method to like drill that yourself, you can kind of absorb it, it's like a really weak apprenticeship, but a mass medium compatible apprenticeship.

You'll probably be unsurprised to learn that there are some academic disciplines interested in studying ideas along these lines, specifically for tacit knowledge s. There's a few approaches that rely on similar technique. One story I always love to tell along these lines is so familiar to everybody. And that's the story of being a child in the kitchen with your parents cooking. I don't know if you had this experience, but there's this phenomenon of what we call legitimate peripheral participation where you're in the kitchen, initially you're just watching as a child, but maybe if the child calms down and seems to be attentive, you give them something to stir or you ask them to fetch something from the cabinet — and this isn't make-work, it's not an arbitrary exercise; you're legitimately participating, but peripherally. Then of course you might be asked to crack some eggs, like step up the responsibility. This is so natural in the kitchen. The same kinds of approach can apply in workplace settings.

WALKER: For sure. So I'd like to talk about note-taking software as distinct from note-taking systems for a moment. Do you still do your daily notes in Bear?

MATUSCHAK: Yeah. Let me prefix this by saying that I think many people over-index on the question of software. And I think by far, the most important thing to think about is the way that you're going to think and write. The methods are really the important thing. But that preamble aside. Yes, I use Bear. I like Bear. It's very polished. It's not as powerful as some other systems. I tend to find myself not using more elaborate features quite so much. I know a lot of other people like Obsidian and Roam — these are popular. But yeah, Bear is what I use day-to-day. One thing that I will advocate for, because I think it's pretty important, is that your notes should be yours. You should have a folder of text files or something like that, that are not trapped in some system. You really don't want to end up trapped. That's the opposite of evergreen.

WALKER: If you look at the video of you live streaming yourself taking your notes, you can obviously see you working in Bear, but you also used Bear's back-end to create your like note Wiki? Was that Bear?

MATUSCHAK: No, it's not really Bear at that point. I have a folder of text files and I made a web app that renders the text files. So Bear's not really involved at that point.

WALKER: Got it. I'm not sure where I got that idea. But a lot of people have now tried to obviously replicate that.

MATUSCHAK: Yeah. So if you like that, Obsidian Publish will make a website like that for you. And Roam Garden will do it for Roam. And Craft also knocked off my site. This is great as far as I'm concerned. I'm like a weird software researcher person. And my theory of change is there's like a trope that like ideas don't matter, execution is all that matters in Silicon Valley. And I think that's approximately true as far as a startup is concerned. You don't want to be diving into a startup with the fundamental theory of what you're doing super unsolved. So my goal for my work basically is to make the ideas so mundane that they're just obvious fodder for startups to execute. And that way I don't need to execute them at scale. II'll stay on the tinkering, inventing side.

WALKER: Can you describe your process for reading a non-fiction book and committing its most important ideas to memory? So say you're sitting down, it's a Tuesday morning, you're at your desk with a cup of coffee and you open up a non-fiction book that's going to be a key source for something you're writing. Where do you start? Are you doing inspectional reading first? How does it end up in prompts in evergreen notes? What is that whole process?

MATUSCHAK: If I have a motivation, like I know it's a key source of something I'm writing, then my experience reading this is really going to be mediated through that motivation. So I will often start with a list of questions that I want to answer from this book. Often they already have answers from other resources and I will flip through the book's table of contents, its headings, its indexes, looking for kind of the density of like, "Where am I gonna find this answer?" And I will then like jump around to those areas and read first and last paragraphs of sections, first really just to evaluate: is this a book that I want to spend more time on? That's the key thing for me as a para-academic. I run into this issue that there is always a literature. On any subject, there is a literature, and there are tons of books, and the books are long.

And the number of pages published does not necessarily correlate to the amount of insight that the field has about the question. And so the first thing I'm trying to answer is: is this going to help me? Do I find myself wanting to read more? The answer is often no. So in that case, I'll scratch a few notes about my impressions of the book into a note about the book and then move on. If it does seem helpful, then I will usually sit down for a longer pass at this point. And I will, depending on the length of the book, just read the whole thing on a first pass, moving fairly quickly through sections that are less relevant to my questions and more slowly through questions that are highly relevant.

Or if the book is quite long, then I will read just subsections of it. And usually there's an amount of spaced repetition question extraction that's going on just as I'm reading — anything that seems particularly important, not because it's necessarily really big picture, but just because it's table stakes for understanding the author's argument, like they're, depending on like this fact, this theorem, this research result, whatever, I'll extract that stuff. And it'll start in a note about the book, as opposed to an evergreen note. The other thing that I'm doing as I'm making my first pass, is I am very grossly marking up the book. And like you, that mostly just means like lines in the margin. Occasionally I'll write a word or two that's really just to jog my memory about what I plan to do with that.

But I'm not really doing the full reaction, the full synthesis, at that point. I find that I have to load the book into my head before I can digest it. So I try not to do too much in real-time, other than just the memory support stuff. Once I've made a first pass, I'll try to summarise my understanding of the book in the book's terms, in a note about the book. And then I'll work fairly methodically through the parts that I marked up and ask for each part, "Is this some important idea that I want to keep working with?" Often they're grouped or clustered, so it'll depend a little bit on the book, but one common method will be that I will actually extract all of those highlights into a note and then I'll start mashing them around and grouping them and clustering them.

Sometimes I will already see that one cluster relates to some evergreen note I was already developing. And so often it'll just be like supporting evidence for some evergreen note I already have. And then it'll just go in a reference section in that note. So long as it doesn't actually change my view on the topic, the body of the note doesn't change. Also, very often it represents a new insight, like, "Wow, I learned something and this is like relevant to my work." And so now, the task is to summarise that insight in my own terms, supported by what the book has to say, and then connect it to all of the rest of my work. So the question to ask and answer is: So what? What are the impacts here? What will be affected by my understanding this?

Often this work that I'm describing is maybe a day of work or more, for a book I'm reading fairly seriously. And it'll often turn into several more-specific and focused re-reading passes. One other element that I failed to mention but is often one of the most important things from a book, for me, is the bibliography. For a book in particular, and for my work in particular, I usually get a lot more signal if I move from the book to the primary source, whatever it is. Usually the book is describing studies or papers or whatever. And so if I found the material interesting, then I will usually use the bibliography to lever up on whatever seemed important.

WALKER: A hundred percent. And maybe there are a few different books and they're all pointing towards the same primary sources. So you can kind of triangulate and then go straight to the primary source.

MATUSCHAK: Yeah. I'll accumulate lists of "to read" notes.

WALKER: Imagine now that you are writing a non-fiction book — and you've got to suspend your reservations about the effectiveness of books, as examples of transmissionism — and here you're writing a conventional book. Maybe you've got two to three years to write this book. It's about a topic you're interested in, but you're not quite an expert yet. And assume you're writing the book because, firstly, you want to induce yourself to understand the topic better. You're genuinely curious about the topic, but you also think it's an important topic — you know that much at least — and you expect to eventually have things worth saying to the public. So what would your system look like if you were writing that non-fiction book, say for a general audience. It requires a lot of research. You have to read a lot of books and articles, papers spanning multiple disciplines. Would your system basically look the same as the one you've already outlined?

MATUSCHAK: I don't know that it would look all that different. So I should qualify that I haven't written a non-fiction book before, but I've written multi-deca-thousand-word essays. And you stitch half a dozen of those together and you have a non-fiction book. It seems plausible to me at this point that the same insights could apply. One thing that comes to mind is I think the book would already be partially written — not necessarily in form, but something that's happened to me again and again with essays I've written since I've adopted this system is that I'm writing all these little evergreen notes that are distilling insights and I speculatively link these things together into outlines.

So all of these notes about the mnemonic medium: there'll be these little tiny, fine-grained notes, but they'll usually be linked in some big outline about the monic medium. At some point, I'm going to write a monograph about this thing and it'll probably be like 50,000 words. When that happens, I will have much of the material at hand — and the "when this happens" part is to some degree dictated by the nature of those outlines. My work is such that I'm not likely to embark on writing a book on a topic I haven't already done a ton of thinking and writing about. I don't need to do it for revenue purposes or anything like that. So it would really be because I think I have something exciting to say, and the way that I would know that I have something exciting to say is that I have already said many thousands of words worth of things about the topic.

WALKER: I'd love to finish with some random questions. I'll fire them off rapidly, but obviously feel free to take as long as you like with your answers. How do you think about increasing the quality and or frequency of intellectual exchanges in your life? So if you were an academic, you could walk down the hallway, pop your head into a colleague's office and ask them a question. What's your equivalent? And do you think it's essential for knowledge workers to be part of a scene?

MATUSCHAK: I find it totally essential. I find that the old trope is true I really am, to some degree, the average of the people that I talk to. And so the people I talk to need to be really high quality and I take a lot of care in trying to curate them. Twitter is, I think, actually amazing for this. I know many people don't love Twitter for various reasons, but I think it's possible to curate Twitter so that you are talking to some of the most brilliant, interesting people on earth. And then the source of some of the best things that have happened in my life in the past five years has been to turn those Twitter relationships into real life relationships — some great new friends and colleagues and opportunities have come for me that way.

Basically all of the great opportunities over the last five years have come that way. Hosting dinner parties is a really important part of my practice as a para-academic, and attending the local dinner party and salon scene. At least prior to the pandemic, being willing to fly around, spend time in different places and soak up different scenes has been important too. I wish I had more advice here, and I'd be excited for any of your listeners' advice.

WALKER: You make a really interesting point because I think one of the common criticisms of online communities or online connections is like, "Oh, well, how could they possibly be a substitute for real life interactions?" And my response to that is, "Well, that's not quite the point. They're not mutually exclusive. The most important online connections lead to real life interactions."

MATUSCHAK: Totally. And collaborations. Endless conversation. Trips together. Absolutely.

WALKER: Next question — and I have no idea whether you have an opinion on this, this is a total Hail Mary — but what is the best font to read and/or to type in, and why?

MATUSCHAK: I will try to answer this question. So there are some things that are just disqualified because they are poorly made. Use a high quality font made by a reputable foundry. If you were using a Mac, Apple has licensed many good ones, and so most of the stuff installed in your system qualifies. And then in terms of what to use to read and write, fonts affect the way that you think and feel. I find, for instance, that if I use a beautiful serif, book font for my work-in-progress notes, that that feels imposing and it makes me self-edit more. So using something more casual or even typewriter-style, will help me for messier work. Whereas when I'm writing a manuscript, seeing it typeset really beautifully and seriously, like this is like a legitimate, earnest work, it actually helps me like rise to the occasion. So just think about it in terms of manipulating your emotions.

WALKER: That's great advice. What is the best a capella song to perform, and why?

MATUSCHAK: It's useful to think about what makes the voice so powerful. One of the things that's really great about the voice is that you can tune things perfectly. You may not know this, but all the pianos are out of tune. This is because of an issue called temperament where you can't simultaneously tune all of the notes. But voices and un-fretted instruments can tune perfectly. And the consequence of this is that they can make wonderful physiological effects happen where it feels like the air is vibrating and you make a perfect fifth and so on. So anyway, I particularly appreciate a capella music that takes advantage of those phenomena. Barbershop is designed to do that. It's not my favorite style, but I enjoy the effect.

And so a good example of the effect in barbershop is there's a medley of the Hunchback of Notre Dame from a group called the Ringmasters. It's astonishing. At the ending, you'll hear a fifth voice appear because of the effect that I described, even though there's only four singers. And at totally the other end of the spectrum, Jacob Collier's Moon River takes advantage of this effect in really, really interesting ways. I also think the voice is very interesting for groove and dance, like phonic dance. If you're interested in hearing voice for groove and dance, groups like Naturally 7 are wonderful to listen to.

WALKER: Awesome. I'll check those out. Last question. You've had several successful collaborations in your career, especially I'm thinking with May-Li who was an Apple colleague who moved with you to Khan Academy, and also obviously with Michael Nielsen more recently. Do you have a formed theory of partnerships — how they fail or succeed, and whether the successful ones are a net benefit?

MATUSCHAK: I don't have a unifying theory. I'm not sure there's any kind of single advice I can give. The main thing I can observe is that partnerships are amazing and the best work usually happens, at least for me, through them. There really is a more-than-the-sum-of-the-parts effect. They're costly. And so I've had many unsuccessful partnerships as well. You need to work with somebody who you just think the world of, the absolute world of. They are somebody who's just really going to inspire you every day. And I think part of why that's necessary is that good collaborations require just a great deal of trust. It's very difficult to let go creatively when collaborating with a lot of people, because if you're a serious creative, you probably have really strong views on how things should be. But when you're working with someone that you really admire and that you really trust, you're happy to just take your hands off the wheel and know that they're going to make great decisions. Bringing that kind of trust and expansiveness into those relationships and then flowing with what happens, has for me led to some really lovely results.

WALKER: Andy Matuschak, thank you so much for joining me.

MATUSCHAK: Thanks for a lovely conversation.