It's here! Learning Medicine: An Evidence-Based Guide

Dear Readers,

I'm glad you're still here. I've been quiet here on the DrWillBe blog because I've been hard at work on a major project. For over two years, I've been writing an a book on learning in medical school with my friend and colleague, Dr. Peter Wei. 

Since the inception of this blog, I've written enthusiastically about the power of spaced repetition (SR) using the program Anki. No doubt, SR and Anki are extremely potent, but they are not the only components of highly effective learning. There are many ingredients in that recipe, and so I wanted to flesh out a comprehensive methodology for learning medicine well. 

The product is Learning Medicine: An Evidence-Based Guide

Learning Medicine is the result of several years worth of research and experimentation on our part to discover the most effective methods for learning in medical school. We've distilled the insights from those efforts so as to create a guide that is very practical, allowing you to apply these high-yield study hacks immediately. 

Here are some highlights from the book:

• Take a brief tour through the science of learning and memory
• Learn about the power of spaced repetition for building long-term knowledge
• Practically apply spaced repetition using the popular app Anki. We give you very specific instructions on how to make this program work for you. 
• Go step by step through our comprehensive Learning Method, which embodies all the study hacks we discovered in our research.

This is the "instruction manual" so many of you have asked for. It will help you get started using SR decks, like my Pathology Anki deck, and will give you many more tools to continue learning effectively on your own. 

You can purchase your copy of the book here

I hope you you find much value in this book. Let us know what you think by leaving comments on the book site, or here on the DrWillBe blog.

Happy Studying!

-DrWillBe (Alex)

Studying Pathology (And My Anki Deck)

Update 2015-05-28: I've co-written a comprehensive guide to learning in medical school that incorporates Anki with a host of other evidence-backed study hacks. This book is an essential companion to the ideas in this post. 


Disclaimer: The recommendations below are for how to best learn key subjects in med school. In fact, any study advice I give on this blog or elsewhere is always with the aim of learning for the long term. For that should be the primary goal in my mind. However, I just want to be clear: my recommendations are not geared toward destroying class exams. If that is your goal, you need to really pay attention to lecture slides and class material and old exams and such. I also want to be clear about my intentions and own abilities. I'm a regular med student who just likes to share ideas. I'm know there a lot smarter people in med school then me. In fact, I'm sure of it. But I think I've found things that work well and I want to share that information with anyone who will listen. Always be critical and always evaluate whether what I'm saying will suit your needs and abilities. Thanks for checking out Dr. WillBe.

I've been meaning to write this post for a while. With the first years at my school starting pathology last week, I thought now would be the right time.

Pathology is perhaps the most important preclinical subject. Pretty much, this is medicine. You've learned about how the body functions normally. Now it's time to see what happens when things go awry. Spending the time to learn pathology right will pay great dividends in your future clinical career. Having solid knowledge of the diseases that afflict people and their underlying mechanisms is a requisite to being an excellent physician, in my opinion. It's what separates doctors from technicians. We understand the why as much as the what. And with that understanding, physicians can push the boundaries of medicine rather than merely practicing it.

OK. Enough of my soapbox speech. Clearly I think Pathology has intrinsic value, but more practically, Pathology is the most heavily represented subject on the USMLE Step 1 exam. If you want to do well on Step 1, you need to know Path. And know it cold. There is no way around that. That's reason enough for most med students to take Path seriously.

How does one do that? Well, that's what I'm going to talk about today. As in my other posts, I'll discuss what I actually did and do in my own studying, and then I'll suggest what I would have done if I could do it again.

Methods
First, let's talk about methodology. I'm beginning to sound like a broken record, but I can't help singing the praises of Anki, a spaced-repetition flashcard program. I've benefitted so much from this tool that I want to tell everyone I know about it. You can read about the how and why of SR and Anki here and here. Needless to say, I think Anki is awesome for pathology and it has been the core method for my study of the subject. Actually, of all the subjects in med school, I think Anki is most important for Path because of the vast number of facts and ideas that one has to commit to memory.

For Path - and pretty much every other subject I study using Anki - the process goes like this.

Read - Make cards - Immediately review cards after reading - Repeat

It's that simple. What makes Anki challenging for many students, however, is the fact that you need to continue to review old cards even while you plow through copious amounts of new stuff. It's a struggle, I'll admit, but it's the only way to keep all the hard wrought information you learned already in your head. It get's better over time, and when you stop adding new material, the number of old cards you have to review gets smaller and smaller.

Before I go on, I want to clear up a misconception that people have about Anki. Flashcards ARE NOT A PRIMARY LEARNING TOOL! (sorry for the caps, but I feel like I need to scream this). I don't learn from looking at a bunch of disjointed flashcards. Nobody does. And if they do, they're probably doing a pretty poor job of learning. My primary learning tool is a book. I want a story. I need to see facts in context. We all do. That's not what Anki is for. Anki is for retention of information, not first-pass learning. You use it AFTER you've learned and understood something. This is one of the cardinal rules of SR best practices. Don't make a card for something until you understand it.

Many people will crush a 50 page chapter in Robbin's Pathology or something similar and then close the book, deluding themselves into thinking that just because they scanned their eyes over that much text, they actually are going to remember most of it. Then they go to take a test or something, and they find out that they only remember about 10-20% of what they read, if that.

Maybe there are 1 or 2 beasts in medical school who read massive texts without any other consolidation strategy, but for the rest of us, there needs to be some way to retain and maintain the knowledge we gleaned from our primary studies. Lots of cognitive research - and just good common sense- confirms that repetition and review is essential to long term retention. Moreover, the spacing of reviews over time with increasingly longer intervals is key. And that is what Anki gives us in a way that we could never do before. My personal path deck (see below) has over 6000 cards in it. Those facts are all pulled from the sources that all medical students use to learn path. If the number is daunting, that's because it is. There are just that many factoids to remember. Without a digital SR program like Anki, however, I could never make and manage that many physical flashcards. It's awesome to be a student in 2012 :)

So, bottom line. All my recommendations for studying path, and pretty much any other subject in medical school, depend on Anki. I don't believe in binge and purge. Slow and steady wins the race. Come time for Step 1, if you've used Anki diligently and continuously throughout medical school, there will be no need to have to go back and relearn most of 1st and 2nd year in the 4-6 weeks allotted to you for Step 1 study. Rather, all that knowledge you gained will be burned permanently in your mind, ready to access when needed. Then, instead of focusing on content review, you can spend your time mastering the information by doing tons of questions and problems. This is what I'm doing for my own Step 1 strategy. So that's the point of all this. To not forget. To master the material. To build a rock-solid foundation for rest of your clinical career. And to destroy Step 1.

Resources
This is the part that you've probably been waiting for. You're saying to yourself, "OK WillBe, I know you're an Anki zealot. I get it. I'll use it. But what should I make those cards from? What should I study?"

I'm glad you asked.

Because Pathology is such an important topic, there are a lot of resources to choose some. Some are better and more appropriate than others.

Let me first give a list of all the ones I know about and then we'll talk about which ones you should be using.

General
First Aid for the USMLE Step 1 - the gold standard for Step 1 review. Getting through FA should be the minimum goal you achieve in your studies. Everything in this book is high yield. This book might just seem like a litany of bulleted factoids (and it is), but it's surprisingly helpful. There are things in here that don't show up in any other source.

Textbooks
Robbins and Cotran Pathologic Basis of Disease, 8th ed ("Big Robbins") - the Bible of Pathology. This is the book that Pathologists learn from. Med students for decades have used Robbins as their go-to source for path. Most people use it now as a reference, but some students use it as their primary source. Depending on how much time you have and your willpower, you could slog through all of Robbins in your pathology course.

Robbin Basic Pathology 8th ed (9th coming out in April) ("Middle Robbins") - A pared down down version of Big Robbins. This book is more approachable, and more appropriate for medical students learning pathology. The core concepts are the same as in Big Robbins, but there is less information about the histopathology and sometimes less detail in the discussions of pathogenesis. Still, at 946 pages, Basic Pathology is no push over.

Review Books
Rapid Review Pathology 3ed ("Goljan") - In the last few years, this 'review' book (and I put review in quotes because this book is 600+ pages) has emerged as the favorite of medical students. The author, Edward Goljan, is sort of a legend amongst med students because of this text and the accompanying bootleg audio lectures. This book has everything one needs to crush the pathology on Step 1. It's written in bulleted format with high yield factoids in the margins. Many students bypass the above textbooks and learn directly from Rapid Review. This book has proven it's effectiveness over the years, and that's why it continues to be the favorite every year. I'm not so sure though that it's the best anymore (see below). Pathoma may have knocked the king off the hill.

Fundamentals of Pathology by Dr. Sattar (Pathoma) - Pathoma is a new resource that just came out in April 2011. It is a series of video lectures and a review book made by Dr. Hussain Sattar from the University of Chicago. In less than a year, Pathoma has become the hottest Pathology resource around. I learned about it after I was already done with Pathology, but I've been using it to fill in the blanks since then. There is 35 hours of extremely high-yield, high-quality videos and an accompanying textbook that is equally impressive. And all this comes at $100, which is an amazing deal when you think about it. Many have dubbed Pathoma 'the New Goljan'. I think this is correct. The book is more concise. The videos give more explanation. Dr. Sattar is just a gifted teacher. The difference between Pathoma and Goljan is that the latter has proven itself over the years as a tool for success. Students have used it for Step 1 and have done very well. Pathoma, on the other hand, has not had that kind of track record yet. Many med students rave about it on Student Doctor Network and some have used it for Step 1, but not as many as have used Goljan. So that's the downside. Using Pathoma is using an unproven tool. But that shouldn't deter you.

I've used all the canonical resources. I've read Robbins. I've read First Aid. I've read Goljan. And I've been reading Pathoma and using the videos. So I know them all. Pathoma has everything that First Aid has and more. So right there, Pathoma is already as good as First Aid, which people trust without a doubt. There is  significantly less material in Pathoma than Goljan. But it is questionable whether that means much. I'd say everything in Pathoma is high yield. Goljan, despite being a review book, has lots of miscellaneous info that may or may not be useful on Step 1. In fact, I've heard it said that Goljan not merely for Step 1 (although that's how it's used) but also for Steps 2 and 3. So that explains some of the difference.

So while Pathoma hasn't stood the test of time yet, I think in a few years it will completely replace Rapid Review. Because Pathoma is higher yield, better written and more approachable than Goljan, I think one would be better served to use it. One stands a better chance of actually getting through all of Pathoma than Goljan too, which is significant. Better to have a complete view with good detail rather than an incomplete one with lots of detail in some areas but none in other.  If one knew everything in Pathoma inside and out, I think he'd know more than enough pathology to excel in class and on the boards.

Plus the videos are excellent. Until now, everyone has used the bootleg audio from Goljan. I never did like these very much. The quality is poor. There are a lot of jokes and tangents. Plus, there is no visual component. And did I mention it's bootleg? Pathoma, on the other hand, is a whole library of video clips that one can watch as well as listen to. The visual component is key. And with the book, you can read along.

I am really enthusiastic about this new resource, in case you couldn't tell. I wish it was around when I was learning path the first time.

* Full disclosure. I have no ties to Pathoma. I've spoken to Dr. Sattar once just to tell him I loved his work and to keep it up. That's it.  I want to support a good thing when I see it and so that's why I'm promoting it here.

Here is a sample video. Check it out and see if this style of teaching appeals to you.



BRS Pathology - This used to be the go-to source for Pathology review a couple of years ago, but it's since been replaced by Goljan. I really didn't use it, so I can't say much about it. Some people like it. The BRS format is familiar and effective. It's less dense than Goljan, bigger than Pathoma and First Aid. I think with Pathoma on the scene, BRS will become less popular even, but I don't think it's a bad book. If you like BRS books, check it out.

Question Banks
Robbins Review of Pathology - This is a companion book to Robbins. It's all questions testing your knowledge of path. My classmates used this book to study for our exams and we were well served. It's quite detailed, much more so than Step 1 level pathology. The nice thing about it is that every question references a section in the Robbins family of books so you can go read when you get a question wrong. I think this book is worth having.

USMLE World Step 1  - Does this even need an explanation? USMLE World is recognized as the best source for Step 1 prep questions. There is debate about whether or not you should use these questions while still learning material. Some people think you should save UWorld for when you're doing dedicated study for Step 1. I think this is a sound idea. There are plusses and minuses to both. The idea is that if you do UWorld too early, you'll forget what you learned and waste questions. That may be true if you do the binge-and-purge thing. But if you're using Anki, you could use UWorld and make cards for the explanations and thus remember everything. UWorld is as much a learning tool as a testing tool. There is stuff in here that might only show up in Big Robbins or maybe not even. So using UWorld and Anki together makes sense. This is what I do in fact. I've been going through the q's and making cards for things I didn't know so that they're in my review deck.

Kaplan Step 1 Qbank - Another USMLE q bank. I haven't used it. I might though. Word is that the questions can be esoteric at times. Not as high quality as UWorld but people use Kaplan while they're learning so that they don't waste the good stuff (UWorld) before they're Step 1 review time. This isn't a bad idea.

USMLERx  - Questions made by the First Aid authors. It's like being pimped on First Aid. I don't think it's bad. I haven't used it much.

Video and Audio Lectures
Pathoma  - See Above

Online, Free Resources
Web Path
WashingtonDeceit (the Shotgun Histo guy)
Pathology Guy

Putting It All Together - An Optimal Sequence
So we've covered the methods and the sources. Now how do we put it together? What is the optimal way to learn pathology?

As I've said before, I think learning complex subjects like Pathology is best done in layers.  What I mean is, first you want to get an aerial view and become familiar with the landscape. Then fly a little lower in see some detail. Then, zoom in even further and examine the nitty gritty. Studying this way makes the learning more meaningful and effective. If you zoom in to the little stuff right from the get-go, you tend to lose the forest for the trees. That's not good. The detailed stuff makes much more sense and sticks better once you've mastered the big ideas.

I'm recommending here my preferred sources. This is basically what I did. As I said before, when I was first learning, I didn't have the benefit of Pathoma. So my experience has been to back track. Everything that I'm suggesting I've done myself. I just didn't do it in that order. The order I'm giving is the ideal one that I would have followed if I had known better.

Step 1 - The Aerial View
- Watch the relevant Pathoma video(s) at 1x speed
- Follow along in Fundamentals of Pathology
- At the end of the video, make Anki cards for every nugget in Fundamentals
- Cross-reference with the corresponding section of First Aid and make a card for anything not covered in Pathoma
- After making cards, immediately review them

Step 2 - Zooming In
- Read the same material in Robbins Basic Pathology
- Capture any high-yield/relevant factoids that you didn't get from Pathoma. Use your judgment here. You need to decide what's worth remembering and what's not. This is an art that is learned from experience.
- After making cards, immediately review them.

Step 3 - The Microscope
This part is optional. If you're the sort of person who really likes fine detail (or perhaps your class exams require this), you can go a step further.
- Read corresponding section in Big Robbins
- Capture factoids that you didn't get from any other source
- After making cards, immediately review them

Step 4 - Refining
- After having completed an entire topic or chapter (say Cardiac Pathology), use USMLE World or Kaplan or USMLE Rx. Use the questions to help you integrate all the information you've learned.
- Make cards for the ones you get wrong and/or the factoids that you don't already have in your set

You might be wondering when you should be doing all this. My advice is to spread this out as much as you can. So for example, I would do Step 1 on Monday. This should take you the whole day. If it's a particularly large chapter, spread out Step 1 over two days.

On Day 3 do Step 2. Then on Day 4 do Step 3. Take a break on day 5 or catch up on any part of the prior steps that you didn't get to. Remember that you're going to have all of your prior Anki reviews from other subjects coming due too. So make time in your day for that.

Then on the weekend, you can do Step 4 and do some integration questions from UWorld or elsewhere.

How you carry out this sequence will depend a lot on your class requirements, your own learning pace and style and your time commitments. If you notice, I say nothing about studying from lecture slides. That's because I don't think class material is particularly good. If you go to a P/F school, you can follow my suggestions closely. If you don't do P/F, you'll need to figure out how to satisfy your class requirements. You might have required attendance or something like that. I want to be clear. PASSING YOUR CLASSES IS THE MOST IMPORTANT THING. This is just the reality of medical school. Although we'd all probably love for med school to get out of our way so we can learn medicine, the fact remains that passing your med school classes is necessary. So don't jeopardize that by completely disregarding class if you don't have that luxury. With that said, I'm certain that every medical school is in some way, shape or form covering the same material. Robbins is what your professors use to make their lectures. So if you do as I suggest, you're not going to miss anything. But what is emphasized by your class and what is emphasized by Step 1 are often not one and the same. Just keep that in mind.

Goal Setting

In all of this, you need to decide what you can handle and what you want out of your education. School is tough, and sometimes for all the wrong reasons. Mandatory classes, clinical skills  classes(i.e. "tell me about your feelings"), busy work, and sundry other obstacles get in our way when trying to learn. That's just the way it is. So be realistic about what you can achieve.

I think a necessary, feasible and totally effective goal would be to cover First Aid and make cards for all of First Aid. That is the minimum goal. First Aid is not everything you need to know for Step 1, but it's a very large chunk of it. At the end of your preclinical courses, wouldn't it be great to have at least 50-60% of what you need to know down cold? If you aim for knowing 100% of everything, you'll get discouraged and likely quit this whole project and remember even less. Maybe like 20%. Be willing to compromise and be flexible.


My Pathology Deck and More
Disclaimer: I am sharing this deck because I want help other medical students learn pathology. I think making one's own cards is a valuable exercise and is superior to using someone else's, but I recognize that for a lot of people, this task is too much to handle. It's enough to turn people away from using Anki all together. So, I'm offering my cards as a compromise. If using my cards helps people learn Pathology and it encourages use of a system I really believe in (Anki), then I'm a happy man. I put hundreds of hours into building this deck but I see no reason why I should be the only one to benefit. I would have had to make these cards regardless, so why horde them all for myself? All I ask is that you give credit where it is due; if you share these cards with your classmates or anyone else, just let them know where they came from.

Also, I am almost certain that there will be some errors in these cards. There are over 6000 of them. I've done my best to ensure the accuracy and quality of my deck, but some errata have certainly slipped through the cracks. This is true of every learning resource, from First Aid, to Robbins. Please double check the material and I welcome you to post in the comments section any errata so I can fix my own deck. I'll periodically update the files with those improvements and any additions that I make.

A word on the content... This deck is nearly complete. It's covered all the pathology of First Aid. There are tidbits from Pathoma for things not covered by First Aid. There is a lot of Goljan. There is also material related to Big Robbins and from USMLE World. The only subject that needs a little more work is Female Pathology. That's about it.

Feel free to add your own cards to this deck once you take it down from Dropbox. If you feel so inclined, share those updates here by posting links so other people can benefit.

Happy Studying!

Update (04.24.13)
The Pathology deck has been revamped and updated, and is now for sale here on the site. You can find it here: DrWillBe Pathology Anki Deck

All other decks are still available for free. You can find them here

Study Skills

What I used to do
When I went to college, I realized that the one thing I never really learned was how to learn.

You'd think with 13 years of school behind me, in that time, someone would have shown me how to properly amass knowledge. That was a lesson I never got. Rather, like most everyone else, I just concocted my own habits, largely by copying everybody else.

So, for many years I went to class and took notes, but never did anything with those notes after the class was over. I rarely even kept the notes. I was operating under the idea that writing notes in class is a good way to learn because just the process of writing things out will implant the knowledge firmly in mind and trap it there. Wrong!

My main mechanism of obtaining information was reading books. I did truly like to read, especially later in high school. That was how I absorbed information, and at least for the level of high school coursework, reading a textbook once over was sufficient to allow me to perform reasonably well. I remember cracking open my AP World History book for the first time a few hours before my AP World History test and just cruising through it. I remember almost nothing of that futile exercise except that I got the highest grade possible on that AP test. So, lesson is, cramming works, but the information you 'learn' - if it can even be learning - is so fleeting. Its in one ear and out the other. Does this sound familiar to anyone?

As a freshman in college, I realized that I would need to amend my methods, but what could I do? I went to class diligently, took notes, reviewed them sometimes. I read the textbook before class and again after. Before big exams, I'd re-read whole chapters and do more problems. I had abandoned the days of cramming, and I was much more attentive to the time frame in which I needed to operate. I was studying as well as anyone could, I convinced myself. My grades were quite good, so I figured this was the best I could do.

The summer after freshman year, I was taking a general chemistry lab course at Columbia and I was struggling to recall much the stuff I learned just a few months earlier in Gen Chem course. Did I really spend all that time and money just to forget most of what I learned? Sure, I remembered the broad concepts and could apply them, but what about the finer details? One might argue that for the fine details, you can go to a book. But that's inefficient, especially the way I was accustomed to doing it, re-reading whole chapters in a futile attempt to get that information to stay put.

There had to be a better way. I needed to learn how to learn.

Learning How to Learn - Cornell Notes

In the summer of 2006, a few weeks before sophomore year was about to begin, I was in the local library scanning the stacks for books on study skills. I had never thought to read about studying before, but I wanted to see what kind of methods others had devised. My goal was to learn for the long-term. How depressing it was to think about the time, money, and effort I would put into the rest of my college career only to lose most of that to forgetting.

I remember the day vividly. I looked on the book shelf and there was a title calling me.

How to Study in College by Walter Pauk (HTSIC)


I immediately went to the nearest Starbucks and devoured the 200-something pages of that book in one sitting. Yes, I'm a huge nerd.

So what was inside that was so wonderful?

A couple of things. Several chapters were dedicated to time management. I had never managed time. Not methodically at least. I would plan in my head for the next few days ahead, but I never ever followed any kind of set schedule. That was the first thing HTSIC told me I needed to do. And so I did. I found the useful program iCal on my computer and began arranging my life.

6 AM: Wake up
6:15 AM: Breakfast
7:00 AM: Study Organic Chemistry
9:00 AM: Go to class
....
6 PM: Free time (yes, I scheduled free time)
7 PM: Gym
8 PM: Dinner
9 PM: Study class notes
11 PM: Sleep

I found this process of planning my day just as liberating as the book said it would be. I didn't need to waste time thinking about what I'd do next, and I didn't need to worry about not having enough time because I could plan well enough in advance to prevent crises.

Did I stay exactly on schedule? Well, I tried. In life one has to be flexible. I often fudged my schedule and maneuvered blocks of time to conform to what I was actually doing. Study sessions would go longer, dinners longer, sleep shorter, gym completely tossed. But on the whole, I was able to stick to the broad schedule +/- an hour or two. So this was really helpful.

But the best and most important thing from that book was not time management. It was Cornell Notes and the Q method.

The author of HTSIC, Walter Pauk, was a study skills instructor at Cornell. Based on evidence from educational psychology in the early post-war era, Pauk came up with the Cornell Notes method.

Here's a summary of what the method consists of.

(1) Take a plain piece of paper and make it look like the one above. That's Cornell Note paper. I generated mine on MS Word and then kept that template for the rest of college. Pretty simple.

(2) In class, take notes in the note taking area. Keep them concise and clear.

(3) Immediately after class (or at your earliest convenience, but no later than 24 hours after the class) take the most important bits from the body of notes and write 2-3 sentences in the summary box.

(4) Questions in the margin. You write questions pertaining to the details in the note box. Check out the example below.

The questions should be as direct and clear as possible. In essence, you're making mini-tests on each page of your notes.

(5) RECITATION. This is the most important part and what makes the Cornell Notes so effective. Cover up the details with a blank piece of paper but leave the question exposed. Proceed to recite aloud the answer to each question. If you get it right, move to the next. If you get it wrong, do it again.

Does this mean you have to talk to yourself? You betcha! It's weird, yah, but the benefits of doing this far outweigh any embarrassment or uneasiness you might get from talking to yourself. Pretend you're teaching your imaginary friend.

(6) Review. After you're done reciting, skim over the notes again, eying the summaries.

(7) Reflect. When you're done reciting and reviewing, you should think about what you've just learned, connecting it with you already know, integrating the information into the conceptual framework you've already built up on related material.

I began using Cornell Notes sophomore year. I used it in class, when reading textbooks (modified method), when reading everything. And it worked so well. I was able to recall myriad details from my notes with ease, and not just in the first few days after learning them. Weeks, months later, the information was there still firmly implanted.

So why is Cornell Notes so effective?

A couple reasons I think. The most important part of all this is the recitation. That is what sets Cornell Notes apart. Devising and answering specific questions immediately after learning new material gives you instant feedback. You know if you know it and you know if you don't. If there isn't some testing mechanism to assess how much we've retained, we can very happyily read our notes or read a book chapter and feel that we've got it all in our minds. But then, when we are pressed to recall specific information, such as on an exam, you realize that you really didn't have the nitty gritty in there. The recitation and questions gets around this problem by pinpointing precisely what you know and what you don't.

The recitation also makes the learning an active rather than a passive process. You're not just receiving knowledge. You're repeating it, as if you were teaching. People often say that the surest way to learn something is to teach it to someone else. When I was doing my recitations, I felt as if I were teaching an imaginary audience.

I also think there is something about reciting aloud that really makes the stuff stick. You add another sense - hearing - to the learning process that solidifies the imprint that new information makes on your mind.

The anticipation of knowing that I'd have to answer questions after reading my notes or a book chapter made me more attentive to the details. I didn't just gaze over text on a page. I read with a purpose, and I think that primed my mind for maximum uptake.

Another major benefit of Cornell Notes is that now you have a systemic way of studying in the future for an exam or whatever. No more re-reading whole book chapters. Or going over all of your notes. With the questions and summaries, you've got all you need. The weekend before an exam, I'd go straight to the questions without looking at the details. In this way, I avoided having to read over all my notes again. Questions I got right I didn't need to review. I only looked at the details to questions I didn't get right. Rinse, wash, repeat.

I aced my exams, was able to recall most all of what I learned, and all this made me enthusiastic to learn more.

That's a lot of work

An objection that people have to studying this way is that it seems like a lot of work. Compared to what? It's a lot of work to read over all your notes a bunch of times because they just won't stick. It's a real time killer to go back an re-read textbook chapters before an exam. And it's even more time consuming to have to go back an relearn 75% of your course material a year or two down the road when you take a more advanced course that presumes you remember what you learned at the lower level.

You've go to put you're time in. Learning isn't easy. Cornell Notes is front loaded. You put the effort in at the beginning, but after you've made your notes, it's easy sailing. You just recite, review, recite, review. And best of all, it sticks for good.

I know I sound like a zealot about this, but I'm just so grateful to have learned how to learn early in college. I have a system that works, that helps me learn for a lifetime. So naturally I want to share it with others.

In Medical School

Given the effort needed to study like this, I was a bit concerned that I couldn't make it work in medical school. But, thankfully, I can report that it has worked remarkably well, just as it did in college. I've tweaked the process a bit to accommodate the fact that all our learning is now digital. I don't use the Cornell Note paper. Instead, I've been annotating PDFs from class directly and then covering the details and reciting. It works like a charm.

However, I have not been able to review as much as I should. In the past, for me the magic number was three times. That was how many times I needed to recite the information before it was firmly embedded in my memory. And, the greater the span of time over which those reviews were spread, the better the retention. That's something I realized from the very beginning. Three reviews in rapid succession (say, in three days), is not as good for long term retention as three reviews spread across say 3 weeks. Which brings me to my next point.

Spaced Repetition

Spaced Repetition is revolutionary study methodology based on a common sense insight: to remember information for the long-term, you need to review that information several times and those reviews need to be spread out at appropriate intervals. If you review several times in a short period (say, 5x in 1 week) you may remember that information for the short term, but not for the long term.

We all know that the key to memorization of facts and information is frequent review. But how frequently? And at what intervals? Are there optimum times to review such that you'll be most efficient with your time and most effective in embedding the material deeply in your long-term memory?

People have thought about this question for quite some time. Educational psychologists, beginning with Hermann Ebbinghaus in the late 1800's, have tried to understand why we forget and how to combat it. An excellent article in Wired Magazine about spaced repetition discusses Ebbinghaus' early experiments and the line of inquiry that has followed since that time.

[H]uman forgetting follows a pattern. We forget exponentially. A graph of our likelihood of getting the correct answer on a quiz sweeps quickly downward over time and then levels off. This pattern has long been known to cognitive psychology, but it has been difficult to put to practical use. It's too complex for us to employ with our naked brains.

So basically, we forget things exponentially. The time to forgetting after each review increases, and if we knew when forgetting was about to happen, we could review right before that time. If you review too early, you don't maximize the the time you get until the next time you need to review. The graph above summarizes the effect of appropriate spacing.

But how do I know when I should review?

That's where Supermemo and a string of spin-off computer programs come in.

SuperMemo is based on the insight that there is an ideal moment to practice what you've learned. Practice too soon and you waste your time. Practice too late and you've forgotten the material and have to relearn it. The right time to practice is just at the moment you're about to forget. Unfortunately, this moment is different for every person and each bit of information. Imagine a pile of thousands of flash cards. Somewhere in this pile are the ones you should be practicing right now. Which are they?

Supermemo was created by a remarkable computer scientist and learning afficionado named Pietr Wozniak. He, like so many of us, was totally distraught by the fact that he studied hard in college and as a graduate student, and he'd learn, but the stuff he learned would rapidly fade from memory after a period of time. He harnessed his frustration to try to find a way to not forget. To that end, he came up with a program that makes an algorithm that predicts when you need to review certain information. Basically, it's a flash card program, but with a twist. When you answer a flashcard, you rate how difficult it was for you to recall the answer. Based on your ratings, the Supermemo algorithm will schedule your next review of the card.

Supermemo has quite a following on the net and with some learning enthusiasts, but it hasn't caught on the way you think it might. This is such a game-changer, you'd think everyone would be all over it. Here is a way to learn things for good. The legions of people that have used Supermemo to learn just about everything - commonly languages, but a slew of other topics - swear by it. I didn't know about Supermemo or spaced repetition until this year when I learned about one of its spin-off programs, Anki.

The beauty of these programs and the spaced repetition method is that you are being maximally efficient. When we make notes or review cards, the reason most of stop looking at them is because we just don't have the time to completely review old stuff while new stuff just keeps coming. So we don't review the old stuff, even though that is what is necessary in order to not forget. Programs like Anki and Supermemo use their algorithms to only present to you cards that need to be reviewed, not every single card in the deck.

For example, let's say I have 100 cards in my deck that I've generated from class notes. Then I add more and more cards as I learn new stuff. So what happens to the 100 old cards? I need to review them, but how can I review 100 cards while adding more and more every day? The key is that you don't review them all on each day. Those 100 cards will be spread out in the future based on how you rated them the first time you used them. The key is daily review so that the cards don't pile up. So a year from now, when Duke 1st year is long over, I might have thousands of cards. I need to still review them, but I won't have time to review 100's of cards a day. Thankfully, I don't need to. Maybe 20 cards will be in my review deck every day, based on how I rated them. I'll easily be able to handle those many cards on a daily basis. If not for the algorithm that informs the spaced repetition, there'd be no way to review all those cards in a timely way and in a way that commits the info to memory.

So what about the Cornell Notes method?

The question-recitation component of Cornell Notes is still very effective. I'm not throwing the baby out with the bath water. I'm convinced that review and recitation is critical for learning. But in the past, I had no methodical way schedule for review. I'd review my notes and do the recitation arbitrarily - maybe 3 times before a test, at no particular interval. Anki and the spaced repetition algorithm will just help me know when to do my review and recitation. This is an enhancement to an already excellent system.

I'm very excited to experience the results of this new and improved study strategy. My goal is to build a permanent foundation of knowledge. I absolutely hate the idea of spending all the time and effort this year learning all this medical knowledge just to forget 75% of it by next year. And then, while doing clerkships, I'll need to constantly backtrack to re-learn everything. If I could just do it right the first time, that wouldn't be necessary.

Also, since I'm going to have at least four years during my PhD when I'll be away from the clinic, I need to have a way to continually review the material I will have learned by the end of second year. With Anki and the spaced repetition algorithm, I will be able to do that easily. How much work is it to study 10-20 cards a day, every day? 10 minutes max. Slow and steady wins the race I say, and that's how I like to study.

Making those cards takes a lot of time, you might be saying. And you're right. Making the cards does take time, but in my mind, it's totally worth it. The workload is front end, but then I can use the cards for years to come. And I can modify them if need be, and I can search them easily with the programs available.

If you don't want to make the cards but you want to reap the benefits, there are options.

What actually got me on the whole spaced repetition bandwagon is a USMLE Step 1 resource that my good friend Theo at U. of Washington sent me the other day.

It's called GunnerTraining (Funny name, I know). GT was founded by some Harvard residents in 2009 and has quickly become a favorite of med students across the country. I count myself as one of them. GT is a web-based knowledge database that uses the principles of spaced repetition to help students study for the Step 1. It's got flashcards with snippets of high yield knowledge.

The process goes like this:

(1) Study some flash cards
(2) Add pre-made review cards to your personal review bank
(3) Do an immediate review of the cards and rate their difficulty
(4) GT then schedules when you next need to review those cards



GT generates a daily schedule for you to study those cards. Just come back when you have review cards. At the same time, you can move forward and study more cards and put those in your bank, repeat the process. So you're review old cards and learning new ones at the same time, and you're doing it in a way that optimizes the use of your time and your ability to remember your knowledge.

The caveat for all of these SR-based programs is that they are for the long term. You could use GT's resources in a month, but you'd lose most of the benefits that SR offers in terms of long term storage. I'm so happy I found this in the beginning of 1st year. I can do a slow and steady review concurrently with my coursework. I can study the same topics so that I'm not only studying for Step 1 but I'm also reinforcing things I'm learning in class. Two years from now, hopefully I'll be a Step 1 machine : )

So that's what I'm up to these days. I'm passionate about finding new ways to learn and about sharing what I learn with others. I hope someone benefits from this post. If you have any of your own methods that you want to share, please go ahead and start a conversation on this post.

Happy Studying.




UPDATE (12/17/11): It's been more than  year since I wrote this post. Some things have changed. Most notably, I no longer use Gunner Training. I think it's a fine service for many people, especially those who don't want to make their own cards. But I find card-making a valuable experience, and Anki, in my opinion, is a far superior platform that can be used offline, on multiple devices. And I can add my own stuff. And did I mention it's free? And I can add media and such and not be worried about copyright issues. I'm not knocking GT, but I just want to be honest about what my studying actually consists of. There is also a level of redundancy too. There aren't enough hours in the day to do everything. My Anki load is demanding enough as it is.


I'm going to write a more thorough update in a couple days, but for now, that's all.


Crush away :)