Sydney Dolan

This post is meant as a helpful guide to provide tips about reading academic papers. Disclaimer that I obviously didn’t invent reading, I don’t claim to dictate the standards of how or how much anyone should read. In this post, I am purely trying to be helpful with academic reading advice, as there was a period at the start of my PhD that paper reading left me feeling very overwhelmed and stupid. Part of that stupid feeling might be something that happens to everyone when they start research, I also would like to hope though that reading tips and more clarity around academic reading expectations would alleviate that feeling a little bit as well.

When I need to read an academic paper, my reading process is very iterative. For papers that are in my field of study, I will be reading them multiple times to understand them. But the type of reading I am doing each time varies as I determine the relevancy of the paper to my research. For all papers I start with an initial 5 minute reading, which is a quick scan of the paper to determine the contributions. This level reading helps me determine if the paper is relevant to me, and if a more thorough reading is required. Then I proceed with a medium pass of the paper, where I am now more interested in developing a more technical understanding of the methods and results. At this point in reading, I am also taking more extensive notes on the paper. The point of a medium pass reading is that I could answer questions and would remember the paper if asked, but that there will still be some conceptual elements that I am missing. If the paper is a new technique to me, or very relevant to my research project, I then move to an in-depth reading. In this reading I am thoroughly reading every single element of the paper, and trying to replicate some of the results on my own. These in-depth readings will likely take multiple days and involve some sort of experimentation or replication of results on my own, independent on the research process. Given the time it takes to do each, factually there are just many more readings I do at a 5 minute high level than in the full in depth level (reiterated by figure below).

More specifics about the reading techniques I am using in each reading type, and what I look to do in each type of reading, are listed below with specific example documents linked.

The 5 Minute Pass Reading

What this type of reading serves to do: In this type of reading, you are scanning the paper to quickly determine the specific contributions and unique angle of the paper. In a 5 minute pass of a paper, you should be able to name the methods that are used, but you will not be able to answer more deep theoretical questions. A quick reading (hence the suggestion of 5 minutes max) is only meant for you quickly characterize the paper and make a judgement call if you want to read it again more thoroughly.

Where to read: The abstract in its entirety, the second half of the introduction (typically, this is where the contributions are listed), a high level pass of the methods (the headings, any figures to describe the method, any sections that talk about specific methods), a high level pass of the results (read the figures and tables to understand the performance improvement the authors are highlighting). An example of the sections I personally read in a 5-minute pass of a paper can be found here. An extended walkthrough of the things I am paying attention to in this specific example is at the end of the post.

Why you would stop reading the paper here: After identifying what method type the pair is using, you have learned the the paper is not relevant to your field of study (either in application or in method). For you to make this judgement call, it means you have to understand the method and problem motivations of the paper well. If you have never heard of the method, or if you have never learned anything about it previously, it is probably best to advance to the next level of reading (medium pass).

If you are just starting research in a certain topic area, likely you will be doing more thorough readings. You should be seeking to understand the field and learn the papers in your area, not just what is the one paper or two papers you are building off for your research project. As you build fluency in your field of study, your ability to scan and make quicker judgements and understandings of papers will improve, and there will be more papers that you can rely only 5 minute pass readings. 

The Medium Pass Reading

What this type of reading serves to do: Once you’ve completed the 5 minute pass and you are going to read the paper again, the goal of your second reading should be to more concretely build a mental model of the papers methods. In this reading, you are wading in greater technical depth and now have to connect different elements of the proposed design with the results. This requires you to pay attention to the structure of the methods. In this reading, you are taking notes to refer back to, but these may not be extensive for every section of the paper. These notes should be sufficient such that you can re-read them, remember the paper, and be able to discuss the paper a little if asked.

Where to read: abstract again, full introduction this time, read the methods and track the notation, read the results section writing and not just the figures, read the reference titles of relevant papers (and give these associated papers a 5 minute pass). An example of the sections I personally read in a medium pass of a paper can be found here. There is also an example at the end where I show a specific paper and what I am reading when I am going through a paper more thoroughly.

Why you would stop reading here: You now understand the method, but you don’t want to implement it. You can answer the majority of theoretical questions about the methodology proposed, but you never tried to replicate the work yourself, so there will be certain elements missing that just never got discussed in the paper. This sort of reading serves to improve your understanding of the field. While you may not implement the specific proposed method, you should have the technical understanding to grasp the method selection (i.e. why the design exists, how it was implemented, how it addresses gaps in the field).

What does it mean to be understanding something? It means you understand why a method is selected, and that you cna similarly formulate methods in that technique. It requires you to have taken a class before, or have done the method, for you to claim you know it, but since youve done it before, you dont have to do it again. Escalating to an indepth read is required if you have never tried out a method before or taken a class or ever learned about it, otherwise your understanding is surface level

Sometimes you need to read papers just to understand what exists in the field (maybe a collaborator just published a paper you should be aware of), now youve improved your understanding such that future papers in this area may only stay in the 5 minute reading section.

The In-Depth Reading

What this type of reading serves to do: You are trying to understand every detail the author mentioned so that you can replicate the paper. You are now moving beyond a conceptual understanding and now understanding certain elements of the method related to the implementation that don’t necessary get explicitly written in the paper (i.e tuning parameter, threading, computation). You are coding their work and replicating the method locally. As you are reading, you are taking detailed notes to refer back to and explain key terms in the paper.

Where to read: abstract again, full introduction, full literature review, full methods and track notation, full results, full conclusion, read the references and read the papers they benchmark against. An extended walkthrough of the things I am paying attention to is at the end of the post.

Why you would stop reading here – This sort of reading is the highest level of understanding you can have about a paper. You know every element of the paper itself well, and have confirmed the takeaways and understanding of the authors through some sort of experimentation on your own.

You cannot always do a deep dive on a paper, just because it takes so much time to replicate someone else’s work. To fully deep dive into a paper, you may even have to contact the author for questions about things that are unclear in the paper. You don’t have to do this deep dive on everything, but you should be doing it fairly frequently at the start of a research project. You will expand your academic community and you need to understand who are the key players that exist in the field. By paying attention to the universities and authors in your area, you will find other potential collaborators, other good papers, and have an easier time learning when new papers come out that you should be aware of.

Common Questions/Concerns About Reading

Below are some questions I’ve gotten from past research mentees about reading, and some brief bullet points I have of advice

How can I become a faster reader?

• The more you read, the faster you get! Read in your free time, reading doesn’t have to be academic, the fiction you read before bedtime to help wind down can also contribute to your reading abilities long term.
• Learn to read without subvocalization. Subvocalization is when you read at the speed that your inner monologue is sounding out. Websites like Spreeder allow you to copy text in and set the speed that words appear on screen such that you’re able to increase the speed that text shows up on screen faster than your internal monologue. This can help you improve your scanning abilities/reading speed.
• Guide your eyes. Use your finger to point where you’re reading. your eye will have an easier time following and you can increase the movement of your finger to increase the speed.
• You’ll notice a lot of skimming techniques are built on the premise that academic papers should follow a certain narrative structure (as discussed in this blog post), YES! This is why certain writing techniques lead to easier readability (ex. the expectation that contributions should be at the end of the introductions section, descriptive topic sentences that explain the whole point of a paragraph)
• Remember that reading for speed is not reading for comprehension. If you need to understand something deeply, take notes on key phrases, re-describe what is going on in your own words. These notes can help your future self if you have to refer back to this paper again in the future.
• Start a reading group (with peers in similar research areas). You can
  • All read the same paper and discuss takeaways. This structure helps you see how others read papers and what their interpretations of the same document were.
  • Each person read a paper and then you get to learn the lessons (medium pass) from others without having to read more. This structure is great for getting to read tons of papers.

I don’t like to read. It makes me depressed because it makes me feel inadequate and behind

• Same, welcome to academia (kidding)
• Reminder that working in academia is for the purpose of learning! We’re here to learn, not to compete against one other.
• This means that learning new things through learning papers is a good thing. Learning about what others are doing helps you build off each other. What someone else has done does not detract from your future success
• Research is hard and this feeling is normal (i.e insecurity in reading papers, feeling like all the good ideas are gone, feeling like you’ll never do something as nice)! Trust the process and trust the guidance of your research mentors. Just because there are topics in similar areas doesn’t mean your work is null and useless

Results are the most important, I don’t see the point of reading

• I see the point, the famous phrase is publish or perish for a reason.
• However, if you are truly not reading at all, you could be doing work that someone else has done before because you never bothered to understand the field and see.
• If you are never reading, you are relying on the scoping of the project/your collaborators to ensure the topic has not been done. Academia and the amount of work out there is huge, and you have a responsibility to do this due diligence yourself.

I don’t have time to read.

• Research requires us to wear many hats, sometimes you won’t get to read as much as you’d like because it is a few weeks where you are submitting a paper instead. In general, try you best, and read what you can.
• Don’t use the first bullet point to give up on reading entirely though. Track how much your reading anyways, phd thesis have upwards of 80+ citations, these are papers you have read!
• Time management in research is challenging for everyone, some suggestions I have if you are struggling to find time to read are: join a reading group, schedule time in your calendar for reading, set up google scholar alerts for authors you like and read those immediately when you get a notification.

What does it mean when my PI/research advisor tells me to read a paper?

• Typically, when someone is giving you a paper, they are expecting a medium to in-depth pass. In these cases, you should take notes and be prepared to have a conversation about the paper
• If you don’t think the paper is relevant after your first 5 minute scan, you can always ask supplemental questions about what made that paper relevant/why they selected.
• Generally, when someone is giving you a paper, they are doing this to be nice and direct you to research they think is relevant for your project and understanding of the field. I would recommend reading the papers, particularly if this is a collaborator that is expecting to work with you and may reference this paper again in the future.

Example Reading – Making Every Event Count: Balancing Efficiency and Accuracy in Event Camera Subsampling

The associated paper I am using as an example is located here. It is titled “Making Every Event Count: Balancing Efficiency and Accuracy in Event Camera Subsampling” by Hesam Araghi, Jan van Gemert, and Nergis Women from Delft University of Technology.

5 Minute Pass

This highlighted example of the things I am paying attention to in a 5 minute pass. I’d like to discuss this example, I’ll include some images of the pages I am talking about as well.

On page one, one of the first things I want to do in a 5 minute pass of a reading is confirm that the paper is addressing some problem in the area I am working in. Usually, somewhere in the first few paragraphs, there’s a more specific problem definition, I have highlighted that specific sentence (one I scanned to find) that gave me the problem definition I was looking for. As I talked about in this technical writing blogpost, introductions follow an hourglass shape, where the technical complexity increases as you read on. The initial few sentences were a more general purpose definition of event cameras, and the sentence I highlighted represents a problem description fit for a technical more aware audience.

Once I noted the specific technical problem that the paper is working on, I now want to scan the paper quickly to find the contributions the paper discusses. These are typically located at the end of the introductions section, so I jump there. The description of their solution and what they note is new about this analysis is on page two, and I read this specific section closely to understand what elements the paper is calling out as novel or noteworthy.

With the contributions of the paper listed, I am now trying to find the corresponding sections in the methods and the results to confirm these contributions/understand the extent of the contributions relative to the baseline. One of the contributions mentions a comparison of different types of subsampling methods, so one of the things I want to determine is how they define these subsampling methods. I see the heading subsampling types and I go to read this section to get a little more information.

On the next page, I see this neat little figure, which is designed to explain more about the different types. I use this to help inform my understanding of contribution one listed in the introduction.

I skip reading about the specifics of each method, because this high-level description is enough to understand the gist of what the author is saying. I see on the following page, theres a dataset description. Coming from an ML or RL background, I always read this because it informs what specific application and problem, and existing body of work that this method is comparing against.

On the following page, I am interested in looking at the results. For a 5-minute pass reading of the paper, this means paying attention to the tables and figures presented. If the figures aren’t explainable on their own, I also check the associated topic sentences to see the main takeaways the authors have for the paper.

On the final page, I also read the discussion that reiterates the main points of the full paper. Given the context of the contributions, what I observed in the results, and the points they highlighted in the discussion, I now have a quick working model of the paper.

Medium Pass

This highlighted example shows what changed relative to the 5 minute pass. On my second reading of the same paper, I am now going to go much slower, and pay much more attention to the specific methods and results. When I go back through the introduction a second time, I am now reading it more thoroughly, particularly how they are scoping the problem. On my second pass of the paper, I noticed that Figure 1 was actually really effective, and in my first pass of the paper, I missed reading it. This is the downside of 5 minute passes. You will miss important context that improves your understanding of the paper. Without a doubt, you will find that you missed something important as you go through the paper a second time.

The next big difference between a 5 minute pass and a medium pass is that I am spending a lot more time on the method. For me, I am already familiar with this topic area, so in a paper-specific case, I am skipping the related works section, as I have read many of those papers before. The methods section contains a fairly big difference from the 5 minute pass, now I am reading the entire methods section much more closely. Concurrent to the sections I am highlighting in purple, I am also taking notes on my own notes document where I am trying to explain the methods section in my own words.

In the experiments section, (aka the results), I am also being as thorough. Instead of trying to get the high level gist (i.e. this method outperformed others), I am also taking note of interesting phenomena that the authors mention that aren’t captured by the figures. In my personal notes, the results section is a dialogue. It’s both what the authors note about their papers, and in my personal notes I will also write my opinions (i.e. it was interesting this occurred, it was surprising this occurred, I’d be interested in XX as future work). Note taking isn’t just a summary, it’s also about developing a connection to the paper so that you can remember it when someone asks you about it, and that you can relate it to other methods. You will remember your own opinions and, this will help you remember these papers if you ever have to speak about them

Another thing I want to call attention to in the medium pass example is the references section. Reading the results section, I realized citation [19] showed up a lot. Thus, I am going to check out that paper as well to give it a 5 minute pass (at least). The papers I check out in the references, are a mix of papers that were cited a lot in the paper (like [19] which is another method that was compared against, and papers that seem topically relevant to this specific paper. I made the decision on what papers to read on a few things. First, in this references section there are papers that I am already familiar with [2],[14]. There is no reason for me to re-read these papers, I remember them. The next consideration in what papers I choose is title and venue. [3], [8], and [17] all stuck out to me based on their title. I recognize that [17] is from ECCV, which is a quality computer vision venue, and thus that adds credibility to that specific paper and reiterates that the topic matter is something in my area of interest when reading this paper.

In-Depth Pass

In an in-depth reading of a paper, I am reading everything in the main paper, so I don’t really see a point in highlighting specific sections. I will read everything. The notes go from a more haphazard “interesting idea on X” style of note-taking, and instead, I am actually replicating pieces of the method or trying to understand the associated proofs myself. For me, sometimes it’s helpful to make presentations explaining key concepts to myself, and then when I baseline against other papers, I already have some explanatory graphics that explain the premise of other techniques.

On the references side, there will be more papers I read from the references section. However, this is dependent on the specific things I am reasoning about as I try to replicate the papers reasoning and results. Thus, it’s not as simple as just reading the titles or checking the venue of a paper. Additional papers I am reading at this point are to help my conceptual understanding, and thus it’s not easy to pick them out initially by some silly rule.

Final Thoughts

This guide is meant to provide some tools to help with the reading process and differentiate between the different reading skills that exist when reading academic papers. Hopefully, through the definitions I provided and examples, you learned some techniques or skills to help with scanning or clarify what it means when someone asks you to read a paper, and how to approach it.