If you would like to have a successful career in science, your academic success will be judged regularly:
- Whenever you submit a research proposal, your academic accomplishments will make up a part of the evaluation score.
- When you submit a paper to a prestigious journal, the editor will check your publishing history.
- During the annual performance review, your boss will look at your academic achievements.
- And obviously, whenever you look for a new job the selection committee will scrutinize your academic CV.
Being aware of the criteria that are used for the evaluation of academic success will allow you to work on improving them. Thankfully, barely any profession defines success as clearly as we do it in academia.
Here is a list of academic performance indicators, divided in three groups based on importance:
- Primary academic success indicators include research quality, h-factor, number of publications, prestigious awards, important patents, and secured research grants
These are the things that academic employers value the most. Focus on polishing these performance indicators to make your CV stand out.
- Secondary academic success indicators include Invited presentations, student review scores, consulting requests, ranking of the journals you publish in (unless they are in the territory of Nature), and Altmetrics
These performance indicators serve to show your growing reputation as a scientist. These academic success measures could be a part of your CV, but probably will not be the main highlight of it.
- Tertiary academic success indicators include the number of paper downloads, reviewer invitations, number of followers on professional social networks, number of visits to your professional website, and personal interactions.
Tertiary success indicators are useful because they offer early feedback on the possible future success of a scientist. For example, it might take years before starting to accumulate citations for a paper but a high number of paper downloads is likely a signal for many citations a year or two from publishing. Similarly, many likes from peers for your social media status update about an ongoing research project is a sign of interest that the research results might later have.
The figure below summarizes the three levels of academic performance indicators.
In this article, we will look in-depth into the meaning and use of these scientific success indicators. I will also offer you tools that can be used to measure them.
Research quality
Research quality, by definition, has to be the No.1 item in the list of scientific success parameters of a researcher. To execute a high-quality research project, you will have to choose an important subject, develop a good hypothesis, come up with a thorough research plan, and devise an appropriate methodology for finding trustworthy answers.
The problem is, how to measure it? “Research quality” is a little abstract and fairly subjective therm. Only true experts on a particular topic would be able to give some indication of the quality of a research project.
Instead, the scientific community often reverts to academic performance indicators that can be measured much more easily, like publications and the number of citations.
Publications
Getting your papers published in well-recognized peer-reviewed journals is an important measure of academic success (in some science fields monographs are more important). This means you got past the gates of your co-authors, the editor, and the reviewers. When your paper hits the daylight, go treat yourself! You deserve it.
Getting published is especially important in the early stages of research career. I explain why in this article.
Number of citations
Publishing is one thing, but the question is, whether someone besides the peer reviewers actually reads your papers. One estimate suggests that 82% of all published papers in humanities are never cited. In Natural sciences, the situation is better, with no one referring to 27% of the published papers. Receiving citations to your articles gives some proof that other researchers care about your papers and your work is having an impact.
h-index
Once you have published multiple papers, there is a new success parameter to consider: h-index. It is considered by many scientists the most important indicator of academic performance.
The h-index is calculated as the number of publications for which the author has been cited at least the same number of times. For example, someone who has published 8 papers of which 6 have been cited at least 6 times will have an h-index of, you guessed it, 6. Look at the figure to see an illustration of this principle.
Want more citations?
Get the LEAP cheat sheet and start writing better research papers.
Research funding
For an established researcher, the amount of acquired research funding is probably the most important quantitative parameter of academic career success.
Funding not only keeps the boss happy on the payday. It also allows you to do better research, because of being able to buy new equipment, hire help, and realize your ideas. All of this will contribute to other success indicators, including publishing more and better papers.
Research Project Canvas is my secret tool for writing winning research proposals. See how to use it and get the template here.
Awards
Awards from prestigious organizations serve as a third-party approval of your academic success. Be active in applying for them and once you receive one, highlight them in your job applications and research project proposals.
Patents
Patents show that you have original, useful ideas. Interest in your patients from companies demonstrates that these ideas have a practical application. Furthermore, patents also give evidence of your originality and therefore serve as an indicator of your academic performance. Texas A&M, for example, included patents in the tenure track evaluation process in 2006. Note: the importance of patents strongly depends on the research field.
Job offers
The search committee of a tenure track position will carefully evaluate all your academic performance indicators. Of course, you will not be looking for a job that often. But to know your chances, check out the CV of someone who recently got appointed in a position similar to what you are aspiring to reach. Compare their CV to yours to estimate your own chances in the job market.
Measuring publication count and h-index
Scopus, Web of Science, Google Scholar, and PubMed are the key databases of research papers. All of them also provide a citation count for each paper. Searching for a researcher’s name in these databases will return statistics about this person’s citation count throughout the years, h-index, and possibly some other parameters.
For some of you, it might already be enough to use these search engines to find out your academic performance. But registering a personal profile on certain websites will make it easier to find and follow your key academic performance indicators. Not only that; some of the tools we will look at also calculate secondary performance indicators, like downloads of your papers and the number of views they attract. Let’s look at the key tools that can do this.
Get more citations
Writing impactful papers is the the main prerequisite for getting cited. My book offers a writing system called “LEAP”. The system not only helps to write efficiently; it actually allows to structure your thoughts about the research.
Google Scholar
Google scholar is probably the No.1 search engine for academic publications. But it also holds information about any author who has cared to register a profile in it.
As you can see in the example from my profile, it shows the total citations, citations per year, the h-index, and the i10 index (number of publications that are cited at least 10 times).
Registering a Google Scholar profile will not only allow you to check your academic performance; it will also increase your visibility on the internet. When someone Googles your name, this profile will likely show up high in the ranking. I will write more about why this is important in my book on presenting research.
A quick tip: Google Scholar also allows following your selected scientists to receive updates about their publications. Following yourself will mean that you receive an email when someone cites your work. Admittedly, like many other scientists, I have done it. It is a flattering feeling to know that someone cares enough about your work to cite it. Besides, since my work is cited in papers on similar research topics to mine, from time to time I stumble upon papers that I read myself.
Mendeley
Mendeley is an automated reference manager does the job great (I wrote about it in this article). It was acquired by Elsevier in 2013 so it is integrated with the Science Direct database (also owned by Elsevier)
Mendeley offers some very interesting statistics about the performance of your papers, including the number of citations and views. You can see a screenshot of my profile below. A registered profile is required to view your own academic performance.
If you click on any of the articles on Mendeley, you will get more information about the citation statistics. One particularly interesting field is the number of readers. This shows the number of researchers who have added the paper to their Mendeley library. Since Mendeley is one of the most used citation managers, the reader count gives a good indication of the overall interest in your paper. As such, this can serve as early feedback for the relative number of citations that the paper can later expect to attract. Because Mendeley is owned by Elsevier, the reader statistic is limited mostly to the papers that are published by Elsevier.
Publons
Publons was originally a website that tracks and showcases peer review and editorial contributions for academic journals. In 2017 it was bought by Clarivate Analytics and since then it has evolved as a more general website that tracks the performance of researchers and research papers. The statistics dashboard looks like the one below.
To explore all that Publons offers, you will have to create a profile. But even without having one, you can search for your name (or the name of another scientist) and see your main academic accomplishments.
Clarivate Analytics (the owner of Publons) also owns Web of Science so you will notice that the emphasis of this tool is on papers that are indexed by Web of Science.
InCites
Citing habits vary a lot between different research fields. For example, the average citation rate in molecular biology is about four times higher than it is in mathematics. It would not be fair to compare the citation statistics researchers from these two fields using the same scale.
Clarivete analytics offers a tool called InCites that shows the citation statistics in different science disciplines throughout the years. Go here to look for citation rates in your own field (you will have to log in from a university that has subscribed to the service).
iCite
The US National Institute of Health has also recognized the problem and developed a Relative Citation Ratio (RCR). The RCR normalizes the citation rate of a paper relative to other papers in the same field. As such, it allows comparing the performance of researchers in different disciplines.
iCite is a tool that calculates the RCR. The tool works with the PubMed database, meaning that the comparisons are limited to different fields of biomedicine.
Other tools for measuring academic success
There are, of course, many other tools that offer statistics about the academic performance of a scientist. ORCID, as well as the previously mentioned Web of Science and Scopus search engines, will all show the citation statistics (although not as elaborately as the dedicated tools I described earlier). Web of Science and Scopus are paid services though, meaning that you will not be able to access them unless you are logged in from a university that has subscribed to these databases.
ORCID identifier is probably the most important of these three. It is a nonproprietary, publisher-independent identification number that many journals accept along with the submissions of a manuscript. Letting the published know your ORCID identifier will make sure that your paper is uniquely assigned to you right after it is published.
There are, of course, even more websites that offer academic performance statistics. ResearchGate and Academia.edu are two of the most popular. Both are private ventures that aim to serve as social networks for researchers. The usefulness of the scores that they use for evaluating the academic performance of scientists, however, is somewhat questionable.
Registering a profile with any of the websites we discussed will not only allow you to find statistics about yourself. Crucially, it will also make sure that others can find you and see your academic performance. Although anyone being able to peek at your academic accomplishments might feel a little intimidating, this is part of our job.
Actually, you should want others to easily find your work because this will continue to polish your academic performance. Some examples of people who you want to enable finding you include peers looking for your papers, reviewers of your research proposal, and potential research partners. A strong presence on the internet can be a useful push for your academic career.
Finally, you can also use the tools to spy on the academic accomplishments of other scientists. This can give a good first estimate of the scientific output of the researchers who are successful in your field. Naturally, you should not bite your nails as a young researcher by looking at the stats of seasoned professors. Check the academic performance of someone who is at the next academic job level that you are aspiring to reach. Then plan your research for the next couple of years to attain a similar output.
Secondary and tertiary academic success indicators
Up until now, we have been mostly discussing the primary academic performance matrics. But there is more to a researcher’s success than publications, funding, and citations. The secondary academic performance indicators include invited presentations, consulting requests, student review scores, journal review invitations, and personal interactions. These can serve to show if your academic career is heading in the right direction, but mostly they will not be a key part of your CV. That is, unless your performance is in some way outstanding in any of these parameters. If that is the case, you might emphasize them in your academic CV.
The tertiary academic success indicators include likes, shares, blog views, visits to your website, number of followers of your academic networks, comments on your posts, professional emails, and other online activities. These can serve as an indicator of your recognition, especially if you are trying to grow your online influence.
Judging your online influence is tricky, but considering the enormous impact that outreach on the internet can have, the academic community is working on ways to measure it. These scientific performance indicators share the name Altmetrics (short for alternative-metrics).
Alternative metrics
We, scientists, are used to measuring our “impact” by the number of publications, the number of citations, h-index. But these metrics only consider the impact on other researchers. Guess what?! There is a world outside the walls of a university. For example, an article discussing a political situation in a particular country might not receive many citations. But at the same time, it can have a large impact on the society of that country.
Altmetrics capture the impact outside of the narrow circles of scientists. The score is made up of diverse online sources, including the number of shares of papers in social networks, blogs, and news; it adds mentions in governmental policies, in Wikipedia, in patents, and perhaps some other online sources.
The Altmetrics approach is gaining traction. Funding agencies and academic employers are starting to take these outreach statistics seriously. I suspect soon job applicants will be required to include the Altmetric score in their CV.
There are several Altmetrics tools that will calculate the score of your article or yourself. Each will do it in a slightly different way since there isn’t really an ideal equation for how to weigh the likes and mentions on which social media site. Here are two of the most popular tools.
Altmetric
Altmetric is probably the most widely recognized alternative metrics tool. The score is displayed in the middle of a colorful circle (see the example). Notice, that below the circle the different sources that have mentioned the article are shown. Clicking on any of them will allow you to trace back any of the original documents, posts, and people who have mentioned it.
In order to view the score, you have to first drag the “Altmetric It” button (see below) to your bookmarks bar.
In case you are not familiar with it, the red square shows which is your bookmarks bar:
Now click the button whenever you are visiting a webpage of a research paper and the statistics will show up.
NOTE: For now, the button works with Chrome, Safari, and Firefox, but you can check the Altmetric website to see if there are any updates.
PlumX Metrics
PlumX is another tool for calculating Altmetrics. Here is an example of how the score looks like. Just like with the previous tool, you can click on the individual items to see more detail about the sources, and users that have shared or mentioned the particular paper.
One big difference compared to the previous tool is that you can not use it on any paper. The tool has to be embedded on the website you are visiting to be able to see the score. But this is not a big problem. PlumX is owned by Elsevier which is why it is displayed on the websites of many journals belonging to this publisher. You should also see it when viewing a research paper on Scopus or Mendeley.
Retraction watch
Let me finish with another academic performance metric. Only in this case you certainly do not want to appear in it. Retractionwatch.com is a blog that reports retractions of journal papers. It also hosts a list of authors who have retracted the most papers. In this way, it is a sort of reverse Nobel prize for scientists. In November of 2020, Yoshitaka Fujii is in the lead with 183 retractions.
Speaking of Nobel prize and retractions, I wrote an article about a Nobel prize winner who actually improved her reputation after a retracted paper. In the article, I will show why for most of us, though, consequences would be grim.
Know your academic performance
You might have noticed that I know many ways for measuring the academic performance of a researcher. I did not have to do much research to write about them in this article. That is because to increase my chances for a successful academic career, I took the time to learn how my performance as a scientist will be evaluated. Knowing this, I know which activities to prioritize so that I can build a strong CV.
I suggest you do the same. The earlier you start, the higher your chances of a successful academic career.
A guide for writing papers that get cited
Knowing how to write research papers can make the difference between being invited to apply for a tenure track position and sending out CVs to random professors in the hope of a miracle.
Hi! My name is Martins Zaumanis and in my interactive online course Research Paper Writing Masterclass I will show you how to write research papers efficiently using a four-step system called “LEAP”. You will learn to to visualize your research results, frame a message that convinces your readers, and write each section of the paper. Step-by-step.
And of course, you will see how to answer the infamous Reviewer No.2 🙂
2 comments