Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Science - Postgraduates & Research Support: Bibliometrics and Citations Analysis

A library research support guide for the Faculty of Science postgraduates and researchers at the University of Johannesburg.

Bibliometrics & Citation Analysis

How can you measure the quality of a research paper? More importantly, how can you determine whether your research is making an impact and is considered important? An objective way is through citation analysis.

Citation analyses can be grouped according to some broad types based on who/what is being evaluated (Source).

  • Ranking Journals
  • Ranking Researchers
  • Ranking Articles
  • Ranking Universities & Countries

Terms & Definitions

Altmetrics

In scholarly and scientific publishing, altmetrics are non-traditional metrics proposed as an alternative to more traditional citation impact metrics, such as impact factor and h-index. The term altmetrics was proposed in 2010, as a generalization of article level metrics, and has its roots in the #altmetrics hashtag. (Source)

Bibliometrics

Bibliometrics is statistical analysis of written publications, such as books or articles. Bibliometric methods are frequently used in the field of library and information science, including scientometrics. For instance, bibliometrics are used to provide quantitative analysis of academic literature. (Source)

Citation analysis

Citation analysis is the examination of the frequency, patterns, and graphs of citations in articles and books.It uses citations in scholarly works to establish links to other works or other researchers.Citation analysis is one of the most widely used methods of bibliometrics.

Citation impact

Citation impact quantifies the citation usage of scholarly works. It is a result of citation analysis or bibliometrics. Among the measures that have emerged from citation analysis are the citation counts for are those for an individual article, an author, and an academic journal. (Source

Citation impact: Article level

One of the most basic citation metrics is how often an article was cited in other articles, books, or other sources (such as theses). Citation rates are heavily dependent on the discipline and the number of people working in that area (Source)

Citation impact: Journal ranking

Journal ranking is widely used in academic circles in the evaluation of an academic journal's impact and quality. Journal rankings are intended to reflect the place of a journal within its field, the relative difficulty of being published in that journal, and the prestige associated with it. (Source). The basic journal metric is the average citation count for the articles in a journal; other metrics include (Source):

  • Impact factor – Reflecting the average number of citations to articles published in science and social science journals.
  • h-index – Usually used as a measure of scientific productivity and the scientific impact of an individual scientist, but can also be used to rank journals. 
  • SCImago Journal Rank – A measure of scientific influence of scholarly journals that accounts for both the number of citations received by a journal and the importance or prestige of the journals where such citations come from.

Research Output

Research output is defined as textual output where research is understood as original, systematic investigation undertaken in order to gain new knowledge and understanding. Peer evaluation of the research is a fundamental prerequisite of all recognised output and is the mechanism of ensuring and thus enhancing quality. (Policy on Research Output)

Article

Using citation analysis to measure research impact


The landscape of science and research is rapidly evolving. Gone are the days when all members of a university department would celebrate the successful publication of a colleague’s paper.1 Earlier, scientists would simply consider the number of papers they had published as a measure of their academic standing. Today, the focus is increasingly shifting from whether a researcher has published a paper to where he/she has published it and the impact that piece of research has on the scientific community and the world at large.2

How can you measure the quality of a research paper? More importantly, how can you determine whether your research is making an impact and is considered important? An objective way is through citation analysis.

Citation analysis

Why count citations in the first place? The list of references directing readers to prior relevant research is considered a fundamental part of any research paper.3 A reference or citation is a form of acknowledgment that one research paper gives to another. Research is additive—scientists build on past work to discover new knowledge. To identify gaps in existing research and choose a research topic, researchers read the relevant published research and use this existing material as a foundation for arguments made in their own research papers.

11 reasons to cite previous work

  1. To direct readers to an authentic source of relevant information
  2. To help other researchers trace the genealogy of your ideas
  3. To acknowledge pioneers and peers
  4. To direct readers to previously used methods, and equipment
  5. To criticize or correct previous work
  6. To substantiate your claims and arguments with evidence
  7. To show that you have considered various opinions in framing your arguments
  8. To highlight the originality of your work in the context of previous work
  9. To guide other researchers in their work
  10. To build your credibility as an author
  11. Finally, because not citing sources can amount to plagiarism4

What are the various citation-based metrics?

Citation analyses can be grouped according to some broad types based on who/what is being evaluated.

  1. Ranking journals: Journals are ranked by counting the number of times their papers are cited in other journals. Journal-level metrics are generally meant to serve as an indicator of journal prestige. The most well known of these is the journal impact factor, from Journal Citation Reports®(a product of Thomson Reuters). The journal impact factor is calculated as the average number of citations all articles in a journal receive over a specific period of time.5
  2. Ranking researchers: Various citation metrics are now used for this purpose. Researchers are ranked by counting the number of times their individual papers are cited in other published studies. These metrics are also used to evaluate researchers for hiring, tenure, and grant decisions. A researcher-level metric that is gaining popularity is the h index,6 which is calculated by considering a combination of the number of papers published by a researcher and the number of citations these papers have received.
  3. Ranking articles: Article-level citation counts may provide an accurate evaluation of the quality and impact of a specific piece of work, regardless of the author. Unfortunately though, such metrics are rarely considered because obtaining these data is tedious and time-consuming.7
  4. Ranking universities and countries: There are databases that rank universities and countries by considering their overall research output through criteria such as citable documents, citations per document, and total citations. These metrics help determine which universities and countries have the most and/or best scientific output. For example, Scimago Research Group (http://www.scimago.es/ ) releases annual reports of institution- and country-wise rankings.

How can citation analysis help you?

Researchers today are faced with increasing pressure to get published. Academic departments are expected to meet specific levels of publication output. Clearly, there is a lot at stake in the assessment of research quality for both individuals and institutions. Given this, governments, funding agencies, and tenure and promotion committees are looking toward simple and objective methods to assess increasing research volumes in the least possible time. To this end, they are turning more and more to citation analysis for objective parameters of impact assessment.

Citation analysis

Pitfalls of citation analysis

When using citation analysis, it is important to bear in mind some of its limitations3,7:

  • It overlooks the disparity in discipline-wise citation rates, that is, the fact that citation patterns differ among disciplines and over time.
  • It ignores the fact that certain manuscript types such as letters and case reports offer inadequate scope for citation and typically have short reference lists. The sentiment of the citation is not considered; that is a negative citation (one used to refute a prior claim) is given as much merit as a positive citation (one used to further the claim being made). So even a paper that has been cited simply to discredit it can work to the author’s advantage in citation analysis.
  • It does not account for author contribution on papers with multiple authors: such citations are as meritorious as those to single-author papers. Citation analysis attributes equal importance to all authors of a paper, regardless of their individual contribution.

Thus, sole reliance on citation data provides an incomplete understanding of research. Although citation analysis may be simple to apply, it should be used with caution to avoid it coming under disrepute through uncritical use.3 Ideally, citation analysis should be performed to supplement, not replace, a robust system of expert review to determine the actual quality and impact of published research.8

Future of citation analysis

Given the shift to online interfaces by more and more journals and repositories, digital information is now available at a few clicks. With the advent of linking tools and digital archives of research papers, scientific literature is more easily retrievable than ever before. Therefore, it is only to be expected that the population of researchers turning to citation data will continue to grow. In such a scenario, researchers cannot afford to undermine the importance of citation analysis.

So next time you are preparing for a promotion or applying for a new position, consider using citation analysis as a means to bolster your eligibility. Use the citation count feature offered by online databases like Web of Science to compile your citation data and employ multiple citation metrics to highlight your research output.

Bibliography

  • Dodson MV (2008). Research paper citation record keeping: It is not for wimps. Journal of Animal Science, 86: 2795-2796.
  • Thomson Reuters. History of citation indexing. Essay in Free Scientific Resources. [http://thomsonreuters.com/products_services/science/free/essays/history_of_citation_indexing/]
  • Smith L (1981). Citation analysis. Library Trends, 30: 83-106.
  • Garfield E. Citation indexing-Its Theory and Application in Science, Technology, and Humanities. New York: Wiley, 1979.
  • Garfield E (2006). The history and meaning of the journal impact factor. The Journal of the American Medical Association, 295: 90-93.
  • Hirsch JE (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences USA, 102: 16569-16573.
  • Neylon C and Wu S (2009). Article-level metrics and the evolution of scientific impact. PLoS Biology, 7: 1-6.
  • Moed HF (2007) The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review. Science and Public Policy, 34: 575-583.



This post Using citation analysis to measure research impact was originally published on Editage Insights.

Prof Patel Altmetric examples - click on the metrics to view the articles Altmetrics

Ranking Researchers

Various citation metrics are now used for this purpose. Researchers are ranked by counting the number of times their individual papers are cited in other published studies. These metrics are also used to evaluate researchers for hiring, tenure, and grant decisions. A researcher-level metric that is gaining popularity is the h index, which is calculated by considering a combination of the number of papers published by a researcher and the number of citations these papers have received. (Source

The following resources can help researchers manage their career through citation counts and the h-index.  

 

Scopus is a subscription database known primarily as an alternative to Web of Knowledge, as it offers a similar article, author, and journal-level metrics, but uses slightly different algorithms to calculate them. Metrics include standard options such as times cited and h-index, as well as original offerings like SJR and SNIP from SCImago. Scopus recently launched “Altmetric for Scopus,” a third party application that runs within the sidebar of Scopus pages to track mentions of papers across social media sites, science blogs, media outlets, and reference managers.

This Thomson Reuters subscription database helped usher in modern bibliometrics with its introduction of the h-index in 1982. Web of Knowledge includes Web of Science, for article and author queries, and Journal Citation Reports, for finding accredited and impact journals. Its metrics include times cited, h-index, impact factor, Eigenfactor, and field-based journal rankings. While many of these metrics have been criticized for not fully representing scholarly value in certain disciplines, they are still considered the gold standard in traditional bibliometrics.


  

Google Scholar Citations provide a simple way for authors to keep track of citations to their articles. You can check who is citing your publications, graph citations over time, and compute several citation metrics. You can also make your profile public, so that it may appear in Google Scholar results when people search for your name. Best of all, it's quick to set up and simple to maintain - even if you have written hundreds of articles, and even if your name is shared by several different scholars. You can add groups of related articles, not just one article at a time; and your citation metrics are computed and updated automatically as Google Scholar finds new citations to your work on the web. You can choose to have your list of articles updated automatically or review the updates yourself, or to manually update your articles at any time

Ranking Journals: Accredited Journals

2022 Accredited DHET Journal List

How to use
Select the buttons below to hide certain columns. Use the search function to search through any particular column. The Reload Data button will reload the data from the data set on the server. The CSV will download the data set to your device as CSV. Please note the set is large and it will appear as if the browser has malfunctioned, please allow 2 mins to download the complete set before a refresh is done. The preferred browsers to use is Firefox, Chrome and Edge. Other browsers not tested. Note: This data set is published once a year

In South Africa, only articles published in accredited journals are considered for subsidy. Journals included in the lists below are considered "accredited" by the Department of Higher Education and Training (DHET) and will be taken into account for government subsidy and NRF evaluation.

Ranking Journals: Journal Citation Report

What is the Journal impact Factor?

Journals are ranked by counting the number of times their papers are cited in other journals. Journal-level metrics are generally meant to serve as an indicator of journal prestige. The most well known of these is the journal impact factor, from Journal Citation Reports® (a product of Thomson Reuters). (Source

The journal impact factor (IF) is a measure reflecting the average number of citations to recent articles published in the journal. It is frequently used as a proxy for the relative importance of a journal within its field, with journals with higher impact factors deemed to be more important than those with lower ones (Wikipedia)

 
Journal Ranking Lists
 

To find the Impact Factor of a journal, search the Journal Citation Report  database. Journal Citation Reports (JCR) uses Web of Science data to calculate the impact factor Journal Citation Reports is published annually in two editions. Only the editions and years to which your institution subscribes appear on the home page.

  • JCR Science Edition contains data about more than 8,000 journals in science and technology.
  • JCR Social Sciences Edition contains data about more than 2,600 journals in the social sciences

Journals Ranking: Elsevier Journal Analyzer

Journal Metrics by Elsevier

Elsevier now provides three alternative, transparent and accurate views of the true citation impact a journal makes:

SCImago Journal Rank (SJR) is a prestige metric based on the idea that 'all citations are not created equal'. With SJR, the subject field, quality and reputation of the journal have a direct effect on the value of a citation. SJR is a measure of scientific influence of scholarly journals that accounts for both the number of citations received by a journal and the importance or prestige of the journals where such citations come from. It is a variant of the eigenvector centrality measure used in network theory.

The SCImago Journal & Country Rank is a portal that includes the journals and country scientific indicators developed from the information contained in the Scopus database. These indicators can be used to assess and analyze scientific domains.

This platform takes its name from the SCImago Journal Rank (SJR) indicatorp, developed by SCImago from the widely known algorithm Google PageRank. This indicator shows the visibility of the journals contained in the Scopus® database from 1996.

Journal Ranking

Google Scholar Metrics

Google Scholar Metrics provide an easy way for authors to quickly gauge the visibility and influence of recent articles in scholarly publications. Scholar Metrics summarize recent citations to many publications, to help authors as they consider where to publish their new research.

  • Browse the top 100 publications in several languages, ordered by their five-year h-index and h-median metrics.
  • You can delimit the journal list per Subject fields
  • To see which articles in a publication were cited the most and who cited them, click on its h-index number to view the articles as well as the citations underlying the metrics.

Altmetrics