After seeing all of the recent fiascoes in the news about journals publishing fake articles & journals, data being changed, etc., I was trying to figure out a better way to help patrons search for quality information. How can we help our patrons without having to review all of their search results? Plus who has the time to review everyone’s search results?
I know we can and should show them what to look for and we should continue to educate our patrons on quality literature. I am not suggesting we stop educating our patrons on how to identify quality information, but this takes time (yours & the patrons) to demonstrate these skills and time for the patrons to constantly evaluate a resource. Plus how many patrons will actually go through the steps to review an article or author? There has to be an easier way.
My next thought was to have patrons review the impact factor of all the journals or to look at the number of times an author is cited; however, this would require a great deal of work. Patrons would have to open a new window to review Scimago (http://www.scimagojr.com/) or Eigenfactor (http://www.eigenfactor.org/). Also where can you find reliable information on all authors’ citations?
So how can we mesh these two review items into a constantly used resource? My suggestion is to place the impact factor of a journal and the author’s information on PubMed©. While this would require additional work by NLM, it would greatly benefit patrons to quickly see the reliability of the journal and the expertise of the author.
PubMed© already lists the related articles. This would be addition impact factor information to guide patrons to the best resources. Perhaps a drop down menu or even just a link on the citation to the journal/author information would provide the necessary information to help patrons gauge the quality of the journal & article. I could really dream big and suggest miniature graphs or a rating system, but I must also be reasonable and understand this would require a great deal of work.
By just providing a link to the information, patrons, at the least, would not have to search for impact factor information on their own in addition to searching for the literature. I know how many patrons are going to click on a link. At least the option would be there.
This may be just wishful thinking. What do you think? Impact factor information on PubMed is it a good idea, bad idea, or wishful thinking?
T Scott
June 4, 2009 at 2:34 am
I like the thinking behind what you’re trying to do, but there are two issues here:
First is the fact that impact factors themselves are controversial — there is considerable debate about the degree to which they accurately reflect the quality of a journal, and even more concern about how they are used. Too often, impact factors are thoughtlessly considered to be a quality indicator without due consideration of the nuances involved in making a careful judgment about what the impact factor really means. I fear that including the impact factor in Pubmed would exacerbate the tendency to rely solely on impact factor as a criterion of quality.
The second issue is more pragmatic. Thomson-Reuters considered the Journal Citation Reports (the database which provides the IF) to be proprietary. To include impact factors in a publicly available database would be to give away a commodity that generates considerable revenue for them. I can’t see that they would ever agree to it.
Time consuming though it is, I think the approaches that you describe at the beginning of your post need to be the core of what we do — teaching people about the complexity of the information landscape, about the resources that are available to them, and about the necessity of bringing critical judgment to bear.
alisha764
June 5, 2009 at 4:21 am
Thank you for commenting and pointing out two very important issues and the importance of reviewing articles. I completely agree nothing can replace reading and evaluating articles for yourself; however, there are instances when you cannot review everything. My post was intended to contribute to the conversation on ‘ranking’ journals/articles and providing this information in PubMed or through another means.
Perhaps I should have extended my post to include information from a previous post, Bibliometrics: A place for everything, where I briefly mentioned the idea of ranking journals by using impact factor, h index, social bookmarking, and other means. I have been circling around this idea for a while. By providing graphs or numerical rankings of some fashion it would help busy patrons who cannot review every item. It is similar to the way information is sometimes ranked in point-of-care tools.
It is just like with RSS. I have done individual sessions helping staff set-up their RSS readers. I always start with recommending subscribing to some more well known journals (NEJM, JAMA, etc.) and some journals in their field. Additionally, I suggest they follow news feeds and blogs. Some do not understand why they need to follow these additional feeds. All they want is the TOC. Well, even though you scan the TOC of several journals you may miss something or deem something is not important and not read it. These missed items will often times be picked by the news or bloggers. So while it is still important to review the TOC, it is also beneficial to have an additional source to help you identify items of interest/importance that may have been missed or overlooked.
Should you read the article and evaluate it & the journal for yourself based on the evidence . . . yes. Does everyone have time do an extensive evaluation on every item . . . no. Teaching EBM and helping patrons evaluate information is something librarians should always do; however, I still think we should consider the idea of graphs/rankings as an additional way to help patrons.
You are correct that impact factor and citations alone would be an injustice to many articles and journals. A new author would be hindered by only being considered based on the number of times he/she has been cited. Including blog citations, and social bookmarking would help; however, as you have stated, ensuring accuracy in determining the quality of a journal/article when utilizing any ‘ranking’ system is difficult.
I do not know how we can incorporate the various ways to rank a journal/article for easy scanning. Nor do I know all of the items that should be included in the ranking system. It is indeed a difficult and controversial issue. But considering the online forms of communication (Twitter, blogs, Facebook, etc.) another system is needed to easily display the ‘ranking’ information when patrons search on PubMed or other sites/databases. Creating a new ‘ranking’ system will certainly take time and possibly several trials; yet, for me it was time to stop circling around an idea and contribute to a conversation on improving the ranking system to provide better help to our patrons. Thank you for being an additional source by pointing out the flaws in my proposal.
Thank you for contributing to the conversation,
Alisha
a user
June 4, 2009 at 4:08 am
PubMed may already use a number of rankings… including citation reference counts. However, it’s not nearly that simple. If a new journal or article is published, it contains 0 citations, so it would remain “unpopular” and thus fall in the relevancy rankings. That could contribute to it remaining obscure and thus creating a cycle where un-cited articles could never increase.
This is of course, an oversimplification, but there are many more factors to take into account than citation count alone.
alisha764
June 5, 2009 at 4:23 am
I agree, there are several factors to consider. And as I stated in the reply to the previous comment, perhaps I should have extended this post and mentioned the idea of journal rankings and/or graphs. The idea was not to diminish journals/articles or hinder new authors, but to suggest some way for busy physicians and new people to the field to quickly scan the literature and see in a graph or numerical form the ranking of information. It is just like with point-of-care tools ranking of information. A number or level is assigned to the information so those quickly reviewing will know the quality of the recommendation.
I completely agree we should continue educating patrons on how to evaluate information. Nothing can replace reading the article and reviewing the data for yourself; although, to help a busy physician some form of ranking or way to quickly scan would be very beneficial. Perhaps not just basing it on the number of times a journal has been cited but on other items, number of times reviewed online or bookmarked on sites like CiteULike.
Again, my idea was only a rough suggestion to contribute to the conversation on the topic of providing journal/article rankings.
Thank you,
Alisha