Google Scholar/WoS Cited Reference smackdown
OK, overblown title. We’ve been doing a workshop this year called “Got Impact?” about why/where/how to do cited reference searching & analysis. Web O’ Science and Google Scholar often have different citation counts for the same article. Usually Google has a higher count, but not always. Frankly, I’ve been trying not to think about why this is. This week I decided to at least take a little peek. I sat down with a paper from 2005 with a manageable number of citing references and printed out the “cited by” lists from both. (The starting paper was Burkle A. Poly(ADP-ribose). The most elaborate metabolite of NAD+. The FEBS journal. 2005;272(18):4576-89. ) Google Scholar (hereafter inconsistently referred to as GS) said it was cited by 52 (of which it would only show me 50) and Web O’ Science (WoS) said 46. Here’s how it broke down:
- 33 papers were on both lists. Dup de dup.
- 13 papers were in WoS’s list but not in the GS list. However, most of those 13 could be found as original articles in GS, but didn’t link up as citing the Burkle paper. Could have been because of a parsing error – in some but not all citations, Burkle’s name has an umlaut over the u. You’d think GS wouldn’t get tripped up by that – I certainly didn’t try to pick a trick article. I didn’t do any digging to try to find a pattern with those 13 items.
- 6 articles on the GS list were in journals that WoS covers, but not on WoS’ list – some seemed to be in that “epub” stage without a final citation, but others just looked like journals where ISI’s indexing was a bit behind.
- 4 articles on the GS list were duplicates, mostly to one Russian-language article that appeared in different stages of translation in the search results. There was a single entry for that article on the WoS list.
- 4 citing “items” on the GS list were not journals – three were book chapters from publishers using the same platform for books as journals, and one appeared to be a dissertation. Of course none of these were on the WoS list.
- 2 citing articles on the GS list were for journals not indexed by WoS
So what to tell a researcher wanting a comprehensive citation search? Well… use both. Unfortunately, there is a large amount of duplication. The best tool I know of to grab a whole page of GS results is Zotero. But exporting a list from ISI to Zotero doesn’t always work well, and GS records are so dirty, there’s still loads of manual cleanup to do before you can expect it to merge cleanly with an ISI result for deduping.
I think the loser of this smackdown was me.