I think it is fair to say that the relationship between real-world importance and quantifiability is an inverse one. This view freaks out my accounting students and is antithetical to the "impact factor" culture in which AACSB -- indeed our society as a whole -- wallows into intellectual mediocrity.
With this bit of metaphysics out of the way, you might want to check out some of Ronald Bailey's recent columns at reason.com on the epidemic of bogus, high-impact-factor research being churned out by so-called "hard-science" types. Here's a teaser, from April 2012, regarding bad science in oncology:
* * *
Last week, the scientific journal Nature published a disturbing commentary claiming that in the area of preclinical research—which involves experiments done on rodents or cells in petri dishes with the goal of identifying possible targets for new treatments in people—independent researchers doing the same experiment cannot get the same result as reported in the scientific literature.
The commentary was written by former vice president for oncology research at the pharmaceutical company Amgen Glenn Begley and M.D. Anderson Cancer Center researcher Lee Ellis. They explain that researchers at Amgen tried to confirm academic research findings from published scientific studies in search of new targets for cancer therapeutics. Over 10 years, Amgen researchers could reproduce the results from only six out of 53 landmark papers. Begley and Ellis call this a “shocking result." It is.
The two note that they are not alone in finding academic biomedical research to be sketchy. Three researchers at Bayer Healthcare published an article [PDF] in the September 2011 Nature Reviews: Drug Discovery in which they assert “validation projects that were started in our company based on exciting published data have often resulted in disillusionment when key data could not be reproduced.” How bad was the Bayer researchers’ disillusionment with academic lab results? They report that of 67 projects analyzed “only in 20 to 25 percent were the relevant published data completely in line with our in-house findings.”
Perhaps results from high-end journals have a better record? Not so, say the Bayer scientists. “Surprisingly, even publications in prestigious journals or from several independent groups did not ensure reproducibility. Indeed, our analysis revealed that the reproducibility of published data did not significantly correlate with journal impact factors, the number of publications on the respective target or the number of independent groups that authored the publications.”
* * *
If impact factors are so unreliable in hard sciences, what should we expect from "soft" social science disciplines like law, economics, accounting and finance?
Here's another valuable Baily column, from August 2012, on point:
http://reason.com/blog/2012/08/17/shoring-up-the-mantra-of-science-take-no
Impact factors are an easy way for P&T committees and department chairs to avoid the hard work of making professional judgment calls. Reliance on impact factors as the sole measure of research productivity and quality is, among other things, unprofessional and lazy. How to communicate this message and turn it into reform is a tough question. But we need to start somewhere.
Kurt S. Schulzke, JD, CPA, CFE
Associate Professor of Accounting & Business Law
Director - Law, Ethics & Regulation
Corporate Governance Center
Kennesaw State University
+ 1770-423-6379 (O)
+ 1404-861-5729 (C)
http://coles.kennesaw.edu/centers/corporate-governance/
My research: http://ssrn.com/author=804023
----- Original Message -----
From: Robert Bird <[log in to unmask]>
To: [log in to unmask]
Sent: Sun, 21 Oct 2012 17:42:07 -0400 (EDT)
Subject: Re: impact factor and law reviews
The decentralization effect, that you ably state here, was one reason that crossed my mind too.
Impact factor is briefly defined in W&L as : "Impact-factor shows the average number of annual citations to articles in each journal (rounded to two decimal places)." The full explanation: http://lawlib.wlu.edu/LJ/method.asp#impactfactor
Thomson- Reuters: " The impact factor is one of these; it is a measure of the frequency with which the "average article" in a journal has been cited in a particular year or period. The annual JCR impact factor is a ratio between citations and recent citable items published. Thus, the impact factor of a journal is calculated by dividing the number of current year citations to the source items published in that journal during the previous two years (see Figure 1)." http://thomsonreuters.com/products_services/science/free/essays/impact_factor/
There of course may others.
Robert
Robert C. Bird
Associate Professor, University of Connecticut School of Business
Research Fellow, University of Connecticut School of Law
Northeast Utilities Chair in Business Ethics
Editor-in-Chief, American Business Law Journal
University of Connecticut
2100 Hillside Road, Unit 1041
Storrs, CT 06269
[log in to unmask]<mailto:[log in to unmask]>
My research: http://ssrn.com/author=56987
From: Academy of Legal Studies in Business (ALSB) Talk [mailto:[log in to unmask]] On Behalf Of Maurer,Virginia G
Sent: Sunday, October 21, 2012 5:33 PM
To: [log in to unmask]
Subject: Re: impact factor and law reviews
Wouldn't this be evidence for the proposition that there are so many more law reviews of quality -- since they each publish so few articles and have so many law professors and others compering for space -- and high quality research is found and cited across a greater set of journals.
There was a question mark after that sentence.
We've been saying this all along, and here is evidence. I think.
Ginny
Virgina G. Maurer, M.A., J.D.
Professor of Business Law and Legal Studies
Director, Poe Center for Business Ethics
Darden Restaurants Professor of Diversity Management
Warrington College of Business
352 256 0295 (cell)
352 376 2867 (home)
________________________________
From: Academy of Legal Studies in Business (ALSB) Talk [[log in to unmask]] on behalf of John Allison [[log in to unmask]]
Sent: Sunday, October 21, 2012 5:22 PM
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: impact factor and law reviews
I would have to know more about how the impact factors are calculated, in law and in the business journals.
John
John R. Allison
The Spence Centennial Professor of Business, and
Professor of Intellectual Property
McCombs School of Business
University of Texas at Austin
[log in to unmask]<mailto:[log in to unmask]>
From: Academy of Legal Studies in Business (ALSB) Talk [mailto:[log in to unmask]] On Behalf Of Robert Bird
Sent: Sunday, October 21, 2012 4:04 PM
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: impact factor and law reviews
Hi everyone,
In reviewing the impact factor of law reviews and other business disciplines, I've noticed some startling differences. The impact factors for the Journal of Finance (4.02), Academy of Management Review (6.169), Journal of Marketing (3.75), and others are in a roughly similar strata. Law, however, is quite different. According to Washington and Lee's ranking (http://lawlib.wlu.edu/LJ/index.aspx), out of the 1,685 law journals ranked only one (Columbia Law Review) barely scores an impact factor above 3.0 (3.07). Only 19 journals have an impact factor above 2.0. Only 93 journals have an impact factor above 1.0, representing the top 5.51% of the discipline.
By now you know where I am going with this and you can surely guess why I am asking. (Hint: it's not for my own personal amusement) Why are impact factors for law reviews so low, especially relative to business disciplines? How should impact factors in law be treated in law relative to our business colleagues?
Your thoughts to the LISTSERV or to me personally are much appreciated.
Thanks,
Robert
Robert C. Bird
Associate Professor, University of Connecticut School of Business
Research Fellow, University of Connecticut School of Law
Northeast Utilities Chair in Business Ethics
Editor-in-Chief, American Business Law Journal
University of Connecticut
2100 Hillside Road, Unit 1041
Storrs, CT 06269
[log in to unmask]<mailto:[log in to unmask]>
My research: http://ssrn.com/author=56987
|
