Re: Bad Flow Data & reviewing -- What can we do?

From: J.Paul Robinson (jpr@flowcyt.cyto.purdue.edu)
Date: Sun Oct 21 2001 - 17:20:54 EST


Well at the risk of being attacked for my spelling and typos (!!), my
stupidity, my previous and many publications omissions, and
probably even unrelated sins (I plead guilty now), I will weigh in on
the bad flow data......so ....

The only real solution to this issue is education of staff, students
and faculty. If we teach our students the right thing, we go a long
way to solving the problem. Legislation, rules, and the like, will only
serve to make more work for us all...  so the responsibility is
squarely upon the shoulders of each and every PI, lab director and
professor out there...

First, I will say that as editor of Current Protocols in Cytometry, I
made a decision 5 years ago that I would NOT accept or publish
any flow figure which had axes labeled "FL1", "FL2", etc...at the
risk of being beaten up, I will note that a great number of you are
guilty of trying to publish figures with almost totally meaningless
axis labels......  we almost succeeded .....in about 1500 pages I
think we found 2 that got through and they are being removed as
we speak.....Cytometry used to accept this all the time - a figure
labeled with an axes which say Axis 1 Vs Axes 2 (read FL1 Vs
FL2 here) would even be laughed at by editors of Science, but we
published these frequently in our own journal....I think we are
getting better now, but we have to be careful

So Rule # 1: Label the axes in a meaningful way

Second, why do we publish half of the plots and histograms we
publish? A good number of them are probably published because
we have used them for subjective evaluation purposes and not
quantitative evaluation (I know I have done the same....)  so
perhaps if we designed our experiments better, we would be able to
come up with quantitative measures that would give us number of
molecules per cell (or something akin to that) rather than a picture
that looks more messy on one side than the other.

So Rule #2: How about being more quantitative?

Third, the real issue to me is the publication of relatively bad
looking histograms or dot plots from which very significant
decisions are made. Thus we constantly see "a representative
histogram" and then some numbers perhaps to prove a point (OK I
have done this too...)

So Rule #3: Representative histograms (plots) may not be so
useful after all...

Finally, the issue of post-review. My dad used to say it was no
good closing the gate after the horse got out. I never did know what
he meant as a kid since we never owned a horse. Bottom  line is,
post-reviewing our own papers would be a very difficult thing to
implement....post-reviewing other Journals papers might be a little
bit arrogant.....I just can't imagine a website that is essentially
contributed to by flow jocks (and Jills) commenting on the nasty
quality of data in Science and Nature, Cell etc etc.... Not sure that
anyone that every puts their name to such a list will ever manage
to get a paper published in such a journal again......

So Rule #4: Let's not get into the business of publically post-
reviewing articles published in other journals...

And really, really finally, I have one more rule....

So Rule #5: lets not make up a list of rules.......

May all your plots be exorcised with care......

Paul Robinson

J.Paul Robinson, PhD             PH:(765)4940757
Professor of Immunopharmacology
Professor of Biomedical Engineering
Purdue University          FAX:(765)4940517
EMAIL:jpr@flowcyt.cyto.purdue.edu
WEB: http://www.cyto.purdue.edu



This archive was generated by hypermail 2b29 : Sun Jan 05 2003 - 19:01:35 EST