Reviewing – The Basics Toby Charkin – Executive Publisher SfN - October 2012 What is Peer Review? Peer Review has 2 key functions: • Improves the quality of the research submitted for publication by giving reviewers the opportunity to suggest improvements Pre-Submission Peer Review Authors • Acts as a filter by ensuring only good research is published. Helps to determine validity, significance and originality Reviewers Editor Publication • Importance and clarity of research hypothesis • Originality of work • Delineation of strengths and weaknesses of methodology, experimental / statistical approach, interpretation of results • Writing style and figure / table presentation • Ethics concerns (animal / human) “ Novelty” Issues to Review “ Technical” Quality Purpose of Peer Review Check the manuscript for 4 • • • • Mistakes in procedures or logic Conclusions not supported by the results Errors or omissions in the references Compliance with ethics standards – Has the protocol been approved by an appropriate Ethics Committee? • Originality and significance of the work Quality of the Work Are the methods appropriate and presented in sufficient detail to allow the results to be repeated? Are the data adequate to support the conclusions? Methods Results 1. Do all “methods” have a “results”? 2. Have all “results” been described in the “Methods”? 5 Conclusions 1. Are all “conclusions” based on “results”? Presentation of the Paper 6 Writing • Clear, concise, good English? • But no need for reviewers to act as language editor Title • Specific, and reflecting the content of the manuscript? Abstract • Brief, and describing the purpose of the work, what was done, what was found, and the significance? Figures • Justified? Clear? Sharp, with fonts proportionate to the size of the figure? Clear and complete legends? Tables • Can they be simplified or condensed? Should any be omitted? Trade names, abbreviations, symbols • Properly used where indicated? Abused? Comments to the Editors • Comment on novelty and significance • Recommend whether the manuscript is suitable for publication or not, usually – Accept / Minor revision / Major Revision / Reject Reviewer makes a recommendation Editor makes the decision • Confidential comments will not be disclosed to author(s)! 7 Comments to the Authors • Provide specific comments on the design, presentation of data, results, discussion, and conclusions – Do not include recommendations for acceptance / rejection – Require or suggest other experiments or analyses – Distinguish between “needs to change” and “nice to change – Use neutral language and be constructive • Ensure that that the comments to the author(s) are consistent with your recommendation to the editors Privileged Document This manuscript is a confidential document. The data • Are and remain the exclusive property of the authors • Should not be disclosed to others (who may use the information in their research) If you have printed the manuscript • It must be kept confidential until the review process has been completed • After final decision by the editor it must be destroyed If you have shared responsibility for the review of this manuscript with a colleague, you should provide that person’s name and affiliation to the editors 9 Reviewers Should only accept to review manuscripts • In their areas of expertise • When they can complete the review on time Should always avoid any conflicts of interest • If in doubt, consult with the editor Are not allowed to “use” the data Must provide an honest, critical assessment Must analyze the strengths and weaknesses of the research, and provide specific suggestions for improvement 10 Oversight Function: Ethics The reviewer also has the (unpleasant) responsibility of reporting suspicion of • Duplicate publication • Plagiarism – Also self-plagiarism • Data fabrication or falsification • Ethics concerns • Etc., These problems are normally followed up by the Editors and the Publisher 11 Review Process (i) Regular articles are initially reviewed by at least two reviewers When invited, the reviewer receives the Abstract of the manuscript The editor generally requests that the article be reviewed within two weeks • Limited extensions sometimes acceptable Articles are revised until the two reviewers agree on either acceptance or rejection, or until the editor decides that the reviewer comments have been addressed satisfactorily The reviewers’ reports help the Editors to reach a decision on a submitted paper • The reviewer is the recommends; the editor decides! 12 Review Process (ii) If a report has not been received after 4 weeks, the Editorial office contacts the reviewer If there is a notable disagreement between the reports of the reviewers, a third reviewer may be consulted The anonymity of the reviewers is strictly maintained • Unless a reviewer asks to have his/her identity made known to the authors 13 Review Process (iii) • Reviewers must not communicate directly with authors • All manuscripts and supplementary material must be treated confidentially by editors and reviewers – The manuscript cannot be distributed outside this small group As author As editor As reviewer As reader • The aim is to have a “first decision” to the authors within 4-6 weeks after submission of the manuscript • Meeting these schedule objectives requires a significant effort on the part of the Editorial staff, Editor and Reviewers • If reviewers treat authors as they themselves would like to be treated as authors, then these objectives can be met 14 January 2012 As a researcher, you wear many hats! Take Home Message The best reviewers tend to view themselves as teachers and mentors rather than critics ! January 2012 New initiatives How can we support and improve the peer-review process to the benefit of Editors, authors and reviewers? 16 Tools for Reviewers For Editors • Plagiarism detection tool at time of submission – Checks the manuscripts against the “entire” published literature • “Find a Reviewer” tool, based on “Scopus” database • Free access to ScienceDirect – All content published by Elsevier For Reviewers • Free access to Scopus – The world's largest abstract and citation database of peerreviewed literature and quality web sources with smart tools to track analyze and visualize research • Reference-linking and resolution in PDF of the manuscript 17 Peer Review Grand Challenge March to May 2012 - web-based Challenge invited submissions on any aspect that could significantly add to the current peer-review system. To explore how publishers and Editors can help early career researchers become reviewers, or how reviewers can be recognized by either their institutes or publishers. Winner: Elsevier Reviewer Badges and Rewards scheme by Simon Gosling (a tangible recognition system for reviewers – e.g. Most Comprehensive Review”, another for the “Most Helpful Review”, and one for the “Most Contributions in a Year) For more information on this initiative, please contact Clare Lehane, Executive Publisher, STM Publishing, [email protected] or visit www.peerreviewfuture.com 18 Neuroscience Peer Review Consortium Authors whose papers are not accepted by one journal in the Consortium and wish to submit their manuscripts to a second participating journal can request that the previous set of reviews be forwarded. Reduce the number of times a manuscript must be reviewed - it reduces the workload on reviewers and speeds up publication time. http://nprc.incf.org/ Many Neuroscience journals are part of this initiative, including Elsevier titles such as Brain Research, Neuropsychologia, Neuroscience, NeuroImage etc Reviewer Guidance Programme The Reviewer Guidance Pilot Program aims to provide participants with the experience of independently reviewing at least two manuscripts inside a specially-created EES (Elsevier Editorial System). Guide participants in how to write review reports in such a way that they answer the needs of both the Editor and the author “The Reviewer Guidance Program is not only an experience that helps early career researchers become better reviewers, but also to be more critical in analyzing their own papers before submitting. In addition, this is a great opportunity for junior scientists to network with their more senior peers.” For more information on this initiative, please contact Irene KanterSchlifke, Publisher for Pharmacology and Pharmaceutical Sciences, STM Publishing ([email protected]) 20 Published reviewer reports In this pilot we have published reviewer reports alongside the final article in Agricultural and Forest Meteorology Open reviewer reports increase peer-review transparency and assist good articles to gain authority and provides public acknowledgement of reviewers’ contributions Results so far: The pilot launch attracted positive international media attention. It was also suggested that open reviewer reports could play a useful role in training early career researchers as reviewers. So far, reviewer reports have been published alongside around 13 manuscripts. For more information on this pilot, please contact Gilles Jonker, Executive Publisher, Physical Sciences, [email protected] 21 Open Peer Commentary In this pilot, we have asked experienced researchers to submit a one page comment on an article for the journal Physics of Life Reviews. These comments are published in the same issue as the article. On average, five comments are published with the article and the author can write a rebuttal article. Results so far: Since the pilot was launched in January 2010, the journal has seen an increase in papers (2011 - 85 and 2010 - 74; previously the journal received around 12 papers per year). There has also been a sharp increase in usage – roughly 3,000 downloads per month compared to 2,000 per month in 2009. For more information please contact Charon Duermeijer, Publishing Director Physics, [email protected] 22 PeerChoice Traditionally, Editors chose to approach reviewers they consider are suitably qualified, or who would find the subject matter interesting. But what if the reviewer could select the manuscript themselves? For a year now, we have been experimenting with this additional peer-review system on the journal Chemical Physics Letters. A selected pool of reviewers receives an overview of the new submissions. If they like a paper because it matches their expertise and interest, they can decide to review it. Results so far: The time taken to review the manuscript has been slightly reduced, while the time taken to accept an invitation has been halved. For more information on this pilot, please contact Egbert van Wezenbeek, Director Publication Process Development, Publishing Services, [email protected] 23 January 2012 Thank you For author and reviewer training webcasts: www.elsevier.com/trainingwebcasts For reviewer information and guidelines: www.elsevier.com/reviewers Effective Reviewing Dr Michael Rugg – Editor-in-Chief of Neuropsychologia • Brainard, D.H. How to write an effective manuscript review. Optics and Photonics News, 2000, June, 42-43. • Roediger, H.L. Twelve tips for reviewers. APS Observer, 2007, 20, part 4 • Sternberg, R.J. On civility in reviewing. APS Observer, 2002, 15, part 3. • Sternberg R.J (ed.) Reviewing Scientific Works in Psychology. American Psychological Association, 2006 Effective Reviewing • Review others as you would want them to review you • Turn the review in on time!!! • Remember that no paper is perfect – avoid a ‘find the fatal flaw’ mind-set • Don’t go overboard summarizing the paper – couple of sentences focusing on the main point • Big picture first – summarize your conclusion and, briefly, its rationale. Focus on whether the paper tackles an important topic, and whether the findings and their interpretation advance the field (or could potentially do so). Expand on the reasons for your conclusion in a series of paragraphs that each deal with one concern or issue • Don’t be picky or get bogged down in minor details. A reviewer is not a copy-editor • Resist the temptation to re-write the paper – restrict recommendations for revision to those necessary to address your significant concerns. • Be constructive and respectful even when being critical - avoid patronizing comments that imply you’re smart and the authors are not.