SciBeh-Topic-Visualization

peer, publish, publication, review, preregistration

Topic 22

peer publish publication review preregistration scholarly journal scientific register author research quality editorial reproducibility transparent

Enhancing Peer Review of Scientific Reports
May 29, 2020 · · Original resource · webpage

Academic peer review of scientific manuscripts often falls short. It invariably slows and sometimes prevents the publication of good research. And it sometimes leads to the distribution and amplification of flawed research. Prestigious journals sometimes publish research grounded on shaking theory that used weak measures and inappropriate analyses to reach dubious conclusions. Failings of peer review play a principal role in those problems. Journal editors typically perform journal tasks off the side of their desks, on top of everything else. They may handle manuscripts outside of their expertise. Sometimes it’s difficult for them to know who to ask to review. When they do identify prospective reviewers, many decline the request or don’t reply. When I was Editor of Psych Science, I often sent 6 or more invitations to get 2 acceptances. Thus, it sometimes takes weeks to get a few people to commit to reviewing. Some of those fail to deliver on time or at all, despite multiple prompts (doubtless sometimes for good reasons – one never knows what’s going on in another person’s life). So even under the best conditions, peer review takes a long time. And rarely is the decision to accept as is. Good news for authors is a reject/revise/resubmit decision. Thus, it typically takes many months between initial submission and eventual acceptance. The other day I saw an email about a manuscript submitted to Psych Science in January that had just been accepted. I thought, “Wow! Quick!” Is that time well spent? I believe that generally it is. Many reviewers provide assessments that are detailed, clear, insightful, well-informed, and constructive. Many editors strive to understand the work well enough to fairly assess the manuscript and, if it has potential for their journal, making the manuscript as good as it can be. As an author/co-author I have many times been furious with editors and/or reviewers (indeed, even now it takes me days to steel myself to read an action letter), but very often editors’ and reviewers’ input has (I believe) led to major improvements.
covid-19
concern
science
research
publication
preprint
collaboration
quality
academia
rapid review
peer review
peer, publish, publication, review, preregistration
library, open, preservation, psa, tool
Publication by association: the Covid-19 pandemic reveals relationships between authors and editors

During the COVID-19 pandemic, the rush to scientific and political judgments on the merits of hydroxychloroquine was fuelled by dubious papers which may have been published because the authors were not independent from the practices of the journals in which they appeared. This example leads us to consider a new type of illegitimate publishing entity, “self-promotion journals” which could be deployed to serve the instrumentalisation of productivity-based metrics, with a ripple effect on decisions about promotion, tenure, and grant funding.
covid-19
hydroxychloroquine
judgements
conference, webinar, series, acm, ci'20
peer, publish, publication, review, preregistration
Post-Publication Peer Review for Real

The inefficiency of the current peer-review system has been discussed for many years, and now there is a surge of various countermeasures aiming to solve the problems. Post-publication peer review (PPPR) has emerged as one of them, and some scholars expected that it would be the definite solution. Unfortunately, a decade of trial has not turned out to be as fruitful as expected. We assessed that the biggest reason for this situation was the lack of incentives among contributors, and proposed that publishing review commentaries as independent and qualified publications in a dedicated section of a journal might solve the problem. Specifically, we took the open peer commentary section of Behavioral and Brain Sciences as a model of such an incentivised structure, and pictured a possible implementation of this idea in the current web-based environment. Potentials of this new PPPR format were suggested.
transparency
open science
peer review
peer, publish, publication, review, preregistration
library, open, preservation, psa, tool
A completely re-imagined approach to peer review and publishing: PRINCIPIA
Sept. 10, 2020 · · Original resource · blog

With regards to the proposed system, I found it hard to follow (had to keep scrolling up and down to refer to different sections to get the whole picture!), but here's what I understand:Authors publish articles and then submit it (in essence, submitting a pre-print to be reviewed for a journal), paying for the article to be reviewed (with a bidding system)Reviewers receive part of this review fee (but it seems the journals can decide to keep some?)Journals are formed by editorial boards, and the reputation of the journal depends on who is on the editorial board. It also depends on the impact of the articles published(?—I read this point coming in the end, but wasn't sure how the two interacted) Journals are shifting and change the moment there is a change to the editorial boardThe reviewers also come from the editorial boardsTo join the editorial boards one also needs to bid a joining feeI did find it all very complex, and wondered what the incentive was, as an author, to buy in to such a system. Primarily, not knowing what the criteria for the review I'd be paying for would be a red flag for me. So this, I think, needs to be better defined and transparent.
incentive
scientific practice
publishing
journal
clarity
impact
proposal
peer review
system
conference, webinar, series, acm, ci'20
peer, publish, publication, review, preregistration
Replication crisis or an opportunity to improve scientific production?

Science is undergoing a crisis that has been referred to, since the early 21st century, as a crisis of confidence and a crisis of replication. This article reviews questions pertaining to the replication crisis; questions addressing the quality and credibility of the sciences; specifically, questions linked to what are known as false positives, null results, and questionable research practices (p‐hacking, harking, cherry‐picking). As an outcome of our review and analysis, a set of recommendations to strengthen the elaboration of reliable and valid research studies is provided. Changes are needed in order to foment meta‐research, open science practices and replication studies; notably, changes are needed in the instruction of research methods; in the use and interpretation of statistical data, as well as in research culture in general. We conclude that the replication crisis presents an opportunity to improve research practices and the quality of scientific production in all fields of research, including research in education.
statistics
science
reliability
open science
replication crisis
validity
improvement
opinion, science, society, insight, economist
peer, publish, publication, review, preregistration
Peer review should be an honest, but collegial, conversation

Nature asked authors and editors for advice on how to improve peer-review communication.
communication
survey
advice
criticism
peer review
conversation
author
personal experience
opinion, science, society, insight, economist
peer, publish, publication, review, preregistration
From peer review to “science without the drag” via PsyArXiv
May 26, 2020 · · Original resource · article

Psychological research could play a critical role in informing policies during times of crisis and uncertainty. However, as stated in a previous post by Patrick Forscher, Simine Vazire, and Farid Anvari during this digital event, issues with generalisability, replicability, and validity may limit the practical implications of our research. The problems of reliability are compounded by the fact that in times of crisis, the rapid dissemination of evidence is as critical as its quality. Thus, the field finds itself in a challenging situation, where the need for quality and rapid knowledge challenges the utility of psychology research. As Ulrike Hahn put it in a previous post during this digital event; there is a need for proper science without the drag. But researchers have not sat idle. On the contrary, some have started building an infrastructure for crisis knowledge management, described in the earlier post by Stefan Herzog. Others have started posting their findings as “living documents”, as described by Yasmina Okan in her post. And many more have published their findings as preprints as soon as they emerge, without the “drag” introduced by peer review.
covid-19
research
psychology
behavior science
peer review
peer, publish, publication, review, preregistration
library, open, preservation, psa, tool
How to Identify Flawed Research Before It Becomes Dangerous
July 20, 2020 · · Original resource · news

Scientists and journalists need to establish a service to review research that’s publicized before it is peer reviewed.
covid-19
science
media
journalism
reliability
research
policymaking
peer review
study design
humanity, statistic, wrong, communicate, fit
peer, publish, publication, review, preregistration
Opinion: Authors overestimate their contribution to scientific work, demonstrating a strong bias

Teamwork is an essential component of science. It affords the exchange of ideas and the execution of research that can entail high levels of complexity and scope. Collaborative science also leads to higher-impact publications relative to single-authored research projects (1). Published articles are a key product of scientific work, bearing considerable impact on researchers' academic stances and scientific reputations (2). As such, determination of the relative contribution of each coauthor to the collaborative work is of much significance, and is often reflected in the order of the authorship byline or in comments describing the differential contribution of each of the coauthors to the article (3).
science
contribution
collaboration
opinion, science, society, insight, economist
peer, publish, publication, review, preregistration
Tenzing: documenting contributorship using CRediT

Scholars traditionally receive career credit for a paper based on where in the author list they appear, but position in an author list often carries little information about what the contribution of each researcher was. “Contributorship” refers to a movement to formally document the nature of each researcher’s contribution to a project. We discuss the emerging CRediT standard for documenting contributions and describe a web-based app and R package, tenzing, that is designed to facilitate its use. tenzing can make it easier for researchers on a project to plan and record their planned contributions and to document those contributions in a journal article.
publication
article
r package
planning
credit
peer, publish, publication, review, preregistration
library, open, preservation, psa, tool