| الوصف: |
This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/13149713 . This review reflects comments and contributions from Nicolás Hinrichs, Saeed Shafiei Sabet, Melissa Chim, David Makoko, Chalermchai Rodsangiam, Martyn Rittman, Queen Saikia, Konstantinos Geles, Neeraja M Krishnan, Vanessa Bijak, and Stephen Gabrielson. Review synthesized by Stephen Gabrielson. The meta research piece explores factors for formal testing of efficacy of interventions that aim to improve reproducibility and replicability. Results are summarized in bubble charts and the authors mainly claim that the evidence-base of research seems limited across the board. Minor comments: I appreciate the breadth of the overview on the matter that the maps provide, as well as the clarity in the portrayal of single issues, but I find the lack of metrics accompanying the claims worrying. These would need to be summarized, at the very least, so as to enable end-users of this kind of material; it is actually surprising, given how carefully the methods were designed. Regarding the authorship, it would be good to be more precise about which subject areas they had expertise in, and hence (as a limitation of the study) where there were gaps in expertise. It would have been valuable to present the discipline percentage of the pre-screened studies cohort to assess the presence of potential bias in the screening procedure and/or the absence of intervention studies in these specific fields of study. Other than that the review is methodologically sound. Perhaps the authors can explain how to interpret the evidence gap maps a bit more. For instance, it is not intuitive to me what causes overlapping bubbles versus two well-separated bubbles. In regards to the section "Disciplinary scope of interventions", the map itself provides an amazing portrayal and even has a useful granularity on the quantification of the issues, so I'd expect metrics to sustain these specific ideas here (at ... |