Skip to main content
PLOS Unveils New Way To Measure Open Science Practices

PLOS Unveils New Way To Measure Open Science Practices

December 2022

NISO Member News

San Francisco | December 12, 2022

The Public Library of Science (PLOS) today announced the release of the first results of a new initiative to measure researchers’ Open Science practices across the published literature, ‘Open Science Indicators’ (OSI). These initial results show increasing prevalence of good practices in research data and code sharing, and increasing use of preprints by researchers. PLOS is releasing the OSI dataset, and the associated framework used to develop it, to support Open Science initiatives in the wider community.

Earlier this year PLOS partnered with DataSeer to develop and measure, using artificial intelligence and natural language processing, a set of numerical “indicators” linked to specific Open Science characteristics and behaviors observable in thousands of published research articles. Today, PLOS is making available the data on three Open Science Indicators: data sharing, code sharing, and preprint posting, and will be developing additional indicators in 2023 guided by feedback from the community.

OSIs will help PLOS better understand researchers by establishing a baseline for the current state of adoption for Open Science practices and allow PLOS to track progress over time. As a framework for measuring and monitoring Open Science, OSIs have broad potential applications, from complementing the aims of the UNESCO Open Science Monitoring Framework Working Group to meeting the needs of organizations that wish to better understand Open Science practices, and assessing the impact of policy changes, like those set forth in the recent OSTP memo.

“To support our goal of increasing adoption of Open Science practices we need reliable evidence to understand those practices, but until we developed Open Science Indicators, there were no tools that could tell us everything we needed to know,” said Iain Hrynaszkiewicz, Director, Open Research Solutions at PLOS. “Open Science Indicators will enable us to better understand and support the researchers we serve, and by sharing OSIs openly we can help others support the adoption of Open Science practices.”

“We’re very proud to be helping PLOS with the OSI project,” said Tim Vines, Founder and Director of DataSeer. “It’s so much easier to make good choices around Open Science incentives when you’ve a clear picture of sharing behaviors across the whole of your portfolio.”

PLOS’ initial dataset examines data-sharing and code-sharing behaviors, and preprint posting in both PLOS articles, and the same Open Science practices in the wider scientific literature. The data cover approximately 61,000 PLOS research articles published between January 2019 and June 2022, as well as a comparator set of 6,000 publicly available research articles from PubMed Central (10% of the PLOS sample). The OSI method will be further developed and improved over time.

Data highlights:

  • All Open Science Indicators are increasing in PLOS articles and in comparator articles from 2019 to 2022
  • 23% of PLOS articles had associated research data that have been shared in a repository, according to good practices in 2019, rising to 28% in 2022. For comparators the rates are less than half those observed in PLOS articles, with 9% of articles using detectable data repositories in 2019, rising to 15% in 2022.
  • While PLOS articles are more likely to link to an associated public dataset than comparable articles published elsewhere, when viewed over time, a generally positive trend is seen in repository usage and data sharing, by any method, overall.
  • Code-sharing rates among PLOS and comparator articles are similar, rising from 6% of articles in 2019 to 16% of articles in 2022. The dataset also reveals, for the first time, insights into which research projects generate code and software and those which do not.
  • PLOS articles are more likely to have an associated preprint than comparable articles published elsewhere. Overall, 21% of PLOS articles in the dataset have an associated preprint, compared to 19% of comparator articles.

In September, PLOS provided a summary of our plans and some preliminary results. With this first complete release of the data, we explore only a fraction of the many ways in which the dataset can be analyzed to understand Open Science practices. For example, readers can delve into data- and code-sharing practices, differentiating between sharing as Supporting Information (SI) or in a repository. Data on research topics and geography can also be used to explore trends in different research communities. In the future PLOS plans to expand on this dataset with new data points and ways of presenting them, additional publication years, and new indicators related to other aspects of Open Science practice.

About These Organizations

About the Public Library of Science

PLOS is a nonprofit, open access publisher empowering researchers to accelerate progress in science and medicine by leading a transformation in research communication. Since our founding in 2001, PLOS journals have helped break boundaries in research communication to provide more opportunities, choice, and context for researchers and readers. For more information, visit http://www.plos.org.

About DataSeer

DataSeer brings the power and scalability of Artificial Intelligence to promoting Open Science. We help stakeholders like journals, funding agencies, or institutions monitor & showcase their research outputs. We also guide individual researchers through sharing the data, code, and protocols that underlie their articles. Through our partnerships with research stakeholders, DataSeer will empower the Open Science revolution. Find out more at https://www.dataseer.ai/.