Skip to main content

NISO Alternative Assessment Metrics (Altmetrics) Initiative

In June 2013, the Alfred P. Sloan Foundation awarded NISO a grant to undertake a two-phase initiative to explore, identify, and advance standards and/or best practices related to a new suite of potential metrics in the community. This initiative was a direct outgrowth of a breakout discussion group during the altmetrics 12 meeting in Chicago, IL. This project, which was accomplished in two phases, is seen as an important step in the development and adoption of new assessment metrics, which include usage-based metrics, social media references, and network behavioral analysis. The NISO Altmetrics Initiative also explored potential assessment criteria for non-traditional research outputs, such as data sets, visualizations, software, and other applications. The first phase, which took place from 2013-2014, exposed areas for potential standardization and the community collectively prioritized those potential projects. The second phase, which took place from 2014-2016, advanced work in several areas and developed those into recommended practices prioritized by the community and approved by the membership.

Phase 2 Projects

NISO Voting Members reviewed and approved a proposal to develop several standards or recommended practices during Phase 2 of the Altmetrics Initiative. Areas/topics to be addressed are:

  • Development of specific definitions for alternative assessment metrics – This working group will come up with specific definitions for the terms commonly used in alternative assessment metrics, enabling different stakeholders to talk about the same thing. This work will also lay the groundwork for the other working groups.

  • Definitions for appropriate metrics and calculation methodologies for specific output types – Research outputs that are currently underrepresented in research evaluation will be the focus of this working group. This includes research data, software, and performances, but also research outputs commonly found in the social sciences. The working group will come up with recommendations for appropriate metrics for these research outputs and will develop guidelines for their use.

  • Development of strategies to improve data quality through source data providers – Data quality is essential before any alternative assessment metrics can be used for research evaluation. This working group will look at issues of data quality and will recommend strategies to overcome these issues, or to clarify the limitations of particular assessment metrics.

  • Promotion and facilitation of use of persistent identifiers in scholarly communications – Persistent identifiers are needed to clearly identify research outputs for which collection of metrics is desired, but also to describe their relationships to other research outputs, to contributors, institutions, and funders. This working group will work closely with other initiatives in the space of identifiers.

  • Descriptions of how the main use cases apply to and are valuable to the different stakeholder groups – Alternative assessment metrics can be used for a variety of use cases from research evaluation to discovery. This working group will try to identify the main use cases, the stakeholder groups to which they are most relevant, and will also develop a statement about the role of alternative assessment metrics in research evaluation.

Documents

Related Publications

Recommended Practice

NISO RP-14-2014, NISO SUSHI Protocol: COUNTER-SUSHI Implementation Profile

Defines a practical implementation structure to be used in the creation of reports and services related to harvesting COUNTER reports using the NISO SUSHI Protocol.

ISBN: 978-1-937522-45-2
Document Details
Standard

ANSI/NISO Z39.93-2014 The Standardized Usage Statistics Harvesting Initiative (SUSHI) Protocol

This standard defines an automated request and response model for the harvesting of electronic resource usage data utilizing a Web services framework that can replace the user-mediated collection of usage data reports. It was designed as a generalized protocol extensible to a variety of usage reports. An extension designed specifically to work with COUNTER reports is provided.

The standard is built on SOAP (Simple Object Access Protocol) for transferring request and response messages. The GetReport method is used for transferring ReportRequest as the input message and returning ReportResponse as the output message.

The standard is accompanied by a versioned Web Services Description Language (WSDL) to describe the Web service namespace and operations, and an XML schema constraining the syntax of the SUSHI transaction. Rules for report naming are outlined and complemented by an external reports registry, which provides for the definition of both COUNTER and non-COUNTER reports.

ISBN: 978-1-937522-47-6
Document Details
Standard

ANSI/NISO Z39.7-2013 Information Services and Use: Metrics & Statistics for Libraries and Information Providers Data Dictionary

This standard identifies categories for basic library statistical data reported at the national level, and provides associated definitions of terms. In doing so it deals with the following areas: reporting unit and target population, human resources, collection resources, infrastructure, finances, and services. In addition, the standard identifies new measures associated with networked services, databases, and performance. The standard is not intended to be comprehensive in scope. Instead, it presents a framework for comparable library data by describing common elements pertaining to libraries of various types in the US. It does not address detailed statistics for specific areas where it seems more appropriate for experts in those areas to make recommendations (e.g., music, government documents, maps).

ISBN: 978-1-937522-15-5
Document Details

NISO Altmetrics Phase One Output

NISO has released a White Paper summarizing the Phase 1 work, summarizing the findings from the meetings and interviews, along with the identification of potential action items for further work in Phase II of the project. More than 250 ideas were generate

Committee Roster

Steering Group

Altmetrics WG A: Definitions & Use Cases, Co-Chairs

Altmetrics WG A: Definitions & Use Cases, Members

Rodrigo Costas

Centre for Science and Technology Studies, University of Leiden

Beth Martin

University of North Carolina Charlotte, J. Murrey Atkins Library

Altmetrics WG B: Output Types & Identifiers, Co-chairs

Altmetrics WG B: Output Types & Identifiers, Members

Joy Painter

California Institute of Technology (Caltech) Library

Altmetrics WG C: Data Quality, Co-chairs

Altmetrics WG C: Data Quality, Members

Tilla Edmunds

Director, Web of Science Content Management
Clarivate Analytics

Angelia Ormiston

Johns Hopkins University Press/Sheridan Libraries of Johns Hopkins University

Zohreh Zahedi

Centre for Science and Technology Studies, University of Leiden