#plumx

The confusing world of Altmetrics – A personal opinion, with confusing data

I am a fan of Altmetrics. At least in concept. But I starting to get very concerned with both the tools used to measure them and what the “numbers” are expected to indicate. We would expect that a high “number” in an Altmetric.com “donut” would be indicative, in some way, of the relative importance or “impact” of that article. One would hope it at least points to how well read the article is,…

We work hard to make sure that our customers can use the metrics we gather about their research where they want to see them. Our newest addition to our growing list of widgets is the Grant Widget. This widget allows grantmakers and other funders and foundations to embed important information about grants that is contained in PlumX into their web pages and systems.

Below is an example of an grant from Autism Speaks. The name of the grant is “Identifying Genetic Variants on the Y Chromosome of Males with Autism.” This grant has the following research output associated with it:

25 Articles

3 News Stories

1 Blog

Here is the widget:

Notice that in the list of research output (artifacts) you can see the Plum Print for each one so at a quick glance you can tell something about what is happening with this research. 

From the widget, you can click the “see details” link to see the Grant Detail page in PlumX.

If you scroll down this page, it also has a table of the output. Below is an example of that.

You can click on any of the titles of the output to see the detailed artifact page.

This grant widget joins our growing list of widgets:

Artifact Widget (Plum Print)

Researcher Widget

Group Widget

Presenting a guest post from Steve Giannoni, Director of Sales (Academic & Govt) UK & Ireland, EBSCO Information Services, on his thoughts on the library's role with the research office.

Having spent almost 10 years of my EBSCO-life working predominantly with Academic Libraries, last week I found myself at a conference attended by over 700 delegates with only 3 people containing the word ‘Librarian’ or ‘Library’ in their job title. I was at my inaugural, and the 25th anniversary, ARMA conference which amongst other things, brings together those responsible for managing and administering the research output of Higher Education Institutions (HEIs) in the UK.

It’s been a long time since I have been exposed to a world beyond library-centric discourse, where topics such as discovery, eBooks, journals, databases and other end-user primed services acquired by information professionals, consumed by students, researchers and academics were set aside in favour of discussions about ‘research output’, ‘quantitative metrics’, ‘grant-funding compliance’ and the ‘REF’ (Research Excellence Framework).

Although the conference was attended by directors, managers and administrators of the research office, the overwhelming agreement from the dozens of people I met with at the event, was that Libraries and Library professionals absolutely have a part to play in the support of research and the data-wrangling necessary to help the research office evidence what attention, engagement and impact the research output of the institution has on the outside world.

Libraries have an inherent expertise in curating data in this digital information age and I believe this competence contributes to the reason why we see more and more ‘research support’ librarian roles coming to the fore. Libraries are increasingly being recognised as important campus partners, aiding the institution’s operational response to the requirements of national assessment processes, not least for example (in the UK), in preparation for the next REF (perhaps in 2020?).

Under the shadow of OA policies which internally and externally mandate newly accepted articles be deposited in the Institutional Repository, the Library increasingly sees its efforts being cross pollinated with the research offices’ need to manage (compliantly) the published output in order to meet funder’s requirements. In Library circles, discussions about the relationship between OA publishing and research impact are set to increase and I expect Libraries to be more involved in the research office’s activities as they attempt to showcase their competencies and enhance their presence within the institution. Fledging OA University Press initiatives in the UK and North America are drawing on library knowledge and experience to take them forward – further evidence of a changing role.

Back at the ARMA Conference, there were many parallel sessions spanning the 2-day event which focused on impact, metrics and research funding. A common theme emerged in that as competition grows in the pursuit of grants and funding, there appears to be an ever increasing interest in applying organisations to look beyond Citation metrics as a measure of the past success of an institution’s research output. Has the research world suddenly moved away from Citations as a means of impact measurement and replaced the ‘Gold-Standard’ with a newer, hipper alternative?

Well, no. This is where PlumX comes in. Referred to as more of an ‘All-metrics’ rather than Alt-metrics tool, PlumX provides both Citation metrics alongside other quantitative ways of tracking the output of a researcher, a department an institution or alternatively, research that has been produced as a direct result of funding from an external source, such as a research council.  It’s no wonder that the funders of research are themselves investing in PlumX, so that they can track, measure and analyse the various ways in which the research they are funding is being interacted with. An example of this is Autism Speaks who in July 2014, began using PlumX for this very reason.

PlumX can be used as supporting evidence within a narrative (e.g. case studies) when applying for future research grants now that social, economic and political engagement with researcher’s outputs can be interpreted in ways not before possible when looking at Citations alone.

The integration between PlumX and DSpace makes it simple for DSpace customers to get started quickly to have PlumX metrics, dashboards and analytic reports.

When we work with new customers to implement PlumX we always start with the best place for them. One good example is their Institutional Repository. When a customer has already gathered their research output into an IR we work with them to create the best integration for both initial setup and ongoing maintenance.

DSpace offers a function known as communities that allows users to divide their research into groups and hierarchies. These communities usually reflect their institution, for example, schools and departments. PlumX mirrors the communities to automatically create PlumX research impact dashboards for each DSpace community.

Below is an example from Tomas Bata University in Zlin in the Czech Republic. In their DSpace implementation they have four top communities, each with several communities underneath in the hierarchy. You can see this in the picture below.

In PlumX, the top four communities are mirrored.

When you click on any of these in PlumX you drill down to the next level in the community hierarchy.

For each level you have all of the features and functions of a PlumX Dashboard, including:

List of research output or artifacts

Table of metrics for all artifacts

Analytic reports

Ability to export the data

Data filters 

Once the community and hierarchies are mirrored in PlumX, PlumX uses the OAI protocol for metadata harvesting to get all of the articles and other research output from the DSpace repository. Then, PlumX gathers the metrics and creates the reports. The research output does not need a DOI in order for PlumX to find metrics.

Once the metrics are gathered you can embed the PlumX metrics widget into the DSpace repository. Some PlumX customers have reported that by providing all of these metrics within the IR, authors have seen a benefit to depositing their research because they can go back to the IR to see the reach and impact of their work. We call this the “carrot” approach to IR compliance. We wrote a paper about it that you can access from this blog post.

Below is an example of an article in the University of Manitoba DSpace repository. At the bottom of this article, you can see PlumX metrics in one of the available widgets (there are several).

A closer look at the PlumX metrics:

Notice that you can see metrics for different versions of the article, not just the one in the DSpace repository.

From DSpace you can link to the record in PlumX where you have access to more features such as transparent detail behind the metric numbers and the list of tweets and blog posts about the article or artifact.

We love working with customers and their institutional repositories. We have implemented a variety of IRs including:

DSpace

Fedora

ePrints

bepress Digital Commons and Selected Works

Custom IRs

We work closely with all of our new customers and are constantly learning new things that add to our repertoire of tools for IR integration. 

The Association to Advance Collegiate Schools of Business (AACSB) recently held their International Conference & Annual Meeting (ICAM). Plum Analytics co-founder, Mike Buschman, was invited to be a panelist on a panel titled, “Enabling Impactful Scholarly Research” with Eric Bradlow, Vice Dean and Director, Wharton Doctoral Programs, The Wharton School, University of Pennsylvania and Gregory J. Gordon, President, Social Science Research Network. The panel was moderated by Dan LeClair, Executive Vice President and Chief Operating Officer, AACSB International.

According to LeClair, this was one of the most popular sessions at the conference. LeClair summarized the session. One of the things he said in his summary:

As research outputs take on new forms, including pure-online journals, recorded virtual conferences, and software code, and as people begin to use research in different ways, technology has been enabling us to measure usage, influence, and impact accordingly. With help from new companies, we are finding ways, for example, to track book utilization by chapter, document when a piece of software is used or built on, and register each time an article appears in a doctoral seminar reading list. These alternative metrics will help business schools to demonstrate that their research makes a difference by providing a richer set of tools beyond citation analysis, and to look beyond the current view that refereed journal publications are the ultimate “end game.”

We are happy to announce a new artifact-level metrics page. An artifact is any piece of research output. Some of the most popular artifacts include:

Article

Blog Post

Book

Source Code

Theses/Dissertation

Video

Below is the new Artifact-Level Metrics page. This is the largest redesign of PlumX to date. Since we launched PlumX three years ago, we have interacted with 100s of institutions in dozens of countries. We took all of their feedback to heart to enhance our product.  

This page has several improvements to help you uncover and tell the stories of your research output. 

First, we are now summarizing the metrics in each category. Much like the Plum Print, this will make it easier for you to understand at a glance what is happening in each category.

For example, in this article you can see a metric for the total usage, with a breakdown of each sub-category. This is an easier way to digest these numbers. For full transparency, you can still easily expand the category to see all of the detail and where possible access the underlying data.

Another important new feature of this design is the exposure of the latest tweet and blog post about the artifact. 

Above is an example of this article with the most recent tweet highlighted. You can easily view all of the tweets about this article with a click of the button.

We do a similar thing with blog posts.

Go here to read more about how we handle Tweets, and here for Blog Posts.

From the beginning we knew that seeing metrics for individual articles would never be enough for institutions to be able to really uncover the stories about their research. So, PlumX was built with the ability for customers to define and import their own groups and hierarchies. Here are a few examples:

On the new PlumX artifact page these groups and hierarchies are better exposed.

In the example below this researcher from the University of Pittsburgh is part of the faculty in the School of Medicine. This research is in the areas of Cell Biology and Molecular Physiology and Pediatrics. You can then easily navigate to these other areas from this article to see related research and researchers.

Blogs are an important part of today's scholarly communication ecosystem. Researchers often write about their own or others’ research, share data and carry on conversations that push their field forward. This is why we’ve been working hard to expand our coverage of blog mentions in PlumX. 

PlumX tracks five categories of metrics -- usage, captures, mentions, social media and citations (go here for more information). Mentions is the category where we uncover and track the conversations about the research. Blogs are a new source of Mentions metrics. And, because Mentions are buried in thousands of disparate sites that may not be focused on research topics all the time, these can be difficult metrics to find.

When a blog post refers to specific research output we will collect information about the post, including a snippet, to give you an easy way to see the blog posts in context within PlumX. We will also provide a link to the post itself.

Here is an example of an article we track in the field of economics.

You can see that this article has been referenced in two different blogs. By clicking on the hyperlinked number you are taken to a PlumX page with the title and snippet of each post.

From here you can click on the title in PlumX to see the blog post itself. 

Another interesting Mention that PlumX reveals about this article is in a Wikipedia article about Agency Cost.