Notes on my Book

None of the methods can measure diversity of viewpoints represented or breadth of countries represented (e.g. Education in Israel). The closest we can get is looking at countries as subject matter, and even then mostly in the History section.

We don't know impact of use (Chris Bourg's phrase), what did it mean to the person? Was it central or just a glance? We often don't know if students are using what their professors want them to be using, or if someone satisficed and used what we had just because it was there.

Chris Bourg's Beyond Measure talk says we need more idealism. Focusing on value that is measurable could result in "a devaluing of libraries writ large." Our value is in how we embody values. The ways in which we matter to people can't always be measured - [the stories people hear about libraries are often more about saving lives than serving a specific information need - me] There is a value in us providing an alternative point of view that maybe someone wasn't looking for.

Chu says that faculty may send requests that focus more on their own interests instead of seeing what the university as a whole needs. Chu's colleagues find that faculty focus on research needs over general materials. I don't think our faculty would.

MacAyeal says attendance at ARL Assessment Conference doubled in 6 years: 215 people in 2006, when the conference was initiated and 560 in 2012.

Felix Chu's institution faculty think of the librarian roles as procedural - they keep track of funds and channel order requests to the right people. Especially if you are not a subject expert you may be fighting a perception that you don't know what the library ought to have.

There are sometimes better ways of finding out users' needs than just asking them what they need - their actions count, too.

Data helps you show others that you know what you are doing, and not only that it helps you know more about what you are doing. Knowing what you already have and how it's being used is information faculty don't have, so it makes sense to become experts at this kind of knowledge and decision-making.

You don't have to evaluate your whole collection at once or do all the methods. You could do only a citation analysis of your Psychology collection, for example. However, some methods really work best if you're able to compare different parts of the collection to each other, so think about whether you'll get enough data if you are narrowly focused. If the method involves comparing your library to others, it may be okay to use just a small piece of your own library.

Bodi and Maier-O'Shea say you should always break down your assessment by subject. I'd agree pretty much.

lists

counts of books - ILS, WorldCat

counts of use - ILS, can analyze in an assessment tool

ILL - from ILLiad, WorldCat Resource Share, Docline, or other tool

your budget

other libraries' collection sizes - WorldCat, assessment tools, other libraries' catalogs, NCES Compare Academic Libraries

citations - Web of Science, student work, repository, faculty work

How long are you going to spend on this? When will you finish this phase or round?

How often will you repeat the study?

ARL, accrediting agencies.  I need to actually look up what ARL collects.  Brooks-Kieffer says 18 measures of expenditures, 15 of holdings, 6 of activities, and 5 of staffing. ACRL, which I fill out every year, has 8 measures of print collections, 12 of expenditures, and 18 of electronic resources (holdings and use) - check on this.  NCES, which I also do, has 5 measures of collections, 7 of staffing, 1 of activities, and 26 of expenditures. Our organizations have really liked quantitative data.

Brooks-Kieffer p.7 says the most common data collection tools for academic libraries are ACRL's Academic Library Statistics and ARL's statistics publications, also mentions data collection by NCES and IMLS. They are way not meaningful.

Johnson p.268 says you're measuring not only the collection itself but how it gets used. What good is a collection if it doesn't get used? She also says we think we're measuring quality, but the "real intent is to measure the collection's utility" p.268

You are looking at: amounts of items, specific titles, formats, age, language of materials, extent of coverage (Johnson says breadth and depth, though I don't really talk about this), and use. All formats! Though evaluation was invented in a print-based world, electronic resources are such an important part of collections now that an evaluation would be useless if it did not take them into account.

see Henle and Cochenour about how they involved the reluctant arts selector.  Communication needs to be open from the beginning!

You need to ask people what they need to know. meet with your admissions office, or the programs you're supporting. This not only gets their attention but means your data will be relevant and useful.

http://flowingdata.com/category/tutorials/

ManyEyes http://www-958.ibm.com/software/data/cognos/manyeyes/

http://www.tableausoftware.com/public//

https://developers.google.com/chart/

https://venngage.com/

In a very general sense, you're always learning about your collection so you can manage it better. This is always the case, whatever your specific goal.

You need to know what you need more of

might help you decide what to cut

it might create a baseline so you can measure improvement

to figure out how you're supporting a particular program, esp if it's new

to train a new person

helps you speak language that will jibe with your administrators, assessment, data-driven decision-making

Lynch et al - study of 6 large public universities CEOs, Chief Academic Officers say "all respondents said they pay attention to qualitative input ... as well as quantitative assessments." p.222  CEOs preferred comments from members of the academic community over comparisons to other institutions, even if the former is anecdotal.  Three CEOs liked to have both kinds of data and one said comparative data was the only serious measure of quality. But many were skeptical of whether the data had been manipulated, whether they described something meaningful or not. Most were distrustful of numbers describing the size of the collection as measures of its quality.  All 6 presidents felt that circulation was important, as did all the CEOs.