Project Outcome : Year in Review

2016
Report

Table of Contents

20
15

20
16

Library Impact

Wherever public libraries are working, possibility lives.

People who work in public libraries know that library services open new opportunities for anyone who enters – putting people on the path to literacy, technological know-how, or a better job. We see evidence of this every day – what libraries have long been missing is the data to support it.

Throughout this Annual Report, when you see this color, please click for interactive functionality and more information.
Through Project Outcome, we are learning more than ever about the benefits that library users see in their own lives. Of over 17,000 surveys, we heard:

Why does this matter?

Because the impact of public libraries extends beyond the individuals who use them – it strengthens and empowers the community around them. How a parent reads to a child has lasting effects on that child’s literacy. Knowing how to use computers and the Internet is essential for people who need better jobs or further education to support their families. Communities with better social connectedness and civic engagement have been associated with less unemployment.
Project Outcome was designed to help public libraries understand and share the true impact of their services and programs – arming libraries with the data they need to continue that impact. We provide libraries with simple survey instruments and an easy-to-use process for measuring and analyzing outcomes. The results can be used to demonstrate the value of library services, make plans to improve them, and decide how to allocate limited resources.

The results from our first year follow.

Our impact is just getting started!

What
Project
Outcome
Did
What
Libraries
Did

Made outcome measurement accessible to all libraries

Adopted outcome measurement in their communities – even faster than we expected

Provided libraries with ready-to-use patron surveys across 7 essential library service areas. Surveys measure immediate patron benefit following a library service or program, and longer-term behavior change.

More than 225 library systems implemented Project Outcome surveys in their communities.

Over 17,000 survey responses were collected within the first year across 774 programs & services– an average of 64 surveys every month. The most popular surveys were Summer Reading, Education/Lifelong Learning, and Early Childhood Literacy.

Provided interactive Data Dashboards for libraries to analyze, interact with, and take action using their results.

Utilized the data visualizations to better understand and talk about their results internally with staff and externally with their Boards.

Provided more than 50 training resources to support library adoption of Project Outcome, like step-by-step guides and talking points to explain outcomes to library users.

Downloaded free tools and resources more than 3,000 times. The most popular resources were Outcome Measurement Process, What is Outcome Measurement, and Getting Started – helping libraries get informed on the basics.

Spread the word about why outcome measurement matters – and how Project Outcome can help

Learned about outcome measurement and shared their experiences with one another

Provided practical guidance for adopting and implementing Project Outcome to individual libraries, state libraries, and other groups at 3 free in-person trainings held at ALA and PLA conferences and 26 webinars and presentations.

More than 300 library directors and staff from 150 libraries participated in the in-person trainings and presentations, and almost 3,000 participants joined Project Outcome webinars.

Took Project Outcome on the road in presentations at 12 state, consortium, and partner conferences around the country.

Librarians and library leaders joined Project Outcome presentations to share their experiences and encourage other libraries to implement outcome measurement, connecting directly with more than 750 library staff.

Partnered with organizations and leaders committed to helping libraries demonstrate their impact, like the Association for Rural and Small Libraries, Chief Officers of State Library Agencies, Research Institute for Public Libraries, University of Washington Information School (Impact Survey), TechSoup, ALA’s Public Programs Office, Urban Libraries Council, and state library staff.

More than 1,000 libraries registered in the first year alone, from all 50 states plus D.C. and Canada.

Kept the conversation going through articles in library publications:

 

Committed to being open and transparent about our results

Used their survey results to improve their library programs and services

Kept our rapidly growing Project Outcome community informed about how libraries across the country were adopting and implementing the surveys. We shared monthly updates with more than 1,800 Project Outcome users (starting March 2016), all PLA members (starting September 2015), and 132 staff from state libraries and associations (starting November 2015). And we kept the conversation going between updates, with more than 100 Facebook posts.

Strategic planning and advocacy anecdotes:

  • New York library used Project Outcome results in grant proposal submission
  • Texas library used Project Outcome results to identify ways to improve its marketing of children’s programming to parents in the community
  • California library used Project Outcome results to discuss program effectiveness with administration and stakeholders
  • Utah library used Project Outcome results to improve Summer Reading

Never stopped looking for ways to improve Project Outcome

Helped improve Project Outcome, so more libraries can benefit from outcome measurement

Invited input and ideas from Project Outcome users every step of the way:

  • Mid-year evaluation for all users to collect feedback on the overall project
  • Evaluations after all 3 trainings to understand immediate questions and needs
  • Participant feedback after each webinar
  • "Peer sharing" calls for users to exchange ideas and lessons learned

The Performance Measurement Task Force tested, improved, and developed new survey questions. This included a dedicated group of participants and Task Force members who evaluated and made improvements to the Summer Reading survey.

Project Outcome continually evolved its tools in response to user feedback:

  • Expanded the Data Dashboard to display community demographics alongside library impact
  • Improved Survey Portal functionality
  • Streamlined the Project Outcome website to make resources easier to find

Project Outcome users told us what worked for them and why:

  • "Simple, yet powerful"
  • "Free, standardized, easy to use at first glance"
  • "Pre-planned surveys and reporting—the hardest work is already done for me!"
  • "Thinking about the end results for planning purposes"
  • "Outcome measurement can become a reality across [the] nation based on agreed standards—so excited!"
  • "Ability to use these surveys regardless of library size"
  • "Very manageable to implement and we’ll be able to make quick changes based on responses"
  • "Helps develop strategies for programs based on reality of data"

Training evaluations told us that our in-person training became more effective each time and added value for Project Outcome users – especially for smaller libraries and those new to outcome measurement.

Cost to libraries: Free

Spent by Libraries $0

What We Have Learned Together

Data collected from the first year of Project Outcome tell us unequivocally that library programs and services improve the lives of library users. People come to the library not just for books, but for programs that will help them learn a new skill or make a specific change in their lives. In total, nearly 80% of library users surveyed report that library programs and services have had some kind of positive impact on their lives in the last year.

Project Outcome’s first year of survey data resulted in:

75% of Civic/Community Engagement survey respondents intend to become more engaged in their community life

  • Surveys conducted: 26
  • Total survey responses: 664
  • Most Common Program Types:
    Journalist Lectures; Heritage or Cultural Programming

“I didn’t know that these types of resources [about job markets] were provided by the library.”

“It was the single most constructive hour I’ve spent in the last 10 years.”

“Glad that the library and the school are partnering to help us know how to prepare our kids.”

“I’ve been reading and studying the Holocaust for a very long time, and it was always a solitary journey. It was wonderful to be in a group — in a course setting — to learn more ... and to FEEL more.”

“[The instructor] is an excellent teacher. One cannot find in any book.”

“ The ability to gain local insight and gather information quickly, allowing me to move my business forward.”

In addition, we learned that what patrons like most about the programs they’re attending is the educational aspect — from doing a new activity, to learning a new skill, to the librarian or instructor’s style of teaching. And these patron benefits are in high demand. Across every program type, the most common suggestion for improvement was to offer new, more frequent, or more current classes and programs.

What’s Next

Project Outcome is just getting started. Like the hundreds of libraries that now have outcomes data to guide their strategy and support their advocacy, we are taking what we’ve learned to increase the impact of outcome measurement on libraries and communities everywhere.

  • New and improved tools are coming soon...
  • New and improved tools are coming soon. In early 2017, Project Outcome will release new guidelines for libraries interested in administering more advanced surveys and longitudinal studies to understand the impact of their programs and services over time. These guidelines will support library efforts to write and measure their own unique outcomes, and outline how and why to work with partners to measure outcomes.
  • Project Outcome will continue to develop...
  • Project Outcome will continue to develop tools and resources and host regular online trainings to help libraries move from planning to implementing surveys to taking action using the results.
  • Project Outcome will continue to work...
  • Project Outcome will continue to work with libraries to build momentum and sustainability. We will build on the success of our first year to offer even more of the targeted learning libraries need to start measuring their impact and support they need to use the results. From offering regional training and one-on-one library assistance, to building data-sharing partnerships, to expanding our work with state library staff and other types of libraries, we will keep looking for opportunities to improve. We hope you will join us!

APPENDIX

Project Outcome is funded by the Bill & Melinda Gates Foundation and expands the reach of PLA’s Performance Measurement Task Force's work.