Program and Service Development, Implementation, and Assessment
The development, implementation, and assessment of programs and services involves asking “How can I make ideas and information useful?” In order to answer that, I have to know what information and ideas I am dealing with, and then I can proceed to develop, implement, and assess programs and service that revolve around those ideas or that information. This is slightly different from content development and management, but there are many similar skills involved in both goals.
Beyond books, programs and services serve as the “meat and potatoes” of a library in many communities. If libraries were mere information repositories, then why would people identify them any differently than they do bookstores? The reality is that while both organizations make books and other information resources available, it is the manner in which libraries connect people with information that sets them apart. Librarians and other information professionals link patrons with information through the use of programs and services. Such programs and services might include one-time speaker events, such as authors, or ongoing services like a career center, writing help, and so on. The implementation of such services isn’t just a matter of saying “We should have a career center!” or something similar and designating a space for it; it is the ongoing process of determining what a library community needs or is demanding, how a program or service can come to fruition, when it’s appropriate, how it will be paid for, etc.
The assessment of programs and services, as with the application and assessment of other managerial practices, strategies, and decisions (outlined in Program Goal C) can involve a number of different methods, from surveys to feedback forms and beyond.
To me, the idea of developing programs and services is really intriguing. Programs and services aren’t always going to be brand-new and cutting-edge; patrons and stakeholders alike have general expectations about the kinds of programs and services different libraries offer. These programs and services, of course, depend on a number of different factors, such as patron demographics, budget, and staff availability, but I find the possibility of taking established ideas and expectations and working with and around those limitations in unique and innovative ways invigorating.
A number of different courses throughout my time in the University of Southern California’s Master of Management in Library and Information Science (MMLIS) Program offered me the opportunity to think about the development, implementation, and assessment of different library programs and services. Sometimes assignments focused on one particular aspect, such as evaluation methods, as was the case with my Research Methods in Library and Information Management (LIM 504) course assignment. For this assignment, I designed and wrote a focus group moderator guide, meant to assess the opinions of a group of consumers who indicated that they enjoy jelly beans as a snack food.
Another example is an instructional unit I developed for my Instructional Strategies for Information Professionals (LIM 511) course, which I presented during one of the final live sessions for the Fall 2016 semester. This presentation served as the capstone to a series of “Be The Librarian” assignments involving the instructional unit design plan, assessment plan, and other components, and could easily transition into a program or service offered to instructors at an academic library, or be adapted for use at a public or school library.
The third course in which I developed my ability to design, implement, and assess programs and services was the first one in my Research and Professional Applications series (LIM 591A). My teammates and I investigated the social media tools used at the Clinton Presidential Library in Little Rock, Arkansas, and an assessment of how they could either improve their utilization of existing tools or make use of different social media channels. This assignment began with a reference list of 10 sources on the use of social media in libraries in general, followed by drafts of both a paper and our eventual video presentation evaluating the library’s use of social media, and culminated by a final version of both the paper and the video presentation. A library’s social media services are one way in which they communicate with patrons; a library can offer “Facebook Messenger Reference,” for example.
The fourth and final course I used to develop my skills in designing, implementing, and assessing programs and services for enhancing the use of information and ideas is actually a course I am currently enrolled in, the fifth course in the Research and Professional Applications series (LIM 591E). As my internship course, it has afforded me a number of unique experiences to put my education to use, including assisting with the implementation of a Data Rescue event at the UC Davis Peter J. Shields Library, part of the large Data Refuge project and the End of Term Web Archiving Project. I wrote about the experience and what it taught me about library programming in a blog I am currently writing alongside my classmate.
Focus Group Moderator Guide
Artifact course context and relationship to goal. Just as this moderator guide demonstrates my ability to apply and assess management strategies, practices, and decisions (Program Goal C), it also shows how I am capable of applying and assessing programs and services. A focus group is one particular method of obtaining feedback from a target audience; in this way, a focus group can be a program, and the assessment comes from examining the feedback patrons give to the moderator (an information professional) during the focus group. The focus group moderator guide is just that –a guide— and does not indicate any right or wrong answers. Part of assessing a library program or service means knowing what sort of feedback to look for and not tossing any that don’t meet stated criteria, but instead delving deeper into why one may not have gotten the desired information.
For example, let’s say that I work in programs and services at a public library, and library management wants to know what impression patrons have of the on-site career center. We distribute short feedback forms to patrons using the career center and after a month, tally them up. For patrons that left their contact information, we group them into segments based on their rating of the center on a scale of one to five. Library management is obviously hoping we get lots of people rating the career center’s services as a five, but that doesn’t mean we shouldn’t pay attention to those that rated it one through four. In fact, we might need to pay more attention to those users, since they likely have good ideas about how we could make the center even more useful for them. Management determines that holding a focus group is the way to go, and I meet with the people who rated the career center’s services as a one or a two. In this way, I am not only assessing existing library services, I am also developing and implementing new programs and services, if just for the sake of evaluating others. In order for any assessment method to be useful, and produce actionable data, it needs to be developed and implemented with the same careful consideration as any other program or service.
The artifact itself demonstrates how someone can use a focus group as a means of assessment by having a clear, easy-to-understand format. The guide suggests possible questions and follow-up inquiries based on participant answers, ways of tracking and sharing answers with others and establishes a small set of ground rules (policies) for overall expectations of behavior from the participants. It also acts as a set of strategies and best practices to aid in moderator decision-making during a focus group session.
Roles, responsibilities, and process. My LIM 504 course required that I conduct a focus group with my classmates on the topic of my choosing, supposing that my fellow students qualified for the focus group based on prior market research. The guide that I created walked moderators through the focus group process, from identifying the group that guide should be used for, to detailing the questions moderators should ask participants. The moderator guide features possible probing questions that moderators could ask to gain deeper insight into a participant’s answer. This is similar to how an information professional might conduct a research interview in order to better assist a patron, or how a library manager might work with other employees to identify and resolve conflicts or issues.
As a focus group moderator, I pretended to be from the Jelly Belly candy company, where the results of the focus group would impact what sort of flavors the company might make in the future. The process of writing the guide required me to develop useful strategies for handling the information that I gathered during the focus group session. It included best practices that I wrote in clear, concise language, such that other moderators could use the guide for their own groups. At the conclusion of the focus group, I had to evaluate whether the guide I wrote served its intended purpose and if I would need to make any changes to improve its effectiveness: an assessment of an assessment if you will.
Skills and abilities. For any assignment submitted throughout the MLIS program, I utilized various technology skills, since the program exists entirely online. Using WebJunction’s Competency Index for the Library Field (Gutsche & Hough, 2014), these technology skills can be broadly classified as Core Technology Competencies, and include E-Mail, Hardware, Internet, Operating Systems, Software Applications, and Web Technologies for accessing course materials online, creating and editing shared documents, and compositing multiple videos together for submission.
Additionally, for the creation and use of this artifact, I used Collaboration and Leadership skills by working with my peers as a moderator for my focus group, and as a participant in their groups. A moderator may guide focus group members but ultimately serves to “build an environment of trust” (Gutsche & Hough, 2014, p. 7) that empowers participation, collaboration, and honest feedback. Similarly, a library leader may not have “manager” in their title, but he or she still empowers others to speak up, work together, and “take ownership in decision-making” (Gutsche & Hough, 2014, p.10) processes. My moderator guide had to take into account the variability of participant responses so that a focus group leader could “[anticipate] and [adapt] to change and challenges effectively” (Gutsche & Hough, 2014, p. 11). Regardless of whether users are called customers, patrons, or some other term, information centers involve information seekers and gatekeepers. As a moderator for a focus group, I used the guide to seek information from my classmates, but I also acted as a gatekeeper by asking specific questions. I employed these Customer Service skills to gather feedback on how consumers perceive the performance of the Jelly Belly company and what the organization could improve, a process Gutsche & Hough (2014) call analysis, evaluation, and adjustment. My moderator guide exists on the assumption that the Jelly Belly company wants to introduce new flavors, but get customer feedback about what flavors to introduce.
As representatives of an organization, focus group moderators must employ Learning and Innovation skills throughout the focus group process. In the creation of my moderator guide, I not only had to adapt my thinking to consider how focus groups could relate to library and information services but reflect critically on the process: what questions did I not have the chance to get to? Were there questions that participants brought up that provided useful information, but were not otherwise factored into my guide? Did my guide make any incorrect assumptions about participants? Did my guide serve its intended purpose? As an information professional, there will likely be times when I have to process user feedback of some kind, and I will be responsible for critically thinking about how to act on it. Furthermore, writing the moderator guide meant I had to think about what kinds of questions to ask to clarify the various points of view that would provide the best data. Even though my hypothetical focus group participants all had something in common (having purchased jelly beans as a snack food in the past three months), the purpose of a focus group isn’t to get the same answer from every participant, but to understand the variety of responses from people who have at least one other thing in common. Focus group moderators and library managers alike need to “use creative and innovative approaches” (Gutsche & Hough, 2014, p. 13) to solicit diverse ideas and feedback, incorporate those ideas into working strategies, and refine them into best practices that benefit the organization and its patrons.
Citation Rules & Tools Presentation
Artifact course context and relationship to goal. As previously mentioned, this artifact served as the capstone of my “Be The Librarian” assignments from LIM 511. It includes how library instructors can develop, implement, and assess university instructors participating in the unit. The instructional unit focuses on the integration of citation rules and tools into any curriculum, rather than having library instructors give a one-time course during an undergraduate’s freshman year that may not stick with them throughout their college career. This artifact is an example of a program or service that could be implemented at an academic library and assessed, not just by the participating instructors, but by their students. In other words, how well the program does is dependent on how well the teaching librarian teaches the teachers!
Roles, responsibilities, and process. In my LIM 511 class, I was entirely responsible for building the instructional unit, which included a unit design plan, original infographics and activities to use in the unit, an assessment plan to determine if unit participants achieved their goals, and finally, presenting a summation of the unit in a live presentation at the end of the semester.
In developing an instructional unit for a specific audience, I had to consider how suburban research university professors would respond to each component. Though I knew that I would be presenting my unit summary to my professor and classmates, I imagined it as being for other librarians, perhaps those working at an academic library and looking for new programs and services for their staff and faculty. Those same librarians would need “selling points” to market such a unit as a program to their university’s staff and faculty.
When presenting my instructional unit, I had to integrate all of the “Be The Librarian” assignments into a cohesive explanation of the unit, its target audience, why a librarian would want to teach it, and how they could do so effectively. This presentation afforded me the opportunity to clarify portions of the assignment that may have been lacking when I submitted individual parts, such as the connection between the Student Learning Outcomes (SLOs) and the rubric in the assessment plan.
Skills and abilities. For any assignment submitted throughout the MMLIS program, I utilized various technology skills, since the program exists entirely online. Using WebJunction’s Competency Index for the Library Field (Gutsche & Hough, 2014), these technology skills can be broadly classified as Core Technology Competencies, and include E-Mail, Hardware, Internet, Operating Systems, Software Applications, and Web Technologies for accessing course materials online, creating and editing shared documents, and compositing multiple videos together for submission.
For the creation of this presentation, I tapped into my General Programming skills to develop an instructional unit meant to be used for an ongoing program offered in an academic library by an instructional librarian.
I also used Learning and Innovation skills to not only increase my understanding of different learning theories but also to integrate them into an instructional unit geared toward other instructors, so that they could then apply those theories with their own first-year undergraduate students. Since such an instructional unit could draw a diverse crowd of participants, the unit had to build in a level of flexibility, to allow the teaching librarian the opportunity to adapt the unit to his or her audience, solve problems alongside the participants, and address the myriad perspectives inherent in an interdisciplinary group.
I also utilized Leadership and Project Management skills in the development of the entire unit, including the presentation. In assuming the role of an instructional librarian responsible for the unit’s development and implementation, I created an academic library program (or series of programs) that would allow me to inspire instructors from across several disciplines to collaborate and develop content that would accomplish mutual goals. This empowers instructors not to follow a standardized formula in developing their curriculum, but to take ownership of the ideas they generate during the unit and apply them to their specific needs. The instructional librarian would ideally align the library’s mission to foster information literacy and provide access to resources necessary for university members to conduct their research with the focused goals of each instructor for their freshman-level courses. In employing “project management principles and procedures in the planning and implementation of programs and services” (Gutsche & Hough, 2014, p. 11), the presentation and all of the other “Be The Librarian” components that went into it leave nothing left unevaluated. The assessment plan portion mentioned toward the end of the presentation details SLOs for both the instructors-as-students and for the first-year undergraduates that are the participants’ students. It also includes clear expectations for unit participants in the form of a simplified rubric, one that can be easily adapted to reflect the instructional librarian’s chosen activities for the unit.
Clinton Library Social Media Presentation
Artifact course context and relationship to goal. This presentation serves as an examination of a library’s existing services, namely its online services in the form of interaction through social media. While some may argue that a library’s online presence doesn’t constitute a service, I personally beg to differ: any library can turn its online presence into a service by utilizing it to communicate with patrons. The availability of a library or other information center on social networks like Facebook, YouTube, Twitter, and others automatically gives stakeholders –patrons and would-be patrons alike– the opportunity to learn more about the resources available. When those same social media channels bring attention to online resources offered by the library, a library’s social media presence serves to connect patrons to electronic information. This fits under the standard definition of a service, it’s just being delivered through a different medium.
Those seeking to develop, implement, and assess services involving social media would do well to look at what other, similar organizations are doing, as we did with our evaluation of the Clinton Library’s social media presence. We compared their use of YouTube, Twitter, and Facebook with how other presidential libraries use those same channels, and what other libraries utilize that the Clinton Library did not. This is not dissimilar from a library comparing its programs and services to other libraries in the area, or libraries with similar demographics. Whether a library is seeking to develop a new social media service, implement a strategy for communicating with patrons via social media, or assess its existing social media presence, this presentation demonstrates how examining an institution’s programs and services can take many forms.
Roles, responsibilities, and process. For this team project from my LIM 591A project, I served as team leader and was responsible for different components of the presentation, including its design. I worked with my teammates to develop a reasonable order to our analysis of the different social media channels that the Clinton Presidential Library utilized at the time of our presentation. Each person was individually responsible for a particular social media channel and analyzing both its use at libraries in general and at the Clinton Library. We each took screenshots of relatively current examples of the Clinton Library’s use of each of the social networks, which we then integrated into the slides I designed using Apple’s Keynote software. We met over Skype or Google Hangouts several times to discuss who would be talking about what before recording each person’s video portion of our presentation using YouSeeU via the Moodle learning management system. I recorded both the introduction and the portion on Facebook, before wrapping up the presentation with a brief conclusion. Once everyone’s videos were recorded with their slides playing alongside, I composited the video and submitted it to our instructor.
Skills and abilities. For any assignment submitted throughout the MMLIS program, I utilized different technology skills, due to the online nature of the program. Using WebJunction’s Competency Index for the Library Field (Gutsche & Hough, 2014), these technology skills can be broadly classified as Core Technology Competencies, and include E-Mail, Hardware, Internet, Operating Systems, Software Applications, and Web Technologies for accessing course materials online, creating and editing shared documents, and compositing multiple videos together for submission.
Additionally, I made use of Collaboration skills in working with my team to choose specific social media channels to examine, choose relevant literature from our reference list of 10 sources, and eventually present an assessment of the library’s use of these selected social media networks during a video presentation. I led all the team meetings and provided editorial coaching for my teammates when needed, thereby contributing in a constructive manner toward our team’s goals and objectives. Each of us assumed a share of the responsibility needed to bring our presentation to a successful conclusion, rather than viewing the teamwork necessary as burdensome.
Finally, as the leader of Team Phoenix, I demonstrated Leadership skills by recognizing and leveraging the strengths of my teammates and inspiring them to use those strengths to produce their best possible work. This enabled me to handle the more technical and design-oriented portions of our presentation, while my team divided up the research necessary to properly evaluate the Clinton Presidential Library’s social media presence.
#DataRescueDavis Event Blog Post
Artifact course context and relationship to goal. As part of the LIM 591E internship class, students have different experiences in a variety of different library environments. For myself, as a library intern at the UC Davis Peter J. Shields Library, I’ve had the opportunity to put many of the skills I’ve learned about throughout my education to good use. One such example is the #DataRescueDavis event, where I got to help with the implementation of a one-day program in the library’s new Data Science Initiative space.
Though writing a blog (Sweet, 2017) isn’t required by the course, I wanted to keep a public record of the work I am doing throughout my internship and the lessons I am learning as I apply my MMLIS education to work at a real library. While not a journal recounting specifics the same way my instructor-only journal is for the course, my blog post highlights how what I learned at the #DataRescueDavis event can be applied to library programming –from development to assessment– in general.
Roles, responsibilities, and process. Though UC Davis library management already planned the event before my internship began, and continue to assess its success even weeks afterward, I directly aided in the implementation of the event by attending several preparatory webinars, both on my own and with other library staff. I also designed signage and name tags for the event Guides, people including myself that walked participants through using a special Google Chrome browser extension to “nominate” specific web pages for the larger Data Refuge and Presidential End of Term Web Archiving projects.
Over the course of the day, I helped integrate newcomers into my “nomination” team by helping them understand the purpose of our project –to back up data from government websites not just as a response to vast changes proposed by the current administration, but as part of a traditional presidential End of Term Web Archiving project– as well as help them with downloading the extension, familiarizing themselves with the fields, and getting started nominating a particular government web page (where possible). I kept track of who nominated what web pages, answered questions, helped with technical problems that arose, and assigned new pages as people finished. Toward the end of the day, I also helped clean up the space that we used, since we had been there for about seven hours and had far more people than we anticipated show up, resulting in a lot of leftover napkins, paper plates, and other ephemera from our free coffee, bagels, and pizza.
Skills and abilities. The #DataRescueDavis event required that I use several skills from WebJunction’s Competency Index for the Library Field (Gutsche & Hough, 2014). These include Digital Resources Technology, Hardware, Networking and Security, Operating Systems, Public Access Technology, Email, Web Technologies, and Software Applications skills. Already being familiar with Google Drive (including Docs, Sheets, and Slides) helped immensely, especially since so many of the files provided to us by the University of Pennsylvania’s Penn Program in the Environmental Humanities (PPEH) Lab were in these formats. Additionally, during the event, I had to use Google Sheets to track my team’s progress in our assigned sub-primer; with everyone working from their own starting URLs, I frequently had to stop to update the sheet and email that individual a new URL to start on (if they didn’t have to leave).
Working with others at various stages throughout the event, I also utilized several skills in the Personal/Interpersonal set, such as the Collaboration skill. Everyone participating in the event had the common goal of ensuring the continued availability of the EPA’s data, but not everyone was on the same page as far as how to go about doing that. By acting as a Guide or team leader, I was able to effectively help others help us– and likely thousands of other students and researchers.
This also involved many opportunities for my Leadership skills to shine, as I was given complete direction over my team. Though the event organizers (Kevin and Vessela) also participated in nominating pages and occasionally served as Guides for their own small teams, the Guides didn’t interact with each other– teams were generally separate, tackling their own sub-primer of EPA offices. This meant it was up to me to help others work in accordance with the greater UC Davis Library event goal (archiving the EPA’s website), and also inspire them to trust their own judgment about what was crawlable and what was not.
Despite the lengthy webinars I attended ahead of time, I actually had to learn and innovate on-the-go at the event, since I hadn’t gotten the opportunity to actually use the Nomination Tool extension beforehand. Similarly, I didn’t know what website I would be helping to archive, so there was no way of knowing what pages might look like to get an idea of what “uncrawlable” would actually look like to me. I had to critically examine websites, a page at a time, looking for reasonable links to elsewhere on the same office website to push the crawler to get as many pages as possible, and figure out what to do when the content wasn’t where it should have been.
Future Work Environments
While developing programs and services is not necessarily an everyday task, their continued development, implementation, and assessment are vital to the longevity of any library. Patrons might go to a library to get a book or utilize the space in some way, but programs and services are what bring them back or get them to stay for extended periods. If information resources like books are akin to a noun in a sentence, then programs and services act as the verb; at a library, programs and services are what you do.
Gutsche, B. & Hough, B. (Eds.) (2014). Competency index for the library field 2014 (2nd ed.). Dublin, OH: OCLC Online Computer Library Center, Inc. Retrieved from http://www.webjunction.org/documents/webjunction/Competency_Index_for_the_Library_Field.html
Sweet, M. (2017) Save That Data! Unexpected Adventures in Academic Library Programming – The Library Interns. Retrieved from https://libraryinterns.meredithsweet.com/library-programming-adventures