Document Type

Article

Publication Title

Internet Reference Services Quarterly

Publication Date

2026

Keywords

AI, evaluation, databases, vendor tools

Abstract

Library database vendors have recently begun introducing AI tools to their platforms at an increasing pace. While some of these tools cannot be turned off, some can, leaving libraries to make the decision with little in the way of past practice to rely on. There has been considerable research done on the generally available large language models (LLMs) but less on methodologies for evaluating platform add-on tools. This paper outlines the development of such an evaluation process at an R2 academic library in the US, from conception to implementation. It offers practical insights on bringing in stakeholders across the library and recognizing their expertise while also being cognizant that this is added work. The complete evaluation tool is provided for readers and outcomes of three initial evaluations are discussed, along with ongoing concerns and vendor relations. The article is tightly focused on practical applications of the tool and advice for librarians. Readers will find themselves with a tool they can implement in their own institutions and background on its development to aid in scaling it up or down and modifying it to suit their own institutional needs. This will help ensure that library platforms can best meet the needs of their patrons.

Funding Source

This article was published Open Access thanks to a transformative agreement between Milner Library and Taylor & Francis.

Creative Commons License

Creative Commons Attribution 4.0 International License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Comments

First published in Internet Reference Services Quarterly (2026): https://doi.org/10.1080/10875301.2026.2720440

DOI

10.1080/10875301.2026.2720440

Download

Share

COinS