SPOT (Semantic Processing for Object Tagging): AI-Enriched Metadata for Cultural Heritage Collections

SPOT (Semantic Processing for Object Tagging) is a collaborative research project with the Victoria and Albert Museum (V&A) investigating the use of artificial intelligence (AI) to improve metadata quality and search functionality in cultural heritage collections.

SPOT focuses on the identification and documentation of secondary or overlooked elements in collection images, such as objects depicted within artworks, decorative details on artefacts, or background features that are often absent from existing metadata. By analysing a highly heterogeneous collection, including 3D objects, plaster works, paintings, fabrics, and other diverse artefacts, the project aims to significantly enhance the discoverability, accessibility, and interpretative richness of digitised cultural collections. This varied dataset allows us to assess and improve the robustness of current object recognition algorithms across different media, ensuring that the nuances of each material and form are accurately captured and represented.

The project applies state-of-the-art AI techniques, including object detection, image segmentation, and semantic analysis, to extract and record detailed metadata such as object types, spatial coordinates, dominant colours, and semantic relationships. These outputs will be designed to integrate with collection management systems and support user-focused improvements in search and exploration of collections.

Beyond technical development, SPOT examines ethical considerations of AI in cultural heritage such as bias, provenance tracking, and narrative integrity and explores how AI-generated metadata can responsibly support curatorial workflows and diverse audience needs.

SPOT is a foundational step towards larger research exploring scalable AI-driven metadata enhancement across cultural institutions, with the goal of contributing to both academic discourse and practical museum applications.

SPOT is a research project working with the Victoria and Albert Museum (V&A) to improve how people search and explore museum collections online. Many objects in museum collection photographs, like animals, plants, or buildings in the background of artworks, are not mentioned in the current metadata records, making them hard to find. Using artificial intelligence (AI), SPOT automatically identifies these missing details and adds extra information, like what objects are in a picture, where they are, what colours they are, and how they relate to other things. This helps make online collections easier and more interesting for everyone to use, while also ensuring the information added by AI is checked, accurate, and used responsibly.

Anticipated key findings:

- Demonstration of how AI can enrich incomplete metadata by identifying undocumented objects and visual features in heritage collections.
- Creation of scalable workflows for integrating AI-generated metadata into museum systems.
- Development of best practices for responsible, ethical use of AI in heritage documentation, addressing bias and transparency.
- Improved user experiences through richer, object-level metadata that supports advanced search, discovery, and browsing.
- Foundational research to support future large-scale funding applications and cross-institutional AI metadata enrichment projects.