• AI 4 AV (Artificial Intelligence for Audiovisual): Design and Evaluation of a Shared System for Libraries, Archives and Museums (LAMs)

    Author(s):
    Aaron Choate, Tanya Clement, Maria Esteva (see profile) , Ruizhu Huang, Hannah Robbins-Hopkins, Weijia Xu
    Date:
    2020
    Group(s):
    DH2020
    Subject(s):
    Archives, Artificial intelligence, High performance computing, Machine learning
    Item Type:
    Presentation
    Meeting Title:
    Digital Humanities 2020
    Meeting Org.:
    ADHO
    Meeting Date:
    July 20-24
    Tag(s):
    audiovisual, Descriptive metadata standards, Las Historias, storyCorps, Transcriptions, High-performance computing
    Permanent URL:
    http://dx.doi.org/10.17613/b20r-hx50
    Abstract:
    Audiovisual (AV) materials are predominant historical and scientific records of our times, and their numbers are increasing exponentially in collecting institutions. Tasked with preserving and making AV materials available, libraries, archives, and museums (LAMs), need to find efficient and scalable curation solutions. Using machine learning (ML) to generate metadata is promising, but to adopt such methods information professionals must overcome a host of technological and cultural challenges. We are conducting research around the design and evaluation of a system that uses ML to translate audio to text, classify, and describe AV collections using open computing infrastructure that can be used by multiple LAMs. The project leverages IDOLS, a web-based API platform as the gateway to DeepSpeech, and natural language processing learning (NLP) tools installed in High Performance Computing resources. As a testbed we use Las Historias, a StoryCorps collection of oral histories from the Chicano and Latino immigrant populations. The collection’s metadata is the foundation for training classifiers and for validating the results. We are examining the entire workflow, in order to evaluate what a “good” system might be for LAMs working with cultural artifacts.
    Metadata:
    xml
    Status:
    Published
    Last Updated:
    3 years ago
    License:
    Attribution-ShareAlike

    Downloads

    Item Name: mp4 zoom_0.mp4
      Download View in browser
    Activity: Downloads: 123

    Back to Deposits