North Carolina Digital Repository
The documents, policies, and reports below describe the digital preservation functions of the Digital Repository and related projects at the State Archives. Included are procedures that agencies should follow when transferring electronic records to the State Archives for storage in the Digital Repository. For more information on digital preservation in North Carolina, please visit the Digital Information Management section of the State Library website.
The North Carolina Digital Repository is jointly managed by the State Archives and State Library. The State Archives stores both digital and analog materials with enduring historical value in the Digital Repository. Similar to the way in which analog materials are stored on shelves in the stacks and vault, digital materials are stored in the Digital Repository. These digital objects include:
- Born-digital State Agency records scheduled for permanent digital retention in the State Archives
- Born-digital Local Government tax records scheduled for permanent digital retention in the State Archives (pilot program)
- Born-digital Special Collections materials accessioned into the State Archives
- Digitized copies of analog materials previously accessioned into the State Archives
- Publications managed by the State Library
In its role as the custodian of permanent archival records and materials, the State Archives receive records and materials of enduring value from state agencies, local government, and private individuals and entities. In the world of electronic records, it is imperative to ensure that these records and collections transfer in a method that maintains the authenticity of the original files. Electronic files, by nature, are fragile and subject to alteration. The State Archives endeavors to receive electronic records and digital files that are complete and unaltered.
Below you will find guidelines for how to prepare, package, and transfer data to the archives, as well as standards that explain the rationale for why we transfer data.
Digital File Transfer Guidelines
This document will assist with the transfer of electronic records identified for permanent retention to the custody of the State Archives, as prescribed on the agency’s approved records schedule. The guidelines provide an overview of best practices for preparing electronic records for transfer, including the most appropriate file names and formats. This document also provides instructions about the tools used for file transfer and the procedures that should be taken to ensure a smooth transfer process. These tools and procedures allow the State Archives of North Carolina to certify the records and files in its repository as true, unchanged, and authentic.
These guidelines describe the specifications and metadata required to meet the State Archive's standards for transfer. For digitized audio and video, the State Archives of North Carolina does not accept DVDs from vendors or other agencies, as we do not have the resources to convert them to our required specifications. Thus, we require material to be digitized to our preservation standards. The State Archives also requires all metadata for the digitization project to be included as a CSV file, and all XML metadata to be included as sidecars with the files. All incoming digital files should be "bagged" using Library of Congress's Bagger Tool using a SHA-256 hash (for more information see the data transfer tutorials).
File Formats for Transfer of Electronic Records to the State Archives of North Carolina
The File Formats for Transfer of Electronic Records to the State Archives of North Carolina document provides information about appropriate file formats for transfer of electronic records to the State Archives. It is a companion document to File Formats Guidelines for In-House Preservation and Long-Term Retention, which is intended for state and local agencies who are maintaining electronic records in-house. The latter document has descriptions and explanations about file formats, whereas the File Formats for Transfer document lists only those formats recommended, acceptable, and not recommended for transfer to the State Archives.
Verifiable data transfer is a critical component of the State Archives Digital Repository. When data is transferred to the State Archives for preservation in the Digital Repository, the data should be transferred using Bagger or the BagIt command line tool. By transferring data with these tools, we can document that the files you transfer have arrived intact and unchanged. Below are tutorials designed to help you use Bagger.
Bagger GUI User Guide – A user-friendly alternative to the command line BagIt tool
YouTube Videos (Complete YouTube Playlist for BagIt Tutorial):
- Introduction (1 of 10) - 10 minutes
- Installing BagIt: Intro to Installing BagIt, Installing JRE (2 of 10) - 11 minutes
- Installing BagIt: Configuring the JAVA_HOME env variable (3 of 10) - 9 minutes
- Installing BagIt: Installing the BagIt Program (4 of 10) - 9 minutes
- Creating/Verifying Bag: Introduction (5 of 10) - 5 minutes
- Creating/Verifying Bag: Demonstrate Create/Verify a Bag (6 of 10) - 15 minutes
- Creating/Verifying Bag: Verification errors, Transfer a Bag, Conclusions (7 of 10) - 9 minutes
- Retrieving/Verifying/Unpacking Bags: Introduction (8 of 10) - 7 minutes
- Retrieving/Verifying/Unpacking Bags: Demonstrate Retrieve/Verify a Bag (9 of 10) - 15 minutes
- Retrieving/Verifying/Unpacking Bags: Verification errors, Unpack a Bag, Conclusions (10 of 10) - 12 minutes
The State Archives of North Carolina and the State Library of North Carolina have joint mandates to preserve the records and publications of the state of North Carolina.
The Digital Preservation Guidelines govern the operation, management, and scope of the North Carolina Digital Preservation Program. The North Carolina Digital Repository Requirements document lays out the requirements of both the State Archives and the State Library for a digital preservation system.
The State Archives runs a well-established website preservation program for state agency websites, available online at the North Carolina State Government Web Site Archive. State agency websites and social media accounts captured through this program archived in the Digital Repository. State agencies may opt out of the service and preserve websites on their own. For more information about the website preservation program, and for guidance on in-house website preservation, please see the Website and Social Media Archiving section of the Digital Records Policies and Guidelines page.
- RATOM (Review, Appraisal, and Triage of Mail), 2019-2020: RATOM was an Andrew W. Mellon Foundation grant-funded partnership between UNC-Chapel Hill and the State Archives of North Carolina. Its purpose was to expand on the products and workflows developed for the BitCurator and TOMES projects. Deliverables included open source software solutions for the iterative processing of email to support further selection, redaction, and description actions using Natural Language Processing and machine learning.
- TOMES (Transforming Online Mail with Embedded Semantics), 2015-2018: TOMES was a multi-state partnership including North Carolina, Kansas, and Utah focused on developing processes to transferring email accounts out of hosted email solution platforms and converting them into a sustainable open source language. Additionally, the project was developing appraisal techniques using natural language processing. It was funded by the National Historical Publications and Records Commission (NHPRC).
- Electronic Mail Collaboration Initiative & EMCAP (E-Mail Collection and Preservation Tool), 2004-2009: The Preservation of Electronic Mail Collaboration Initiative was funded by the NHPRC to address the challenges posed by electronic mail. The project aimed to establish best practices and develop guidelines, as well as develop a software application to transform email from its native format to XML. Project partners included the State Archives of North Carolina, Kentucky Department of Library and Archives, and Pennsylvania State Archives.