A Federal Agency’s Guide - Transition to Electronic Records

by Dan Rotelli | August 19, 2019

Our current Administration has set a course for achieving transformational change through modernizing our electronic government. This paper outlines how to become compliant with this Federal directive.

Background and Outlook

In March of 2018, the President issued a President’s Management Agenda[i] (PMA) establishing a sweeping vision to focus on improving the following six key areas: our country’s mission, customer service, stewardship, information technology, data transparency, and the workforce of the 21st century.

This will improve agency efficiency, effectiveness, and responsiveness to citizens by converting paper-based processes to electronic workflows, expanding online services, and enhancing management of government records, data, and information.

The Federal Government spends hundreds of millions of taxpayer dollars and thousands of hours creating, using, searching, and storing paper records. Storage facilities and handling electronic requests for paper records force citizens to conduct business where records can only be extracted manually - and at great expense.

All federal agencies are directed to comply with all records management, laws, and regulations. More specifically, agencies must:

  1. Ensure that all federal records are created, retained, and managed in electronic formats with appropriate metadata; and
  2. Consistent with records management laws and regulations, develop plans to close agency-operated storage facilities for paper and other, analog records, and transfer those records to Federal Records Centers operated by NARA or commercial storage facilities.

The Challenges

Federal agencies spend billions of dollars supporting paper-based records practices. Far too many government services still depend on paper. This use of paper causes increased costs for managing the movement, processing, and storage of huge volumes of paper records. Efforts to solve this problem have been inconsistent and ineffective agency-wide.

Antiquated, unsecure technology presents real risks for poorly digitized documents. And taking into account outdated duties being performed with outdated skillsets, it’s reasonable that countless hours in documenting and recording information are at risk.

Silos of information and recordkeeping hurt cross-agency collaboration. The resulting fragmentation of citizen services creates unnecessary delays, and excessive costs related to the delivery of basic services.

Another challenge is navigating the vendors, software, and technology available in today’s marketplace. With intelligent software solutions on the rise, and the existence of proven, legacy solutions – the choice isn’t an easy one.  


The Solution

Federal agencies must adopt a comprehensive, lifecycle approach[iii] to records management. This includes stopping all paper processes to every extent possible and properly digitizing existing paper records.

The Electronic Records Mandate includes the oversight of records management to the NARA. This safeguards the authenticity of archival records, ensures compliance, and maintains record-keeping practices. Furthermore, the addition of strict metadata requirements ensures the plan for master data management stays intact.

This push for electronic records also creates a more nimble and effective approach to the technologies and skillsets needed to ensure the federal workforce meets current and future needs.


The Opportunity

Ultimately, adopting electronic records management will improve citizen services by preserving public access to records, and creating cost-saving efficiencies and processes. It will become easier for the public to connect with the federal government to apply for and receive needed services.

By digitizing records today, agencies also win. Decreasing records storage space opens up more real estate for workers and an opportunity to cut costs. Additionally, streamlining workflows which previously required accessing or creating paper records reduces stress and limits costly errors.


The Deadline for the Acceptance of Paper Records

December 31, 2020 marks the end of NARA’s acceptance of paper records. This forces agencies to direct attention and resources to maintain compliance. While many agencies, such as USCIS and the National Records Center have centralized millions of paper records into a single facility, there is still a demand for digitizing these records with the required metadata that enables search, retrieval, and integration with software applications.

What is the Required Metadata?

Federal agencies are responsible for managing electronic records in accordance with NARA statutes and regulations[iv], and the Federal Records Act[v]. Metadata are the elements of information that answer the critical questions regarding a record. Essentially, the who, what, where, when, and why.

Metadata provide administrative, descriptive, and technical information regarding the structure, and content of the record. Additionally, metadata elements provide contextual information that explains how the records were created, used, managed, maintained, and how they relate to other records[vi]. There are currently 15 required metadata fields[vii].

Where to Start?

Begin exploring electronic records requirements, and related problems / concerns now. Don’t wait for a vendor to tell you they have the all-encompassing solution! Successful agency-wide transition requires a great deal of knowledge, judgement, and wisdom on your part. It’s most likely that a combination of technologies and strategies will best support your agency and processes.

Learn the basics of how automated records systems are created and how they function. When you have a more detailed understanding of data and information management principles and techniques, you’ll better understand how to extract the required metadata and integrate more accurate information.

Your goal is to learn to speak the language of data and be able to perform basic records data modeling. It sounds complicated, but experienced vendors have the expertise to guide you in what you need to learn.

Identify key data and information management players within your agency. Who manages your central databases? Who’s involved with solving data and information management issues? Who is responsible for risk assessment, management, and audits? Begin forming partnerships with these individuals in order to connect the dots between these functions. They may not all coordinate today, and your success will depend on a program which leverage the expertise of each of these areas.


Key Technologies for Success

Successful electronic records conversion and management requires automating the collection of metadata. This is best achieved from the following core technologies:

High-Quality, High-Speed Scanners for Paper and Microform

There’s nothing better than having a high-speed scanner that produces excellent digital copies of records.

When owning a scanning center doesn’t make sense for your agency, look for vendors who will not only create digital images of your records, but also perform the metadata extraction and data integration with your systems. Whether you have microfilm, microfiche, or books of physical records, conversion centers with intelligent processing technology are becoming more and more cost-effective compared to in-house scanning.

Image Processing

Image processing enhances images displayed to users throughout record workflows, removes image artifacts known to interfere with optical character recognition (OCR), provides crisp record versions for permanent archival, and analyzes page structure to assist with automations in downstream processes. Image processing applies to all forms of physical records, not just paper.

Optical Character Recognition

Optical character recognition (OCR) is the electronic conversion of typed or printed text. If you are familiar with OCR, you may think converting paper to digital text is obvious, and does not need to be listed. Although OCR technology has been around for a very long time, there are recent, modern advances that make acquiring accurate metadata much easier.

Great OCR starts by removing everything on a page that isn’t text. This is done through image processing. Humans easily recognize characters on a page, and even words that are misspelled aren’t a problem. But when machines read records, the margin for error is much smaller.

Document Management System

If you don’t already have a document management system, this will be the place to store your electronic records. In order to ensure maximum accessibility to electronic records, choose a document management system which provides both integration with existing (or future) software applications and capabilities for managing required metadata.

A search for 'enterprise content management systems' will reveal top technologies available today (I've also listed some in the links below). More advanced systems include workflow management, collaboration tools, e-signature capability, email integration, and more advanced records searching.

While solutions exist for both cloud and on-premise electronic records storage, there are solid arguments for choosing either option. For most agencies, a blend of cloud and on-premises storage is likely the best option. They key is to consider scalability and long-term costs.

Records Classification

Classification is critical for extracting accurate metadata. Classification is a method of organizing and grouping records using machine learning models that are trained to recognize different records types. This may sound complicated, but today’s technology makes training machines (software) easy, and doesn’t require a data scientist.  

Classification is performed using many different methods such as natural language processing, text-based rules, and by the visual layout of records. Now, even photographs and other media types can be classified.  

Data Integration

After all records information has been processed, it needs to be added back to the record as metadata, and the data in the record needs to be integrated into a document management system for easy retrieval and archival with NARA. Data integration systems will migrate records between systems, generate the text files needed to search and discover the record, and use external databases to verify information found on the record.

Additionally, data integration tools create human-readable records. This includes creating PDF versions of records which are intended to be viewed on-screen or printed, if needed. The previously mentioned image processing techniques are an important part of creating a permanent document image, optimized for human viewing.


Vendor Selection

There are multiple technologies available for every stage of the electronic records journey. Vendor selection is an important consideration. A key component of establishing high-quality vendor relationships is the creation of a requirements list.

The very best way to ensure success with any vendor is through the help of a certified business analyst (BA). A BA has the experience to help guide the creation of a thorough requirements list and will foresee and mitigate future roadblocks and compatibility problems.

Many vendors provide an analyst as part of their services and organizations like IIBA provide BA resources and referrals for additional help.


[i] https://www.performance.gov/PMA/Presidents_Management_Agenda.pdf

[ii] https://www.performance.gov/GovReform/Reform-and-Reorg-Plan-Final.pdf

[iii] http://www.indiana.edu/~libarch/ER/macpaper12.pdf

[iv] 36 CFR Chapter 12, Subchapter B; https://www.archives.gov/about/regulations/regulations.html

[v] 44 U.S.C. Chapters 21, 29, 31, 33; https://www.archives.gov/about/laws/

[vi] https://www.archives.gov/records-mgmt/bulletins/2015/2015-04.html

[vii] https://www.archives.gov/records-mgmt/bulletins/2015/2015-04-appendix-a.html

[viii] https://www.w3.org/TR/NOTE-datetime

[viiii] https://www.govinfo.gov/content/pkg/CFR-2018-title36-vol3/xml/CFR-2018-title36-vol3-part1235.xml



4 Steps to Achieving Wisdom You can Use at Work Today

4 Steps to Achieving Wisdom You can Use at Work Today

How to create an Information as a Second Language program. [Free Guide]

4 Steps to Achieving Wisdom You can Use at Work Today

4 Steps to Achieving Wisdom You can Use at Work Today