Versions Compared

Old Version 19

changes.mady.by.user Jay Hill (Unlicensed)

Saved on

compared with

New Version 20

changes.mady.by.user Jay Hill (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Expand
titleGoals

What does success look like? What metrics can we effect and why it is important to affect those metrics?

Goal

Metric

Why Important?

Reliably extract citations from Authority Documents

>= 80% accuracy where 20% of Citations need to be reworked (e.g., split, merged, rejected …)

If there is poor accuracy requiring extensive human correction, then there is little value.

Reliably extract glossaries from Authority Documents

>= 95% accuracy where only 5% of term-definition pairs need to be reworked. Glossaries are substantially easier to identify and extract than citations.

If there is poor accuracy requiring extensive human correction, then there is little value.

Reliably automate the end-to-end process of capturing, transforming, and loading STIG, NIST 800-53, FedRAMP, eCFR compliance content into the Unified Compliance platform.

100% of identified Authority Documents for all four compliance content contributor sources is loaded into the UCF.

All four Authority Documents sources are related to securing and hardening IT infrastructure for both the private and public sector.

To provide value to customers with Security Operation's requirements, UC needs to maximize the breadth of security coverage to ensure we can provide security guidance for as many IT assets as possible.

Scope and

...

Features

Expand
titleSection Explanation. Click to expand.

The intent of this section is for the following:

Scope Definition: defines the scope of the proposed product (or features), including what will and will not be included helping manage expectations and focus development efforts.

Guideline for Development: provides detailed information on the product’s features, functionalities, user flow, and interface to guide the development team in building the product.Framework: provides high-level evaluation criteria for alternative solutions (build, buy, partner) to evaluate different routes to success.

Expand
titleFeatures

Describe the product features that will bring value to customers and fulfill underserved need(s).

FeatureComments

Use Case / Problem Solved

Automatic Citation and Glossary Extraction from Authority Documents

Citations are passages in the Authority Document that:

contain

Out of all the ancillary information found in Authority Documents, specific passages containing Mandates (requirements)

OR

and related contextual information such as stubs, informational, and informational gathering are extracted from the Authority Documents.

Human-in-the-loop Training for Content Extraction AI Models

Specifically for the Citation and Glossary extractionExpert mappers can validate and make changes about the correctness of the AI-driven Citation extraction helping train the AI model to produce increasingly better results.

Automated Compliance Content Ingestion into the Common Data Format

Initially Authority Documents (initially targeted for STIGs, NIST 800-53, FedRAMP, and eCFR) are loaded into the UCF platform with minimal human intervention.

Monitoring and Logging

Administrators can make informed decisions and take action on the ingestion process based on meaningful information is collected throughout compliance content ingestion process.

Metadata and Data Change Detection

Compliance professionals are kept to up to date with changes made by compliance content providers.

Statistical Data Capture and Reporting

For each step in the ingestion pipeline with critical data capture of AI model accuracyProduct and engineering teams can review statistical results captured during the document ingestion process to help make objective data-driven decisions about the accuracy and effectiveness the process automation.

Expand
titleOut of Scope / Future Functionality

List the known features that are out of scope for this project or might be revisited at a later time.

As is case with the assumptions, it is important to list these out so that architects and engineers can plan accordingly for these later updates.

Feature

Comments

Tagging and Mapping of content to the Common Controls.

This project ends at the AD, Citation and Glossary extraction, transformation, and load.

Follow-on projects will include the tagging and mapping.

Human-in-the-loop for content capture.

Later projects can include additional human validation. To limit scope of this project, steps such as metadata change detection can be reviewed and validated after the fact looking at logs and other information.

Human-in-the-loop for the AD cataloging.

Same as above

Human-in-the-loop for transformation into the common data format

Same as above

Human-in-the-loop for loading into the UCF

Same as above

Corpora Management and Administration

A non-production version of a corpora exists. Any work on the corpora is out of scope for this product.

Corpora Data Loads

The focus of this PRD is to load compliance content into the UCF 4.0 application for customer consumption over the API.

Follow-on projects can tap into the pipeline and use the content for other purposes.

Fit-for-purpose frontend for each step in the content ingestion process.

UX/UI for any other aspect of the process outside of human review and approval of suggested citations and suggested term-definition pairs.

Later product updates could include additional front-end steps for tasks throughout the process.

...