Versions Compared

Version Old Version 10 New Version Current
Changes made by

Jay Hill (Unlicensed)

Jay Hill (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Introduction

Expand
titleSection Explanation. Click to expand.readme

The product requirements document (PRD) is a central document used to align all stakeholders (product management, engineering, QA, designers, and leadership) on the overall objective and vision of the proposed product and is used as a decision-making toolhow we will solve a specific problem with the proposed solution.

When creating the PRD, provide just as much information as needed and nothing more. If the document is too long and complex, it will quickly become outdated, and readers will lose interest.

PRD content and structure vary by organization. Depending upon the product line, company culture, and processes, PRDs could have quite a different look and feel.

In this latest iteration of the Unified Compliance PRD template, we changed the template to help raise visibility of how the proposed product (or feature set) adheres to Unified Compliance’s strategic plan including details on why this product proposal is important to Unified Compliance.

Strategic Planning and Decision Making

How does this proposal fit into our overall vision and which specific initiative does this proposal align with and how?

The UC Strategic Plan for 2024 has two foci:

  1. content and

  2. the sale of that content

This project squarely fit into the focus of bringing in additional content.

Content Ingestion Automation - ETL is a critical aspect of the initiative to “Partner with 3rd Party to Develop Automated Content Mapping”
Expand
Expand
titleVision and Initiative Alignment
titleSection Explanation. Click to expand.

  • Vision and Goal Setting: articulates the vision alignment, problem being addressed, and goals of the product proposal describing what the product is, who it is for, and how it will benefit the users and the organization.

  • Decision-Making Framework: helps in making informed decisions throughout the product development process acting as a reference point for evaluating progress and making changes.

  • Performance Measurement: sets the criteria for measuring the success of the product through specified metrics and key performance indicators (KPIs) including potential financial impact.

  • Basis for Prioritization: helps in prioritizing features based on the product strategy, market needs, and resource constraints.

readme

Describe the problem we are solving, the high-level approach, and goals so that before we get too far into the details, readers will have a good understand of where we are headed.

Expand
titleThe Problem

What problem are we trying to solve? and why it important to our customers and/or to Unified Compliance?

We currently rely on a team of expert mappers to meticulously add content into the UCF. The process works well but is slow. With the advent of automation and AI, Unified Compliance risks attacks from competitors who will use technology to accelerate content acquisition.We risk losing customers to other platforms if we fall behind on the extent of

coverageCustomers are not able keep up to date with compliance requirements in the face of the quickly evolving best practices, implementation guide, new segments, etc.

We will also find it difficult to take on new market segments without automation.

Expand
titleHigh-level Approach

Briefly describe the approach you’re taking to solve this problem. Provide enough information for the reader to imagine possible solution directions and get a rough sense of the scope of this proposal.

The approach is to start with the “left-hand” side of the automation process where compliance content is captured from a small set of sources. Authority Documents, Citations, and Glossaries are extracted from those documentsend-to-end content ingestion process focusing on compliance content through ETL for an identified list of four (4) compliance content providers.

For each content provider, Authority Documents will be identified, then Citations and Glossaries will be extracted from the Authority Documents, transformed into the Common Data Format specification, and loaded into the UC platform.

Automation tools and AI will be used to accelerate the end-to-end process with human assistance to review and approve most critical steps in the process focusing on reviewing and updating AI suggestions.

Once the process is proven out, the intent is to extend the solution to many additional compliance content providers.

The hierarchy of zip files must be maintained to ensure follow-on functions have context.

Citations are passages in the Authority Document that:

  • contain Mandates (requirements) OR

    • Expand
    titleGoals

    What does success look like? What metrics do we measure today that we can we effect and why affect? What metrics should we absolutely add? Why it is important to affect those metrics?

    Expand
    titleRequirements

    Describe the product requirements that will fulfill the underserved need(s) starting off with the use cases, then specific functionality.

    Requirement

    Importance

    Comments

    STIG Pipeline

    Scrape the STIG document library to download all zip files.

    High

    The zip files are multi-level nested zip files.

    Unzip each STIG file to retrieve the XML files.

    High

    Store the XML files for later use.

    High

    Identify which documents within the hierarchy are Authority Documents.

    High

    The zip files may contain readme’s or other files that do not constitute Authority Documents.

    Identify which files are Glossary-specific.

    High

    Some files may solely be Glossaries with term-definition pair entries. Ensure those documents are also processed for follow-on steps.

    Detect file metadata changes from prior processing.

    High

    Potential metadata changes could be a new document or a new version of an old document.

    Only pass new or changed documents further down the pipeline.

    High

    NIST 800-53 Pipeline

    Access the GitHub repository for all NIST 800-53 content.

    High

    Retrieve and store the XML, JSON, and YAML files for later use.

    High

    Identify which documents are Authority Documents.

    High

    Identify which files are Glossary-specific.

    High

    Some files may solely be Glossaries with term-definition pair entries. Ensure those documents are also processed for follow-on steps.

    Detect file metadata changes from prior processing.

    High

    Only pass new or changed documents further down the pipeline.

    High

    FedRAMP Data Pipeline

    Access the GitHub repository for all FedRAMP content.

    Retrieve and store the XML, JSON, and YAML files for later use.

    Identify which documents are Authority Documents.

    Identify which files are Glossary-specific.

    High

    Some files may solely be Glossaries with term-definition pair entries. Ensure those documents are also processed for follow-on steps.

    Detect file metadata changes from prior processing.

    Only pass new or changed documents further down the pipeline.

    eCFR Data Pipeline

    Access the eCFR files via the eCFR APIs.

    Store files for later use.

    Identify which documents are Authority Documents.

    Identify which files are Glossary-specific.

    High

    Some files may solely be Glossaries with term-definition pair entries. Ensure those documents are also processed for follow-on steps.

    Detect file metadata changes from prior processing.

    Only pass new or changed documents further down the pipeline.

    General Pipeline (after initial source-specific tasks are completed, if possible)

    Catalog each source Authority Document.

    High

    Gather all information as is required by the Common Data format.

    Identify and extract Citations from the Authority Document

    High

    Goal

    Metric

    Why Important?

    Automate an end-to-end process to capture all STIG content (approximately 457 documents), perform ETL, and load into the UCF in common data format.

    All 457 STIGs, as Authority Documents, are available for customer consumption via API from the UC 4.0 API Gateway

    All Citations as part of the 457 STIGs are available for customer consumption via API from the UC 4.0 API Gateway

    All Glossaries with term-definition pairs as they related to the 457 STIGs are available for customer consumption via API from the UC 4.0 API Gateway

    STIGs sit at the intersection of Sec Ops and GRC. Organizations need to harden their security posture with DoD approved security measures that are in alignment with the software and hardware vendors.

    IT departments will utilize a variety of software and hardware in their data centers. Reliably extract citations from Authority Documents

    When AI is not needed, 100% accuracy.

    When AI is utilized, greater than 80% accuracy where 20% of Citations need to be reworked (e.g., split, merged, rejected …)

    If there is poor accuracy requiring extensive human correction, then there is little value.

    Reliably extract glossaries from Authority Documents

    When AI is not needed, 100% accuracy.

    When AI is utilized, greater than 80% accuracy where only 20% of term-definition pairs need to be reworked. Glossaries are substantially easier to identify and extract than citations.

    If there is poor accuracy requiring extensive human correction, then there is little value.

    Reliably automate the end-to-end process of capturing, transforming, and loading STIG, NIST 800-53, FedRAMP, eCFR compliance content into the Unified Compliance platform.

    100% of identified Authority Documents for all four compliance content contributor sources is loaded into the UCF.

    All four Authority Documents sources are related to securing and hardening IT infrastructure for both the private and public sector.

    To provide value to customers with Security Operation's requirements, UC needs to maximize the breadth of STIG security coverage to ensure we can match provide security guidance for as many IT assets as possible.

    Automate an end-to-end process to capture all NIST 800-53 content (approximately 36 files with a mixture of json, yaml, and xml documents), perform ETL, and load into the UCF in common data format.

    All NIST-800-53 content, as Authority Documents, are available for customer consumption via API from Ingested compliance content, including Authority Documents, Citations, and Glossaries, are available for access via the UC 4.0 API GatewayAll Citations as part of the NIST-800-53 documents

    100% of identified Authority Documents for all four compliance content contributor sources are available for customer consumption access via API from the UC UCF 4.0 API Gateway

    All Glossaries with term-definition pairs as they related to the NIST-800-53 content are available for customer consumption via API from the UC 4.0 API Gateway

    NIST 800-53 helps IT departments implement proper security controls to proactively take care of their organization's infrastructure.

    Automate an end-to-end process to capture all FedRAMP content (approximately 32 files with a mixture of json, yaml, and xml documents), perform ETL, and load into the UCF in common data format.

    All FedRAMP content, as Authority Documents, are available for customer consumption via API from the UC 4.0 API Gateway

    All Citations as part of the NIST-800-53 documents are available for customer consumption via API from the UC 4.0 API Gateway

    All Glossaries with term-definition pairs as they related to the NIST-800-53 content are available for customer consumption via API from the UC 4.0 API Gateway

    FedRAMP is a government-wide program that promotes the adoption of secure cloud services across the federal government by providing a standardized approach to security and risk assessment for cloud technologies and federal agencies.

    UC can assist federal agencies or organizations working with federal agencies to grow and use secure cloud technologies.

    eCFR

    …

    …

    New Customers

    New Markets

    Scope and Requirements

    Expand
    titleSection Explanation. Click to expand.

    The intent of this section is for the following:

    Scope Definition: defines the scope of the proposed product (or features), including what will and will not be included helping manage expectations and focus development efforts.

    Guideline for Development: provides detailed information on the product’s features, functionalities, user flow, and interface to guide the development team in building the product.

    Framework: provides high-level evaluation criteria for alternative solutions (build, buy, partner) to evaluate different routes to success.

    We are in the migration phase from CCH to UC 4.0. To ensure we don’t elongate the migration process, all new content must come into UC 4.0 and out the API Gateway.

    Reliably catalog Authority Documents, track versions, and detect changes

    100% of all identified Authority Documents from the four (4) source sites are automatically cataloged with 0 documents moving further into the pipeline if no metadata changes are detected.

    Before context is extracted, the Authority Documents must be inventoried, cataloged, and only reprocessed if changes are detected to reduce expensive AI processing resources.

    Scope and Features

    Expand
    titlereadme

    The section focusses on the details of the solution including what is in scope, what is out of scope and additional information to help in the product and engineering collaboration process.

    Expand
    titleFeatures

    Describe the product features that will bring value to customers and fulfill underserved need(s).

    Feature

    Use Case / Problem Solved

    Automatic Citation and Glossary Extraction from Authority Documents

    Allow human experts to view each Citation and relationships between Citations.

    High

    Allow human experts to reject a Citation.

    This could be for one single citation of many that the process got “wrong”.

    Allow human experts to reject the entire Authority Document.

    No need to reject citation by citation, if the entire document wasn’t extracted properly.

    Allow human experts to approve all or individual Citations.

    Allow human experts to change Citations.

    What might they do here?

    Out of all the ancillary information found in Authority Documents, specific passages containing Mandates (requirements) and related contextual information such as stubs, informational, and informational gathering

    .

    Maintain Citation structure.

    High

    Since Authority Documents will contain multiple Citations and passages may have related Citations, that structure must be maintained to know the relationship between Citations.

    Extract Glossary from within Authority Documents.

    High

    Some Authority Documents may have glossary within the document. This will typically be near the end of the file.

    Extract the Glossary details including the Title, source, and all term-definition pairs.

    Extract Glossary from glossary-specific files.

    High

    Some files may only have Glossary entries with term definition pairs.

    Extract the Glossary details including the Title, source, and all term-definition pairs.

    Detech content changes from prior loads.

    High

    Nee discussion here. See questions section.

    Transform the Authority Document into the Common Data Format

    High

    The transformation documentation must be used as reference as to how source document schema structures are related to the Common Data Format.

    Transform the Authority Document related Citations into the Common Data Format

    High

    As above, use the CDF transformation document as reference.

    Transform the Glossaries into the Common Data Format

    High

    As above, use the CDF transformation document as reference.

    Load Authority Documents into the Unified Compliance Platform

    High

    UCF engineering team will determine the optimal approach for loading (API, service, …)

    Load Citations into the Unified Compliance Platform

    High

    Same as above

    Load Glossaries into the Unified Compliance Platform

    High

    Same as above

    Human Validation via a simple front-end

    Since this is all back-end pipeline work with no customer interaction, the user experience needs to be good enough for us to the “dog good” but won’t be exposed to customers or partners in this release.

    are extracted from the Authority Documents.

    Human-in-the-loop Training for Content Extraction AI Models

    Expert mappers can validate and make changes about the correctness of the AI-driven Citation and Glossary extraction helping train the AI model to produce increasingly better results.

    Automated Compliance Content Ingestion into the Common Data Format

    Authority Documents (initially targeted for STIGs, NIST 800-53, FedRAMP, and eCFR) are loaded into the UCF platform with minimal human intervention.

    Monitoring and Logging

    Administrators can make informed decisions and take action on the ingestion process based on meaningful information that is collected throughout compliance content ingestion process.

    Metadata and Data Change Detection

    Compliance professionals are kept to up to date with changes made by compliance content providers.

    Statistical Data Capture and Reporting

    Product and engineering teams can review statistical results captured during the document ingestion process to help make objective data-driven decisions about the accuracy and effectiveness off the process automation.

    Expand
    titleOut of Scope / Future Functionality

    List the known features that are out of scope for this project or might be revisited at a later time.

    As is case with the assumptions, it is important to list these out so that architects and engineers can plan accordingly for these later updates.

    RequirementFeature

    Comments

    Mapping STIGs, NIST, FedRAMP, or eCFR Tagging and Mapping of content to the Common Controls.

    This project ends at the AD, Citation and Glossary extraction, transformation, and load.

    Follow-on projects will include the tagging and mapping.

    Human validation prior to the Citation Extraction-in-the-loop for content capture.

    Later projects can include additional human validation. To get limit scope of this project out quickly, steps such as metadata change detection can be reviewed and validated after the fact looking at logs and other information.

    Expand
    titleUser Interaction and Design
    Link to mockups, prototypes, or screenshots related to the requirements.

    Human-in-the-loop for the AD cataloging.

    Same as above

    Human-in-the-loop for transformation into the common data format

    Same as above

    Human-in-the-loop for loading into the UCF

    Same as above

    Corpora Management and Administration

    A non-production version of a corpora exists. Any work on the corpora is out of scope for this product.

    Corpora Data Loads

    The focus of this PRD is to load compliance content into the UCF 4.0 application for customer consumption over the API Gateway with API Responses in the Common Data Format.

    Follow-on projects can tap into the pipeline and use the content for other purposes.

    Fit-for-purpose frontend for each step in the content ingestion process.

    UX/UI for any other aspect of the process outside of human review and approval of suggested citations and suggested term-definition pairs.

    Later product updates could include additional front-end steps for tasks throughout the process.

    Dictionary term management.

    For glossaries, all that is needed for the terms is the term-definition pair. Once we move into creating and managing dictionaries, we will add more details like part of speech, synonyms …

    Links to
    Expand
    titleProcess Flow Diagrams
    flows and UX

    If we have them, link to:

    • user journeys, process flow, or other diagrams related to the requirements.

    • mockups, prototypes, or screenshots related to the requirements.

    Link here:
    Expand
    titleImpacted Product Components

    If this project is a component to other areas or an update to an existing product, specifically call out where this product will interact with other areas.

    Start writing here:
    Expand
    titleOpen Questions

    List any open questions that come to mind throughout the lifecycle of this initiative.

    Question

    Answer

    Date Answered

    What do we do with deprecated authority documents?

    How do we identify and capture AD versions?

    For STIGs, how do we identify which files are authority documents?

    For NIST 800-53, how do we identify which files are authority documents?

    For FedRAMP, how do we identify which files are authority documents?

    For eCFRs, how do we identify which files are authority documents?

    Specifically, what is required to catalog an AD?

    In this first pass, what should constitute content changes?

    We don’t want to get too crazy and make this a massive project.

    Need to discuss.

    Expand
    titleAlternative Solutions
    Provide a high-level evaluation criterion for alternative solutions (build, buy, partner) to evaluate different routes to success

    .

    Milestones and Launch Checklist

    What is the Unique Selling Proposition (USP)? Relay the key factors that separate our product from the competition and why we are the best possible solution for our prospects based on their unique needs.

    Expand
    titleSection Explanation. Click to expand.readme
    Expand
    titleRisk Mitigation

    The intent of this section is for the following:

    Monetization: Financial impact this product will introduce (if any)

    Risk Mitigation: Identifies potential risks and propose mitigation strategies.

    Launch Readiness: launch checklist including high-level go-to-market plan to ensure cross-departmental alignment.

    High-level Messaging: Includes Unique Selling Proposition (USP) raising visibility of the proposed solution’s value proposition.

    Expand
    titleHigh-level Messaging
    Expand
    titleMonetization

    Will this product be part of an existing subscription or an add-on?

    Will this product be usage based or part of a subscription?

    Identifies potential mostly focused on getting the solution “out the door” and who else is affected outside the product and engineering teams.

    Expand
    titleTechnical Risk Mitigation

    If applicable, identifies potential technical risks and propose mitigation strategies.

    Risk

    Mitigation Strategy

    The eCFR content volume could cause multiple challenges including impact on API responses, searches, dominate other content sources …

    Expand
    titleLaunch Readiness

    Identify any relevant milestones that people should now know about. Will we “eat our own” it” ourselves first? Will this require a beta? and what is the target launch date?

    Date

    Milestone

    Audience

    Description

    TBD

    Dogfood 🐶

    Internal employees only.

    Testing internally

    TBD

    Beta 🎈

    Early cohort of X customers.

    Getting user feedback

    TBD

    Public Launch 🚀

    Roll-out to all users.

    Let’s do it!

    Expand
    titleLaunch Checklist

    This section is a reminder to the product team to make sure all relevant stakeholders are involved as necessary.

    Area

    Question

    Answer (yes/no)

    Instructions if "Yes” (or unsure)

    Product

    Are we introducing new functionality that requires changes to the pricing implementation.

     Yes

    The expectation is that these changes are part of the features in the PRD.

    Customer Success

    Will new training material be needed (or updates to existing classes)?  

     No

    Talk to the Customer Success team.

    Customer Success

    Do we need a new or updated onboarding experience?

     No

    Talk to the Customer Success team.

    Support

    Will new FAQs be required (or updates to existing ones)? API documentation?

     Yes

    Talk to the Customer Support team.

    Support

    Will this functionality require new support processes like new HubSpot workflows or saved replies? Or training the support team on the product?

     Yes

    Talk to the Support team.

    Growth & Data

    Have Do we implemented sufficient need additional tracking in order to measure success and impact on user behavior for the new feature? Will UserFlow be used? Do we need a new Power BI report?

     Yes

    Review within our Product Team

    Growth & Data

    Could this impact CTAs? Or new-user-experience (NUX)?

      Yes

    Review within our Product Team

    Growth & Data

    Are we turning this product or feature on for everyone immediately or are  we are we going to use feature flags for a slow roll-out?

     Everyone

    Not applicable yet until feature flags are ready to go

    Product

    Are we running a Beta for this?

     No

    Review within our Product Team

    Marketing

    Are we introducing functionality where we will want to update or create new web pages? New/updated CTAs?

     Yes

    Talk with Marketing

    Additional References

    Expand
    titleSection Explanation. Click to expand.additional references

    List and link to any other reference sites, documents … that might be important to the reader including the business model canvas (BMC).

    STIG Overview

    STIG Document Library

    NIST SP 800-53 Overview

    NIST GitHub Repository

    FedRAMP Basics

    FedRAMP GitHub Repository

    eCFR Overview

    eCFR Developer Resources

    eCFR XML User Guide

    eCFR API Documentation