Introduction
| Expand | ||
|---|---|---|
| ||
The product requirements document (PRD) is a central document used to align all stakeholders (product management, engineering, QA, designers, and leadership) on the overall objective and vision of the proposed product and is used as a decision-making toolhow we will solve a specific problem with the proposed solution. When creating the PRD, provide just as much information as needed and nothing more. If the document is too long and complex, it will quickly become outdated, and readers will lose interest. PRD content and structure vary by organization. Depending upon the product line, company culture, and processes, PRDs could have quite a different look and feel. In this latest iteration of the Unified Compliance PRD template, we changed the template to help raise visibility of how the proposed product (or feature set) adheres to Unified Compliance’s strategic plan including details on why this product proposal is important to Unified Compliance. |
Strategic Planning and Decision Making
| Expand | ||
|---|---|---|
| ||
|
| Expand | ||
|---|---|---|
| ||
How does this proposal fit into our overall vision and which specific initiative does this proposal align with and how? The UC Strategic Plan for 2024 has two foci:
This project squarely fits into the focus of bringing in additional content. Content Ingestion Automation - ETL is a critical aspect of the initiative to “Partner with 3rd Party to Develop Automated Content Mapping”. The automated content mapping comprises the complete end-to-end content capture, ETL, and mapping to the common controls. This particular product proposal is the “left-hand” side from capture to ETL. |
| Expand | ||
|---|---|---|
| ||
What problem are we trying to solve? and why it important to our customers and/or to Unified Compliance? We currently rely on a team of expert mappers to meticulously add content into the UCF. The process works well but is slow. With the advent of automation and AI, Unified Compliance risks attacks from competitors who will use technology to accelerate content acquisition. We risk losing customers to other platforms if we fall behind on the extent of coverage. We will also find it difficult to take on new market segments without automation. |
| Expand | ||
|---|---|---|
| ||
Briefly describe the approach you’re taking to solve this problem. Provide enough information for the reader to imagine possible solution directions and get a rough sense of the scope of this proposal. The approach is to start with the “left-hand” side of the end-to-end content ingestion process focusing on compliance content through ETL for an identified list of four (4) compliance content providers. For each content provider, Authority Documents will be identified, then Citations and Glossaries will be extracted from the Authority Documents, transformed into the Common Data Format specification, and loaded into the UC platform. Automation tools and AI will be used to accelerate the end-to-end process with human assistance to review and approve critical steps in the process focusing on reviewing and updating AI suggestions. Once the process is proven out, the intent is to extend the solution to many additional compliance content providers. |
...
| title | Goals |
|---|
What does success look like? What metrics can we effect and why it is important to affect those metrics?
...
Goal
...
Metric
...
Why Important?
...
Reliably extract citations from Authority Documents
...
>= 80% accuracy where 20% of Citations need to be reworked (e.g., split, merged, rejected …)
...
If there is poor accuracy requiring extensive human correction, then there is little value.
...
Reliably extract glossaries from Authority Documents
...
>= 95% accuracy where only 5% of term-definition pairs need to be reworked. Glossaries are substantially easier to identify and extract than citations.
...
If there is poor accuracy requiring extensive human correction, then there is little value.
...
Reliably automate the end-to-end process of capturing, transforming, and loading STIG, NIST 800-53, FedRAMP, eCFR compliance content into the Unified Compliance platform.
...
100% of identified Authority Documents for all four compliance content contributor sources is loaded into the UCF.
...
All four Authority Documents sources are related to securing and hardening IT infrastructure for both the private and public sector.
To provide value to customers with Security Operation's requirements, UC needs to maximize the breadth of security coverage to ensure we can provide security guidance for as many IT assets as possible.
Scope and Requirements
| Expand | ||
|---|---|---|
| ||
The intent of this section is for the following: Scope Definition: defines the scope of the proposed product (or features), including what will and will not be included helping manage expectations and focus development efforts. Guideline for Development: provides detailed information on the product’s features, functionalities, user flow, and interface to guide the development team in building the product. Framework: provides high-level evaluation criteria for alternative solutions (build, buy, partner) to evaluate different routes to success. |
...
| title | Requirements |
|---|
Describe the product requirements that will fulfill the underserved need(s).
...
Requirement
...
Importance
...
Comments
...
STIG Pipeline and Goal:
All STIGs (approximately 457) including Authority Documents, related Citations and Glossaries with term-definition pairs are available for customer consumption via API from the UC 4.0 API Gateway
...
Scrape the STIG document library to download all zip files.
...
High
...
The zip files are multi-level nested zip files.
...
Unzip each STIG file to retrieve the XML files.
...
High
...
Store the XML files for later use.
...
High
The hierarchy of zip files must be maintained to ensure follow-on functions have context.
...
Identify which documents within the hierarchy are Authority Documents.
...
High
...
The zip files may contain readme’s or other files that do not constitute Authority Documents.
...
Identify which files are Glossary-specific.
...
High
...
Some files may solely be Glossaries with term-definition pair entries. Ensure those documents are also processed for follow-on steps.
...
Detect file metadata changes from prior processing.
...
High
...
Potential metadata changes could be a new document or a new version of an old document.
...
Only pass new or changed documents further down the pipeline.
...
High
...
No need to promote unchanged files.
Question on what to do with depreciated documents in questions.
...
NIST 800-53 Pipeline and Goal:
All NIST 800-53 (approximately 36) including Authority Documents, related Citations and Glossaries with term-definition pairs are available for customer consumption via API from the UC 4.0 API Gateway
...
Access the GitHub repository for all NIST 800-53 content.
...
High
...
Retrieve and store the XML, JSON, and YAML files for later use.
...
High
...
Identify which documents are Authority Documents.
...
High
...
Identify which files are Glossary-specific.
...
High
...
Some files may solely be Glossaries with term-definition pair entries. Ensure those documents are also processed for follow-on steps.
...
Detect file metadata changes from prior processing.
...
High
...
Only pass new or changed documents further down the pipeline.
...
High
...
FedRAMP Data Pipeline and Goal:
All FedRAMP (approximately 32) including Authority Documents, related Citations and Glossaries with term-definition pairs are available for customer consumption via API from the UC 4.0 API Gateway
...
Access the GitHub repository for all FedRAMP content.
...
High
...
Retrieve and store the XML, JSON, and YAML files for later use.
...
High
...
Identify which documents are Authority Documents.
...
High
...
Identify which files are Glossary-specific.
...
High
...
Some files may solely be Glossaries with term-definition pair entries. Ensure those documents are also processed for follow-on steps.
...
Detect file metadata changes from prior processing.
...
High
...
Only pass new or changed documents further down the pipeline.
...
High
...
eCFR Data Pipeline and Goal:
All eCFR content (TBD count) including Authority Documents, related Citations and Glossaries with term-definition pairs are available for customer consumption via API from the UC 4.0 API Gateway
...
Access the eCFR files via the eCFR APIs.
...
High
...
Store files for later use.
...
High
...
Identify which documents are Authority Documents.
...
High
...
Identify which files are Glossary-specific.
...
High
...
Some files may solely be Glossaries with term-definition pair entries. Ensure those documents are also processed for follow-on steps.
...
Detect file metadata changes from prior processing.
...
High
...
Only pass new or changed documents further down the pipeline.
...
High
...
No need to pass along files with no changes.
...
General Pipeline (after initial source-specific tasks are completed, if possible)
Reliably extract citations from Authority Documents with at least 80% accuracy where 20% of Citations need to be reworked (e.g., split, merged, rejected …)
Reliably extract glossaries from Authority Documents with at least 95% accuracy where only 5% of term-definition pairs need to be reworked. Glossaries are substantially easier to identify and extract than
...
Catalog each source Authority Document.
...
High
...
Gather all information as is required by the Common Data format.
...
Identify and extract Citations from the Authority Document
...
Critical
...
Citations are passages in the Authority Document that:
contain Mandates (requirements) OR
related contextual information such as stubs, informational, and informational gathering.
This requirement is the “brains” in the solution where the entire project’s success hinges on the success of this particular requirement.
...
Maintain Citation structure.
...
High
...
Since Authority Documents will contain multiple Citations and passages may have related Citations, that structure must be maintained to know the relationship between Citations.
...
Extract Glossary from within Authority Documents.
...
High
...
Some Authority Documents may have a glossary within the document. This will typically be near the end of the file.
Extract the Glossary details including the title, source, and all term-definition pairs.
...
Extract Glossary from glossary-specific files.
...
High
...
Some files may only have Glossary entries with term definition pairs.
Extract the Glossary details including the title, source, and all term-definition pairs.
...
Detect content changes from prior loads.
...
TBD
...
Open questions on extent of this requirement and priority.
...
Transform the Authority Document into the Common Data Format
...
High
...
Use the transformation documentation as reference as to how the source document schema structures are transformed into the Common Data Format.
...
Transform the Authority Document related Citations into the Common Data Format
...
High
...
As above, use transformation document as reference.
...
Transform the Glossaries into the Common Data Format
...
High
...
As above, use transformation document as reference.
...
Load Authority Documents into the Unified Compliance Platform
...
High
...
UCF core engineering team will determine the optimal approach for loading (API, service, …)
...
Load Citations into the Unified Compliance Platform
...
High
...
Same as above
...
Load Glossaries into the Unified Compliance Platform
...
High
...
Same as above
...
Human Validation via a simple front-end
Since this is all back-end pipeline work with no customer interaction (for this release), the user experience needs to be good enough for us as “dog food”.
...
Allow human experts to view the proposed Citations along with relationships between Citations.
...
High
...
The solution needs to inform the user as to the proposed citations by type and the relationship between them but need not be a hierarchical structure.
As example, a section in an Authority Document may have an introductory passage that is designated as an informational citation followed by three (3) citations that are identified as mandates/requirements, followed by a final passage that is also informational.
...
Allow human experts to change Citations.
...
High
...
Changes may include, but not limited to the following:
splitting suggested citations
combining suggested citations
changing the citation type (mandate, informational, stub …)
adding missing suggested citations
removing suggested citations
...
Allow human experts to approve or reject the entire Authority Document.
...
High
...
At any time during the review process, need to be able to approve the document for further steps.
OR
reject the suggested authority document and citations with reasons such as
not an Authority Document
too many citations are improperly identified
…
...
Allow human experts to change Glossaries.
...
High
...
Changes may include, but not limited to the following:
title of the Glossary
splitting suggested term-definition pairs
combining suggested term-definition pairs
adding missing term-definition pairs
removing suggested term-definition pairs
...
Allow human experts to approve or reject the entire Glossary.
...
High
...
At any time during the review process, need to be able to approve the glossary for further steps.
OR
reject the suggested glossary and term-definition pairs with reasons such as
not a Glossary
too many term-definition pairs are improperly identified.
…
...
Automation and Monitoring
...
Produce logs for each step in the end-to-end process.
...
High
...
In the content capture step, capture the following statistical information:
Complete count of documents
Count of identified Authority Documents
Count of identified Glossaries.
List of Authority Documents including name, location, change date, and metadata change value (same, new, updated, or deprecated).
List of Glossaries including name, location, change date, and metadata change value (same, new, updated, or deprecated).
Count of ADs per metadata change value (same, new, updated, or deprecated).
Count of Glossaries per metadata change value (same, new, updated, or deprecated).
...
High
...
In the citation extraction step, capture the following statistical information per AD:
Total count of citations extracted.
Count of citations per type: requirement/mandate, stub, informational, and information gathering.
...
High
...
In the human validation step, capture the following statistical information per AD:
Total count of ADs approved and rejected.
Total count of suggested Citations per AD.
Total count of Citations changed and by change type (split, merged, removed …).
...
Critical
...
This is critical information to capture since the measurement of the success of the project hinges on the correctness of suggested citations.
...
In the Glossary extraction step, capture the following statistical information per AD or Glossary:
Total count of Glossaries extracted (assume will be 1)
Total count of Glossaries approved or rejected.
Total count of term-definition pairs extracted.
Total count of term-definition pairs changed and by change type (split, merged, removed …).
...
Critical
...
This is critical information to capture since the measurement of the success of the project hinges on the correctness of suggested glossaries and term-definition pairs.
...
In the transformation step, capture the following statistical information:
Total count of ADs, Citations, Glossaries, and Term-definition pairs entering the process.
Total count of ADs completing the process by status (e.g., success, failure, …).
...
High
...
In the load step, capture the following statistical information:
Total count of ADs, Citations, Glossaries, and Term-definition pairs entering the process.
Total count of ADs completing the process by status (e.g., success, failure, …).
...
High
...
Allow for administrators to monitor, start, and stop each step in the end-to-end process including content capture, extraction, transformation, and load.
...
High
...
| title | Features |
|---|
...
Feature
...
Comment
...
Content Pipeline for STIGs
...
Content Pipeline for eCFR
...
Monitoring and Logging
...
Automatic Citation Extraction
...
Automatic Glossary and Terms Extraction
...
Report and Analytics on AI Correctness
...
| title | Out of Scope / Future Functionality |
|---|
List the known features that are out of scope for this project or might be revisited at a later time.
As is case with the assumptions, it is important to list these out so that architects and engineers can plan accordingly for these later updates.
...
Requirement
...
Comments
...
Tagging and Mapping of content to the Common Controls.
This project ends at the AD, Citation and Glossary extraction, transformation, and load.
Follow-on projects will include the tagging and mapping.
...
Human validation of the content capture.
...
Later projects can include additional human validation. To limit scope of this project, steps such as metadata change detection can be reviewed and validated after the fact looking at logs and other information.
...
Human validation of the AD cataloging.
...
Same as above
...
Human validation of the transformation into the common data format
...
Same as above
...
Human validation of loading into the UCF
...
Same as above
...
Productization of the Corpora
...
A non-production version of a corpora exists. Any work on the corpora is out of scope for this product.
...
Loading of data into a corpus, data lake, or any other target other than UC 4.0
...
The focus of this PRD is to load compliance content into the UCF 4.0 application for customer consumption over the API.
Follow-on projects can tap into the pipeline and use the content for other purposes.
...
UX/UI for any other aspect of the process outside of human review and approval of suggested citations and suggested term-definition pairs.
...
Later product updates could include additional front-end steps for tasks throughout the process.
| Expand | ||
|---|---|---|
| ||
Link to mockups, prototypes, or screenshots related to the requirements. For this PRD, the focus of the user interaction is on the reviewing of suggested citations and term-definition pairs. |
| Expand | ||
|---|---|---|
| ||
Links to user journeys, process flow, or other diagrams related to the requirements. |
| Expand | ||
|---|---|---|
| ||
If this project is a component to other areas or an update to an existing product, specifically call out where this product will interact with other areas. |
...
| title | Open Questions |
|---|
List any open questions that come to mind throughout the lifecycle of this initiative.
...
Question
...
Answer
...
Date Answered
...
What do we do with deprecated authority documents?
...
For STIGs, how do we identify which files are authority documents?
...
For NIST 800-53, how do we identify which files are authority documents?
...
For FedRAMP, how do we identify which files are authority documents?
...
For eCFRs, how do we identify which files are authority documents?
...
Specifically, what is required to catalog an AD?
...
In this first pass, what should constitute content changes?
We don’t want to get too crazy and make this a massive project.
Need to discuss.
...
| title | Alternative Solutions |
|---|
Provide a high-level evaluation criterion for alternative solutions (build, buy, partner) to evaluate different routes to success.
Milestones and Launch Checklist
| Expand | ||
|---|---|---|
| ||
The intent of this section is for the following: Monetization: Financial impact this product will introduce (if any) Risk Mitigation: Identifies potential risks and propose mitigation strategies. Launch Readiness: launch checklist including high-level go-to-market plan to ensure cross-departmental alignment. High-level Messaging: Includes Unique Selling Proposition (USP) raising visibility of the proposed solution’s value proposition. |
...
| title | High-level Messaging |
|---|
What is the Unique Selling Proposition (USP)? Relay the key factors that separate our product from the competition and why we are the best possible solution for our prospects based on their unique needs.
| Expand | ||
|---|---|---|
| ||
Will this product be part of an existing subscription or an add-on? Will this product be usage based or part of a subscription? |
...
| title | Risk Mitigation |
|---|
...
Strategic Planning and Decision Making
| Expand | ||
|---|---|---|
| ||
Describe the problem we are solving, the high-level approach, and goals so that before we get too far into the details, readers will have a good understand of where we are headed. |
| Expand | ||
|---|---|---|
| ||
What problem are we trying to solve? We currently rely on a team of expert mappers to meticulously add content into the UCF. The process works well but is slow. With the advent of automation and AI, Unified Compliance risks attacks from competitors who will use technology to accelerate content acquisition. Customers are not able keep up to date with compliance requirements in the face of the quickly evolving best practices, implementation guide, new segments, etc. We will also find it difficult to take on new market segments without automation. |
| Expand | ||
|---|---|---|
| ||
Briefly describe the approach you’re taking to solve this problem. Provide enough information for the reader to imagine possible solution directions and get a rough sense of the scope of this proposal. The approach is to start with the “left-hand” side of the end-to-end content ingestion process focusing on compliance content through ETL for an identified list of four (4) compliance content providers. For each content provider, Authority Documents will be identified, then Citations and Glossaries will be extracted from the Authority Documents, transformed into the Common Data Format specification, and loaded into the UC platform. Automation tools and AI will be used to accelerate the end-to-end process with human assistance to review and approve critical steps in the process focusing on reviewing and updating AI suggestions. Once the process is proven out, the intent is to extend the solution to many additional compliance content providers. |
| Expand | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||
What does success look like? What metrics do we measure today that we can affect? What metrics should we absolutely add? Why it is important to affect those metrics?
|
Scope and Features
| Expand | ||
|---|---|---|
| ||
The section focusses on the details of the solution including what is in scope, what is out of scope and additional information to help in the product and engineering collaboration process. |
| Expand | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||
Describe the product features that will bring value to customers and fulfill underserved need(s).
|
| Expand | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||
List the known features that are out of scope for this project or might be revisited at a later time. As is case with the assumptions, it is important to list these out so that architects and engineers can plan accordingly for these later updates.
|
| Expand | ||
|---|---|---|
| ||
If we have them, link to:
Link here: |
| Expand | ||
|---|---|---|
| ||
If this project is a component to other areas or an update to an existing product, specifically call out where this product will interact with other areas. Start writing here: |
| Expand | |||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||
List any open questions that come to mind throughout the lifecycle of this initiative.
|
Milestones and Launch Checklist
| Expand | ||
|---|---|---|
| ||
The intent of this section is mostly focused on getting the solution “out the door” and who else is affected outside the product and engineering teams. |
| Expand | ||||||
|---|---|---|---|---|---|---|
| ||||||
If applicable, identifies potential technical risks and propose mitigation strategies.
|
| Expand | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||
Identify any relevant milestones that people should now know about. Will we “eat it” ourselves first? Will this require a beta? and what is the target launch date?
|
| Expand | ||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||
This section is a reminder to the product team to make sure all relevant stakeholders are involved as necessary.
|
Additional References
| Expand | ||
|---|---|---|
| ||
List and link to any other reference sites, documents … that might be important to the reader including the business model canvas (BMC). |