What does success look like? What metrics do we measure today that we can affect? What metrics should we absolutely add? Why it is important to affect those metrics? Goal | Metric | Why Important? |
|---|
Reliably extract citations from Authority Documents | >= When AI is not needed, 100% accuracy. When AI is utilized, greater than 80% accuracy where 20% of Citations need to be reworked (e.g., split, merged, rejected …) | If there is poor accuracy requiring extensive human correction, then there is little value. | Reliably extract glossaries from Authority Documents>= | When AI is not needed, 100% accuracy. When AI is utilized, greater than 95% accuracy where only 5% of term-definition pairs need to be reworked. Glossaries are substantially easier to identify and extract than citations. | If there is poor accuracy requiring extensive human correction, then there is little value. | Reliably automate the end-to-end process of capturing, transforming, and loading STIG, NIST 800-53, FedRAMP, eCFR compliance content into the Unified Compliance platform. | 100% of identified Authority Documents for all four compliance content contributor sources is loaded into the UCF. | All four Authority Documents sources are related to securing and hardening IT infrastructure for both the private and public sector. To provide value to customers with Security Operation's requirements, UC needs to maximize the breadth of security coverage to ensure we can provide security guidance for as many IT assets as possible. |
|