This document outlines the requirements for a tool designed to help Compliance Analysts create new compliance terms within an organization.
Background:
Our current Dictionary solution allows to add a term to an organization’s dictionary but does not help in the creation of these terms.
Purpose
The AI-based Term Creation tool is designed to revolutionize how compliance terms are generated. Current manual methods of creating compliance terms are time-consuming, inconsistent in quality, and challenging due to the sheer volume of sources that can be analyzed. This tool addresses these challenges by providing a fast, accurate, and reference-rich solution for generating compliance terms.
Scope
The tool will be robust and knowledgeable enough to cover a wide range of industries, with the ability to process any digital format sources. Its primary function in the initial phase is to allow users to input a term and receive a comprehensive definition and relevant references.
Future versions can include integration as a plugin for document editors like MSFT Word and Google Docs, further enhancing its functionality to analyze documents and automatically identify and define terms not yet in the organization's compliance dictionary.
Target users
The generator is specifically designed for larger enterprises with established compliance departments. These organizations typically face greater complexity in their compliance requirements and will benefit most from the tool.
While the primary users of this tool are Compliance Officers, the tool is designed to produce definitions that are easily understandable by non-compliance professionals as well. This accessibility broadens its utility across different organizational departments, making it a valuable tool for anyone needing clarity on compliance terms.
The need was validated internally with all mappers, and with AT&T.
Initial Concept and User Interaction
In its initial version, the tool will offer a user-friendly interface where users can input a compliance term and promptly receive a well-defined, accurate definition and references. The focus is on simplicity and efficiency to ensure ease of use. Looking ahead, the integration into platforms like Google Docs will enable users to retrieve definitions directly within their working documents, enhancing workflow efficiency.
Technology
The development of our solution is driven by the challenges posed by manual term creation methods and the advent of recent advancements in AI technologies, particularly in Large Language Models (LLMs) with RAG (Retrieval-Augmented Generation). These technologies offer unprecedented capabilities in processing large volumes of text and generating accurate, contextually relevant information.
Proof-of-concept
The technology and concept was tested in a proof-of-concept.
Feedback from the mapping team on the proof-of-concept was unanimously positive, and results are surprisingly accurate, despite limited sources and training.
Milestones
Next steps would entail:
Q1 2024 Finishing the proof-of-concept:
Refinement of Retrieval Mechanism: Enhance the AI's ability to accurately source and retrieve relevant data.
Enrichment of Sources: Expand and diversify the data sources to improve the tool's comprehensiveness.
Fine-Tuning for Quality: Optimize the AI algorithms to ensure high-quality, accurate compliance term generation.
End Q1 2024 Internal Roll-out:
POC to Production in AWS: Transition the Proof of Concept (POC) into a full-scale production environment on AWS.
Integration with Mapper Team: Provide the tool to the mapper team for human validation, ensuring accuracy and reliability.
Q2 2024 External Roll-out:
Following successful internal use and quality benchmarks, release the tool externally as a feature of the Dictionary solution.
Competition
genAI, is a very fast-moving and intensely competitive field. Today, competition for our solution would come from more general genAI tools like Bard, ChatGPT, and others.
We see niche solutions sprouting up for academic research, legal, etc. It is undoubtedly only a matter of time before someone brings out an LLM that is trained on compliance terms.
Unique Selling Proposition (USP)
Versus the current way of doing things, the manual way, the tool's USP lies in its:
speed,
accuracy, and
the comprehensive nature of the references it provides.
Leveraging advanced genAI and NLP techniques, it promises a significant improvement over traditional methods, offering quick and reliable compliance term definitions. This tool is a time-saver and a step towards more consistent and universally understandable compliance practices.
Versus more general-oriented tools, our solution’s USP lie in:
Niche Expertise Development: Continuously enhancing our AI models with the latest compliance and regulatory knowledge will keep our tool at the forefront of this niche.
Strategic Partnerships: Collaborating with regulatory bodies and compliance experts can improve our tool's capabilities and credibility.
Focused Marketing and Branding: Emphasizing our specialization in compliance term creation in marketing efforts will help distinguish our tool from broad genAI solutions.
Financials
The AI-based Term Creation tool is projected to be cost-neutral, delivering significant internal cost savings and serving as a valuable solution for customers.
Cost Savings:
UCF Mapping Team Efficiency: The team has created 1,351 new terms year-to-date, spending 20-30 minutes per term, totaling approximately 563 hours.
Hourly Cost Savings: With an internal cost of $60/hour, the tool offers an annual cost saving of $33,775.
Revenue Projections:
Pricing Strategy: The tool will be priced at $5/user/month, with additional charges for token credits in case of over-use.
Market Penetration Assumptions:
Existing Customer Base Penetration: Estimated at 50%, with an average of 2 users per account.
Annual Recurring Revenue (ARR): Potential ARR is projected to be $240,000
Cost Projections:
Development and Operational Costs:
NLU Developer: $15,000 for enhancing Natural Language Understanding capabilities.
AWS Migration and UI Refinement: $7,500 for development costs associated with AWS migration and user interface improvements.
Quality Assurance: $2,500 for QA processes.
This financial overview underscores the tool's potential for cost-effectiveness and revenue generation, aligning with our strategic goals of efficiency and market competitiveness.
Conclusion
This PRD outlines the development of an AI-based Term Creation tool tailored for Compliance Analysts, addressing the inefficiencies of manual compliance term creation. Our tool differentiates itself by focusing on compliance-specific term generation, leveraging advanced genAI and NLP technologies. Targeted at larger enterprises, it promises enhanced accuracy, efficiency, and industry-wide applicability.
Moving forward, our focus can be on continuous technological refinement and market responsiveness to maintain a competitive edge against broader genAI solutions, ensuring our tool remains a specialized, valuable asset in compliance.
Strategically, this smallish genAI solution is a first step, a beachhead in our potential approach to meeting the growing demand for genAI solutions in the compliance industry.
Appendix:
Proof-of-concept
A screenshot of results of the live POC made in October 23.
The POC provided the following:
a short definition
a slightly more complete definition
three questions which the user can ask that help explain the term
references upon which the answer is based
The POC could digest as source information for its compliance knowledge both PDF documents, and websites.
Potential metrics
Accuracy Rate:
Definition Correctness: Percentage of terms where the generated definition accurately reflects the intended meaning.
Reference Relevance: Proportion of contextually relevant references to the generated terms.
Error Rate:
Misinterpretation Frequency: Track the frequency of incorrect interpretations or irrelevant definitions generated.
Inconsistency Detection: Measure instances where the tool provides varying quality across similar requests.
Response Time:
Generation Speed: Monitor the average time to generate a term and its definition, ensuring it meets efficiency standards.
Usage Metrics:
Adoption Rate: Track the number of active users and frequency of use, indicating the tool's perceived value.
Repeat Usage: Measure how often users return to the tool, indicating reliance and satisfaction.
Benchmarking:
Comparison with Manual Processes: Compare the quality of terms generated by the tool against those created manually.
Competitor Comparison: Regularly compare the tool's output quality against similar offerings in the market.