Regulatory Relief Process

Utah’s AI Regulatory Sandbox, administered by the Office of Artificial Intelligence Policy (OAIP), is a state-sponsored program designed to clear regulatory hurdles and provide support for companies using artificial intelligence. The program allows companies to innovate and deploy AI-powered products without unnecessary restrictions, while still maintaining reasonable consumer protections.

The sandbox provides a structured pathway for companies developing AI products to operate within a tailored regulatory framework. Rather than navigating the full weight of existing regulations—many of which were designed before modern AI capabilities existed—participating companies can work directly with OAIP to negotiate appropriate safeguards that balance innovation with patient safety. This roadmap addresses the use cases and procedures of the sandbox specifically for AI-in-healthcare applications.

The program is open to any entity or individual that meets all of the following criteria:

• Has customers or intended customers in Utah
• Uses AI as part of their business or product offering
• Would benefit from regulatory relief to bring their product to market or to operate more effectively

Companies participating in the sandbox may receive several forms of regulatory accommodation:

• Regulatory exemptions for activities that may benefit the state in the future
• Capped penalties for regulatory violations, providing financial predictability
• Cure periods (e.g., 30 days) to address compliance issues without immediate penalties
• Safe harbors for adhering to negotiated rules and standards to offer protection against state law enforcement
• Regulatory certainty through tailored mitigation agreements that clearly define expectations

These accommodations are negotiated on a case-by-case basis and are documented in the regulatory relief agreement, which serves as the binding contract between the company and OAIP.

The application and review process consists of eight sequential steps (i.e., document submissions and meetings), followed by ongoing post-deployment monitoring and monthly debriefs. Additional meetings during the application process may be included at the request of the applicant or OAIP.

After the agreement is signed and the product is deployed, OAIP maintains ongoing oversight through two primary mechanisms:

• Monthly updates:

Data reports: The participating company provides OAIP with monthly data reports covering product performance, safety metrics, adverse events, and key performance indicators as defined in the regulatory relief agreement. These reports enable OAIP to monitor the product’s real-world performance on an ongoing basis. 

Evaluation meeting: OAIP and the participating company hold a monthly review to discuss cumulative data, assess ongoing risks and benefits, evaluate whether adjustments to the agreement are needed, and ensure the product continues to meet the standards established during the application process.

• Quarterly updates:

OAIP data reports: The participating company provides OAIP with quarterly data reports. These reports may include additional data, which the company has agreed to provide at a less-than-monthly frequency, as well as summaries, interpretations, and perspectives of trends captured and lessons learned over the three-month period.

Public data reports: OAIP expects quarter annual reports to be accompanied by a public report or public announcement to address possible concerns that Utah citizens may have. At minimum, this report or announcement should disclose: 

• current phase of the pilot, 
• dates of pilot phase progressions with associated benchmark performances that triggered the phase progression
• disagreement rate between AI decisions and human reviewers compared to a measured baseline of human-human disagreement rates

• high-level description of cases on which a human reviewer disagreed with the AI decision (e.g., involved diagnosis or medication prescription)

• Annual updates: 

Annual report: By the completion of the 11th month of the agreement, the company provides an annual report to OAIP. This report summarizes 

• the progression and overall safety of the pilot
• public-health benefits measured (e.g., overall increased access, service to underserved populations, increased convenience, increased medication adherence, reduced overtesting, effective early interventions, decreased ER visits, etc.) 
• lessons learned (e.g., identify most strongly benefitting populations or obstacles identified to large-scale adoption)
• a request and justification for an extension of the pilot (if desired)
• a request and justification for a legislative change (if desired)

In addition, a continuous communication channel remains open between the applicant and OAIP for reporting adverse events, requesting agreement modifications, or addressing any emerging concerns outside of the regular reporting cadence.

Intent: The short proposal serves as the applicant’s initial written submission. Its purpose is to introduce the company and the proposed AI-in-healthcare product, demonstrate the need for regulatory relief, articulate the potential benefits the product offers to Utah residents, and demonstrate the company’s general preparedness for responsible risk management. The short proposal allows OAIP to make an initial assessment of the product’s viability and suitability for the sandbox program before requiring the applicant to invest the time and resources required for developing a full proposal.

Format: The short proposal is typically 2 to 10 pages in length. It should be concise but informative, covering the key areas that OAIP needs to evaluate at this stage. Applicants should prioritize clarity and substance over length. 

Assessment: OAIP assesses the short proposal using a structured rubric that evaluates the applicant’s response across several key areas, including the company’s background and track record, the product description and its intended population, feasibility evidence, the specific regulatory relief being requested, and a preliminary outline of the proposed pilot. The complete rubric that OAIP uses for assessing short proposals is included as OAIP’s website. Additional items may be added to the rubric for individual applicants depending on the scope of the proposed product.

Intent: The extended proposal is the most comprehensive document in the application process. Its intent is to provide a clear and thorough description of the product, its scope, development process, deployment plan, operational design, and post-deployment monitoring, such that OAIP and its panel of external reviewers can perform a detailed risk-benefit analysis. This analysis covers clinical risks, technical risks, privacy risks, and healthcare inequity risks. The extended proposal serves as the primary basis for OAIP’s decision on whether to grant regulatory relief and under what conditions. The final version of the extended (excluding redacted sensitive information) will be posted publicly with the regulatory relief agreement on OAIP’s website.

Format: The extended proposal is typically 20 to 50 pages in length, though it may be substantially longer if the applicant includes appended materials such as formularies, clinical evidence summaries, or technical architecture diagrams. 

Assessment: The extended proposal is reviewed in two stages. 

1. OAIP conducts an internal review of the proposal. A rubric for proposals is available on OAIP’s website. OAIP plans to publish a rubric for applications for provider use in the near future. Additional items may be added to the rubric for individual applicants depending on the scope of the proposed product.
   
   Applicants may ignore rubric items that do not apply to their proposed AI product. OAIP does not condition its acceptance of applications on all relevant rubric questions being answered in the affirmative. We generally accept concrete commitments to future implementations of features, safeguards, etc. where appropriate. Explanations of why such implementations are infeasible or ineffective for the proposed AI product will also be considered.
2. A proposal that meets OAIP’s expectations is shared with external reviewers who bring clinical, technical, and public health expertise. These external reviewers provide independent assessments of the product’s risk profile, the adequacy of proposed safeguards, and the overall feasibility of the deployment plan.

Intent: The product demo or sandbox environment provides OAIP with hands-on access to the product. Its purpose is to demonstrate the end-to-end user experience, give OAIP the opportunity to test the product’s functionality in realistic scenarios, and allow OAIP to conduct red-teaming exercises to probe for potential failure modes, edge cases, or vulnerabilities.

Format: The applicant may provide either a live demo conducted by a company representative or a link to a virtual sandbox environment where OAIP reviewers can independently explore the product. OAIP prefers sandbox environments if the proposed product includes a direct-to-consumer interface and/or a generative-AI component, because it allows reviewers to test the product at their own pace, revisit specific features, and conduct more thorough testing than a time-limited live demonstration permits. Please include the documents containing the test planning, development, and a list of test cases containing details on the test and desired outcome.

Intent: The submission of relevant agreements provides OAIP with full transparency into the patient’s rights, the data privacy protections in place, and the scope of liability assumed by the company. These documents allow OAIP to verify that the company has established appropriate legal and contractual frameworks to protect consumers. Applicants may request certain shared documents to be handled confidentially and designated as protected.

Format: The applicant is expected to provide the following documents:

• Customer service agreement with patients or customers
• Privacy policy governing the collection, use, and protection of patient data
• Data sharing protections (both internal and external data handling policies)
• Insurance or liability policy covering the scope of risk assumed by the company

These documents are reviewed by OAIP as part of the overall assessment and inform the terms included in the final regulatory relief agreement.

The main body of the agreement contains the core terms and conditions governing the regulatory relief arrangement. This includes the identities of the parties, the effective date and duration of the agreement, the obligations of both the participating company and OAIP, reporting requirements (including monthly data reports and quarterly review meetings), conditions under which the agreement may be modified or terminated, and any other general provisions necessary to govern the relationship.

Schedule A provides a detailed explanation of the specific regulatory relief granted by OAIP. This may include statutory exemptions that allow the product to operate in novel regulatory contexts, capped penalties that limit the financial exposure of the company for regulatory violations during the sandbox period, cure periods that provide the company with time to address compliance issues before penalties are imposed, safe harbors that protect the company when it adheres to the negotiated rules and standards, and any other forms of tailored mitigation that OAIP determines are appropriate based on the risk-benefit analysis conducted during the review process.

Schedule A is crafted collaboratively between OAIP and the applicant during the agreement review stage (Step 7) and is designed to reflect the specific regulatory landscape relevant to the product in question.

Schedule B contains the applicant’s extended proposal with any confidential information redacted or removed. This version of the proposal serves as a public-facing record of the product, its intended scope, and the safeguards that were evaluated during the review process. By including a public-facing version of the proposal with the agreement, OAIP ensures transparency and public accountability while still protecting the applicant’s proprietary information and trade secrets.

Applicants who wish to claim confidentiality protections for specific information in the extended proposal should provide a written claim of business confidentiality along with a concise statement of supporting reasons, as described in the short proposal rubric (see OAIP’s website). OAIP may then designate such records as “protected” under the Government Records Access and Management Act (GRAMA).