What is the Pathway to Health AI Governance?

By: Jolie Ritzo, Vice President, Strategy and Network Engagement, Civitas Networks for Health

As artificial intelligence (AI) rapidly reshapes health and health care, it holds the promise of transformative advances in clinical and social care, public health, and health care administration. Yet, this potential comes with profound challenges, particularly for ensuring equitable, safe, and effective use. Governance lies at the heart of responsibly integrating AI into health and health care settings and applications, and Health Data Utilities (HDUs) are emerging as critical entities that can provide ready insight into the necessary inclusive governance structures, technical requirements, and careful considerations that must be in place to protect patients. 

Drawing on themes from the recent NEJM AI article, “The Role of Health Data Utilities in Supporting Health AI,” co-authored by Civitas Networks for Health leaders and other thought partners at Ellison Institute for Technology and Harvard Medical School, as well as insights from a recent expert panel discussion on health AI governance hosted by Civitas member, Big Sky Care Connect this article explores how HDUs can assist with local and state oversight. 

The Evolving Role of Health Data Utilities 

HDUs are a natural evolution from traditional Health Information Exchanges (HIEs) responding to expanding use cases for health data needs and to the complexity of health and health care in the U.S. They hinge on serving a defined geography, inclusive governance, neutral convening, state designation, advanced technical infrastructure, strict attention to privacy and security, value-added services, and outlined use cases. While HIEs have been instrumental in primary health data use—such as using clinical data to advance care coordination—HDUs are designed for both primary use cases and broader secondary applications, including public health, research, cross-sector data sharing, and, in this case, health AI governance. 

HDUs are uniquely positioned to act as stewards of validated, high-quality, multisource health data. By aggregating data from diverse stakeholders and adhering to inclusive governance structures, HDUs can provide the robust foundation required for health AI governance. Their state-designated authority allows them to mediate between local, state, and national stakeholders, ensuring that data used for health AI reflects the needs and contexts of the intended use and populations it serves. 

4 Core Functions of HDUs in Health AI Governance 

1. Promoting Equity through Representative Data: One of AI’s most glaring vulnerabilities is its potential to amplify health disparities. Algorithmic bias disproportionately impacts marginalized populations, who are often underrepresented in data sets.  HDUs can address this by: 

• • Aggregating and assessing more complete data that reflects greater diversity of a state’s population – demographic, race, SOGI, HRSN, SUD, and behavioral health as examples of expanding the data set 
  • Ensuring representative datasets that account for geographic, racial, and socioeconomic disparities and that the data is quality – applying patient matching, deduplication, etc. 
  • Serving as a neutral steward of data, prioritizing equity in its curation and application rather than profit. 

2. Enhancing Data Sovereignty and Privacy: AI’s reliance on vast amounts of personal health data makes privacy and sovereignty paramount. For Native American tribes and other often underrepresented vulnerable groups, data sovereignty ensures that decisions about data use rest with the patient and the community where they live.  HDUs can champion these principles by: 

• • Establishing governance models that include patient and community representation. 
  • Implementing data privacy protocols that respect cultural values and individual consent. 
  • Avoiding centralized, profit-driven data repositories rather acting in favor of nonprofit, state-led stewardship. 

3. Supporting Localized Context for Health AI Models: Health AI models often struggle to account for local contexts, undermining their utility and accuracy in diverse settings.  HDUs are well-suited to bridge this gap by: 

• • Determining baselevel health AI use cases – which health systems are using health AI and how is that impacting data collection. The HDU can then flag the data as AI generated.  
  • Providing localized data that reflects unique community health patterns and social determinants of health (SDOH). 
  • Facilitating state-level scenario planning with key partners and potential collaboration with federal guidance from The Assistant Secretary for Technology Policy (ASTP). 

4. Enabling Health AI Assurance and Responsible Governance: Health AI assurance requires robust governance frameworks to ensure models are safe, ethical, and effective.  HDUs can take the lead by: 

• • Establishing ethical review boards to oversee AI use in health and health care. 
  • Supporting rigorous model evaluation and transparent reporting on AI performance. 
  • Developing and disseminating practical guidelines for organizations to follow, ensuring compliance with evolving regulatory landscapes like the Colorado Privacy Act and recent legal precedents in Texas. 

Key Themes from a Recent Health AI Governance Panel with Big Sky Care Connect 

As it becomes evident that HIEs and emerging HDUs have a role in supporting states with oversight of health AI, more of these organizations are hosting critical conversations with their governing Boards. Big Sky Care Connect serving Montana hosted a panel of experts during its Annual Board Retreat to illuminate key aspects of Health AI governance and its role in advancing care and data collaboration. The panel discussion brought together thought leaders on the topic to share insights on how inclusive governance and health data utilities can shape the future of AI-driven health solutions. 

Moderated by Jolie Ritzo, MPH, the panel featured Lisa Bari, Civitas Networks for Health; Rajib Ghosh, Health Roads; Ahmad Alkasir, Ellison Institute of Technology; and Shamekka Marty, Carequality. Their dialogue provided actionable perspectives on navigating the complexities of AI data and applications in health and health care and highlighted strategies to ensure AI serves as a tool for equity, safety, and effectiveness. 

Here are the key takeaways and learnings from this important conversation: 

• Equity and Representation: Algorithmic bias and unrepresentative datasets often harm vulnerable populations, Medicaid beneficiaries, black and brown communities, and Native Americans. Health AI governance must prioritize equity and inclusion.  
• Patient Involvement: AI governance models need to include patients in decision-making processes to ensure their voices are reflected in policies and use cases that impact them. Build with rather than for needs to be a core principle.  
• Data Sovereignty: Protecting data sovereignty is critical, especially for communities with unique privacy needs, such as Native American tribes. 
• Practical Governance: Establishing nonprofit, state-led stewards and ethical oversight committees is key to ensuring health AI is implemented responsibly. 
• State-Level Preparedness: States must plan to act independently or in coalitions, leveraging scenario planning and public-private partnerships to address the current variability in governance and regulations.  

What is the Path Forward for Health AI Governance? 

Health Data Utilities are critical infrastructure within the states and regions they serve. There is a great opportunity to support health AI governance by ensuring representative data, ethical oversight, and localized context, HDUs can help safeguard that AI fulfills its promise of transforming health and health care without creating greater harm and leaving vulnerable populations behind.  

The shift from theoretical possibilities to actionable solutions for health AI governance will require collaboration across multiple sectors. HDUs are uniquely equipped to serve as the backbone of these efforts, leveraging their technical, organizational, and trust infrastructure to make sure that health AI systems improve health outcomes while preserving privacy and equity. 

With pending changes in the administration and federal agency political appointee departures, the public and private sector needs to act. Given that HDUs are built from public-private relationships, they can seamlessly assist with developing necessary guardrails for health AI data policies and oversight of applications.  

Curious to learn more about this evolving work? Please reach out to the Civitas Networks for Health team to get involved.