COLO Compass: Evidence and practical direction for chief online learning officers.

National Distance Learning Week runs November 2 through 6 this year. USDLA’s 2026 theme includes advancing AI, global education through open educational resources, and the sustainable future of distance education. For those of us in online and digital learning, the past three years have involved a lot of AI activity. We have piloted tools, written guidance, answered faculty questions, and watched vendors add AI features to nearly every platform we license. We have spent considerably less time deciding who should approve those tools, what institutional systems they can access, and what happens when they no longer serve their intended purpose.

We have already paid for this lesson

Northwestern, Georgetown, and NYU are among at least a dozen universities that disabled Turnitin’s AI detector after its false-positive rate turned out to be higher than the company first suggested. Detection gave institutions a way to put off a harder question: does an assignment still measure what it was built to measure?

We made similar mistakes with remote proctoring. During the pandemic, many campuses rushed to proctor everything. Proctoring still has its uses, but applying it everywhere was a decision many of us came to regret. Before that came MOOCs and then the OPM boom, each described as the disruptor that would change everything. Both prompted substantial institutional experimentation but produced more complicated outcomes than early predictions suggested. AI is not near that point. It is, however, one more technology we are asked to evaluate, and the detector episode shows what happens when adoption runs ahead of evaluation.

What Hallmark 5 asks of institutions

In UPCEA’s 2026 Hallmarks of Excellence in Digital Learning Enterprises Hallmark 5, Technology and Digital Innovation, begins with a direct claim: technology is no longer simply a tool that supports digital learning but a strategic institutional capability. The Hallmark covers the systems, policies, data practices, security measures, and operational decisions behind effective digital learning at scale. It stays deliberately software-agnostic. It describes the functions a digital ecosystem has to serve without recommending products.

In 2016, digital technology was framed as an enabler that required reliability, creativity, and support. The 2026 edition treats technology as a learner-centered ecosystem and describes the leadership task as moving from managing tools to designing and governing complex digital environments. The May 2026 Canvas outage, which disrupted final exams at institutions across the country, showed what happens when that infrastructure fails. The Hallmark asks institutions to add digital learning systems to the list of “always-on” technologies, alongside power and network connectivity.

AI runs throughout the document, and the concept index lists AI governance as a primary topic of Hallmark 5 with connections to every other Hallmark. The Hallmark asks for AI strategic frameworks built on three parts: governance that covers pedagogy, learner support, and administrative uses; enablement that gives people access to AI tools and an understanding of them; and fluency for faculty, staff, and students. Institutional AI strategies should address transparency, accessibility, privacy, academic integrity, and human oversight. Institutions also need processes for evaluating AI-related procurement and clear policies on acceptable use by students, faculty, staff, and third-party vendors. Those policies are expected to change as the technology does. One of the Hallmark’s performance indicators calls for reviewing data governance, AI, and IT policies every year.

Identity verification and authentication are also governance decisions. Institutions need to consider academic and regulatory requirements alongside accessibility, privacy, and the costs imposed on learners.  Hallmark 5 does not call on institutions to adopt every new technology. It emphasizes their responsibility to evaluate, govern, and implement innovations that support institutional priorities and learners.

What it looks like on campus

In September, five of the Hallmark’s thirteen authors joined me to talk about how it plays out in practice.

Renee Pfeifer-Luckett of the University of Wisconsin-Stevens Point described how the Universities of Wisconsin built their digital learning environment. The work began with two years of listening sessions across the system’s campuses, open to students, faculty, and staff. The concerns people raised, including accessibility, tool safety, and data privacy, lined up closely with the Hallmark’s themes. The system’s governance model, called RIDE, provides a defined route for proposed changes: Recommendation, stakeholder Input, Decision, and Execution. The system also standardizes certain things across every course, like where students find the syllabus, and keeps a process for exceptions when there’s a pedagogical reason for one.

Emily Boles described a partnership among IT, her online learning unit, and purchasing at the University of Illinois Springfield to slow the spread of free tools into courses. They configured the LMS so that integrations have to go through a review of security, privacy, and FERPA compliance. Her advice to anyone starting at a new institution: talk to purchasing and IT first.

Gloria Niles of the University of Hawai’i System pointed to the challenge of making data governance visible to faculty as a support system rather than a compliance hurdle. A tool may be free to students, but that doesn’t mean it can skip review. She also noted that change management for an RFP starts long before the solicitation goes out. Gloria also noted that change management begins long before an RFP is issued. Once the solicitation is published, the institution is committed to what it asks vendors to deliver. That makes early coordination across academic, technology, purchasing, and compliance functions essential.

Shawn Miller talked about his AI work at Rice University, which requires constant partnership with the CIO, compliance, risk management, and the registrar. A campus AI advisory committee there has drafted values for responsible AI. Those values will shape how Rice evaluates the stream of small AI integrations arriving in the LMS. As AI advances, leaders will need to consider learning ecosystems in which agentic AI tools take part in learning alongside students and faculty. Shawn said that if the authors rewrote Hallmark 5 today, they would go further, especially on the growing number of faculty and students building their own learning applications.

Gayla Stoner described Creighton University’s Institute for Adaptive Intelligence. It isn’t an academic unit. It exists to support the whole university, and part of its work is creating a safe space to pilot AI tools and checking that the institution is collecting the right data, ethically. Gayla tied that work directly to Creighton’s identity as a Jesuit, Catholic institution.

The question everyone asked

One attendee asked how to keep governance from getting in the way of innovation. The Hallmark names the tension itself: institutions are often caught between what counts as enterprise technology and what counts as experimental, and they need to shorten the time it takes to test and adopt new learning tools without giving up data governance, security, or privacy. There’s no tidy answer. Institutions don’t have R&D budgets built for failing fast; when an academic experiment fails, students may bear the consequences. Shawn described the problem well. Higher education is good at starting experiments and not very good at moving them into something sustainable. Once we finally do, the process took so long that nobody wants to change it again. Renee’s answer returned to stakeholder engagement: a known place where ideas can surface and get vetted for accessibility and privacy before they spread.

Where to start this week

If your institution doesn’t yet treat technology as an ecosystem, NDLW is a good week to take stock. Two UPCEA resources can help.

The UPCEA AI Maturity Index helps leaders assess how far their online and professional continuing education unit has come in adopting AI responsibly, strategically, and effectively. It’s a reflective planning tool rather than a ranking, meant to help teams see their strengths, find gaps, and set priorities for investment in policy, people, systems, and strategy. I’d suggest taking it with your IT and academic partners instead of alone. The places where your answers differ will tell you as much as the scores.

The UPCEA AI Hub collects the Capstone Framework the Index is built on. The framework identifies governance, ethics, data readiness, digital literacy, and infrastructure as foundations for sustainable AI adoption in recruitment, student success, instruction, and other institutional functions. Those priorities closely align with Hallmark 5. The Hub also includes a 90-day framework for units starting small, AI study groups and labs, and member case studies. One of these connects directly to the assessment problem above: the Universities of Wisconsin built a collection of AI assignment examples to help faculty move from anxiety about AI and academic integrity toward practical experimentation.

On January 20, UPCEA and EDUCAUSE will co-host Shared Digital Leadership: Where CIO and COLO Strategy Converge, bringing together CIOs and digital learning leaders to discuss shared governance, technology investments, and institutional priorities. Recordings of the fall Hallmarks webinars, including the Hallmark 5 session, will be posted in UPCEA’s member forum, CORe.

AI tools will continue to change, and institutions will continue experimenting with them. The harder work is establishing a process for evaluating those tools that can keep pace without setting aside our obligations to faculty and learners. That’s where I would focus some attention during National Distance Learning Week. 

References

Creighton University. (2026, May 12). Creighton University to launch Institute for Adaptive Intelligence. https://www.creighton.edu/news/creighton-university-launch-institute-adaptive-intelligence

Niles, G., Ingram, M., Irizarry, M., McDonald, R. H., LaBrake, M., Holt, P., Hastie, M., Stoner, G., Boles, E., Miller, S., Pfeifer-Luckett, R., Moore, S., & Herrick-Phelps, J. (2026). Technology and digital innovation. In J. Uranis & V. Cook (Eds.), Hallmarks of Excellence in Digital Learning Enterprises (pp. 63–77). UPCEA. https://upcea.edu/hallmarks-digital

Palmer, K. (2026, August 5). AI detectors are out, new assessments are in. https://www.insidehighered.com/news/tech-innovation/artificial-intelligence/2026/08/05/ai-detectors-are-out-new-approaches-are

UPCEA. (2026). UPCEA AI Hub. https://upcea.edu/resources/ai-hub/

Uranis, J., & Cook, V. (Eds.). (2026). Hallmarks of Excellence in Digital Learning Enterprises. UPCEA. https://upcea.edu/hallmarks-digital

United States Distance Learning Association. (2026). National Distance Learning Week. usdla.org/events/ndlw/

AI Disclosure

This post was developed with the support of an AI research and writing assistant. AI assistance was used to identify and review relevant sources, verify facts and citations, explore related evidence and examples, develop and revise draft language, and identify claims that warranted additional scrutiny or qualification. When possible, research was verified against primary or authoritative sources.

The topic, editorial direction, argument, source priorities, and decisions about what evidence to include and how to interpret it were guided by the human author. AI-generated research and draft language were reviewed and revised iteratively, including efforts to distinguish documented findings from inference, identify limitations in the available evidence, and avoid claims that extended beyond what the sources could support.

AI-assisted content is treated as draft material rather than final work. Before publication, the author reviewed the analysis, verified material facts and source characterizations, and determined that the conclusions reflect appropriate professional and institutional judgment. The author is responsible for the accuracy and final content of the piece.