Continuous Learning for Better Standards of Care

An American Board of Anesthesiology initiative links technology and continual learning and assessment to support patient care and safety.

The Challenge

Like other medical certifying bodies, the American Board of Anesthesiology offers a board certification, a distinction most physicians pursue upon graduating from residency. ABA’s rigorous certification process requires anesthesiologists to successfully complete residency, earn licensure from their state, and pass a battery of written and oral exams to earn initial certification. Thereafter, anesthesiologists are enrolled in the Maintenance of Certification in Anesthesiology® (MOCA®) program, and historically, they took a cognitive exam every 10 years to maintain certification.

ABA recognized that requiring physicians to take a high-stakes exam in an environment with increasingly complex regulatory requirements and an evolving and expanding knowledge base underlying their medical practice was not productive. It encouraged cramming rather than sustained learning. Yet the purpose of the MOCA program was to enable physicians to demonstrate regular acquisition of current medical knowledge and proficiency in providing high-quality patient care.

The Approach

Could ABA enhance assessment of physicians’ knowledge while encouraging lifelong learning? The board turned to advances in technology and learning science for their answer. ABA developed a pilot program that revolved around an observational study and its development of a web-based MOCA Minute platform integrating longitudinal assessment and a customized user learning experience. Experts crafted initial questions related to 16 core testing topics. During the 2014 and 2015 certification exam years, ABA tested its longitudinal-assessment hypothesis that individuals who participated in the MOCA Minute program would perform better on the MOCA certification exam than those who did not participate.

In preparing for the 2014 exam, 1,408 of 2,800 physicians enrolled in the MOCA Minute pilot, and for 2015, 330 of 1,046 eligible physicians used the tool. Those who actively participated in the MOCA Minute in pilot years 2014 and 2015 scored 9.9 and 9.3 points higher, respectively, than those who did not. In 2016, after initial results showed promise and repeatability, ABA expanded the tool’s use to all participants. At each phase, ABA expanded testing topics, beginning with 16 and extending to 22.

Outcomes

MOCA Minute—now enhanced by a mobile application so that diplomates can easily access and answer questions—helps busy physicians integrate learning and retention into their everyday practices. Whether they answer questions correctly or incorrectly, users receive immediate feedback including the correct answer, a rationale, and resource links. The tool embodies these principles:

Continual learning and assessment – Users refine and demonstrate professional knowledge over time rather than episodically.
Customization – The mix of questions fed to anesthesiologists is based on the proportion of time they spend in each practice area.
Flexibility – The tool allows ABA to push out questions and resource links in real time that relate to current challenges.

Importantly, ABA’s research showed that actively using this new tool improved diplomates’ assessed performance. Moreover, in 2017, 77 percent of diplomates surveyed reported that MOCA Minute was serving them somewhat or very well as an assessment tool. In 2018, ABA transitioned the MOCA Minute from a pilot to a permanent component, replacing the traditional cognitive exam, for all participants in continuing certification.

Impact

ABA's groundbreaking work in the use of longitudinal assessment to support the learning and retention of physician knowledge has fundamentally transformed how board-certified physicians across specialties keep their medical knowledge current and demonstrate that they are actively doing so. This change in assessment from episodic to continuous improvement better supports anesthesiologists, in turn, improving the quality of care and safety they provide to their patients. The longitudinal assessment has also proven to be an invaluable tool to quickly educate and assess physician knowledge about emerging medical issues, including the Zika Virus and COVID-19 pandemic. When new diseases emerge, ABA can quickly force questions into diplomates’ queues in real time, providing them with valuable resources to help them learn what they need to know to provide the best possible patient care.

ABA’s move to longitudinal assessment inspired the entire medical specialty certifying community to take similar steps toward using technology and evidence-based adult learning techniques to advance their assessment systems. All 24 medical specialty certifying boards today use, or are in the process of implementing, longitudinal assessments in their continuing certification programs.

“The value of MOCA Minute is that it promotes continuous assessment and guides anesthesiologists’ lifelong learning by providing immediate feedback that helps identify areas where they may want to pursue more intensive study. Additionally, we can gauge the performance of the entire population of board-certified anesthesiologists to identify how much knowledge is acquired and maintained over time.” —Deborah J. Culley, M.D., president, American Board of Anesthesiology; attending physician and vice chair of research at the Brigham and Women’s Hospital in Boston.

“We are all very committed to getting this right because it can enable our physicians to take better care of our patients.” —James P. Rathmell, M.D., former secretary, American Board of Anesthesiology Board of Directors; chair, Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women's Hospital, Boston

“Knowing how to manage our own learning has never been more important. The opportunities to do so have never been more available. —Robert Bjork, Ph.D., distinguished research professor, Department of Psychology, University of California, Los Angeles

References

How to Become an Anesthesiologist
http://doctorly.org/how-to-become-an-anesthesiologist/

The American Board of Anesthesiology “Changing the Paradigm: A New Approach to Assessment in Maintenance of Certification,” 2015 MOC Summit

Anesthesiology, V 125, No 5, November 2016, “Association between Participation in an Intensive Longitudinal Assessment Program and Performance on a Cognitive Examination in the Maintenance of Certification in Anesthesiology Program” Huaping Sun, Ph.D., Yan Zhou, Ph.D., Deborah J. Culley, M.D., Cynthia A. Lien, M.D., Ann E. Harman, Ph. D., David O. Warner, M.D.