Baglia J et al. Generating developmentally appropriate competency assessment at a family medicine residency. Fam Med. 2011 Feb;43(2):90-8.
BACKGROUND AND OBJECTIVES:
Ten years after the Accreditation Council for Graduate Medical Education’s (ACGME) mandate that residency programs evaluate learners’ competency, research is needed to guide efforts to meet this challenge. During an innovative residency redesign, the authors developed a process to effectively measure “competence.” This particular family medicine residency admits six residents per class year and is sponsored by an academic community hospital. Our objective was to generate developmentally appropriate observable behaviors that assess competencies.
Eight steps guided the development of this assessment system:
(1) Generate residency-specific competencies,
1. Relationship-centered care,
2. comprehensive care,
3. information literacy and knowledge creation,
4. leadership and change management,
5. community health partnership,
6. lifelong learning, and
7. self care.
(2) Define residency-specific competencies,
(3) Identify principles of assessment,
1. direct observation is ideal and includes both assessment and feedback,
2. multiple methods are appropriate,
3. assessment is consistent to the extent that visible behaviors are identified, and variations of interpretation are minimized,
4. assessment and feedback are timely and expected, and
5. effective assessment practices are dependent upon ongoing faculty development.
(4) Compose and analyze narratives of excellence within each competency,
(5) Distill standard statements from narratives and organize into Dreyfus levels of competence,
(6) Derive observable behaviors from standard statements to directly correlate behaviors and competency levels,
(7) Design assessment tools (based on observable behaviors) for six residency learning sites, and
(8) Translate assessment tools for ACGME competencies.
The results of this process include an assessment system that
(1) features six tools used with strategic frequency throughout the academic year and
(2) generates global assessment of residents’ performance in both ACGME and residency-specific competencies.
Narrative reflection was an effective method to tie observable behaviors to competencies. The process was time intensive; however, greater efficiency and enthusiasm is expected in the use of these assessment tools, with greater confidence in the program’s capacity to assess training outcomes. Future research should include comparison of these tools with those of other programs.