Are CME activities that change knowledge alone compliant?
A wonderful peer recently posed this question to the blog:
I think there's still confusion out here about the distinction between how we design an activity and what we're supposed to be trying to measure (coming out of that individual activity). It's difficult to measure or demonstrate change in competence as the result of an activity where the learners are from a broad community instead of being on your own staff where you can integrate systems changes and follow outcomes over time for example.
If I understand correctly, it's fine to design activities primarily to close a knowlege gap, but these activities should be contributing to a program level effort to change competence, performance, or patient outcomes. The closest I come to trying to measure competence from a one-time knowledge based activity is in asking learners to respond to case presentations before and after the information is delivered (trying to show intention to apply the new knowledge), or asking if they intend to change anything about their practice in light of the new information (which doesn't usualy get me much information).
So my question is, for an individual activity, is it sufficient to measure change in knowlege, i.e., a pre-test/post-test?
WOW. This is a loaded question. Looking at Criterion 3 "The provider generates activities/educational interventions that are designed to change competence, performance, or patient outcomes as described in its mission statement" and the further clarification on the subject, activities can be designed to change knowledge as long as the overall CME program is designed to change competence, performance or patient outcomes.
Here's what the ACCME had to say about it in a video with Dr. Kopelow.
Personally, I think there are two issues at play here. One being that you have to plan CME activities to change competence, performance or patient outcomes, but you don't necessarily have to be successful. There is no rule that you have to bat 1.000 with each and every CME activity.
Also, as a patient, if I go to my doctor and describe XYZ symptoms, I don't want him/her to say "I know what that is." I want him/her to say "I know how to treat it" (competence) or "I can treat it" (performance). Or if they need more information on how to treat it, at least it goes beyond knowing what it is (which is why I'm a big fan of internet point-of-care even though it hasn't caught on yet).
I think we need to get creative when we develop pre/post tests. Case studies are certainly effective in analyzing changes in competence, but we can also ask more questions that are applicable to practice. Instead of asking what something is, we can ask how it would be treated, even having more than one right answer. In addition, having a definitive answer, as opposed to "all of the above" or "none of the above" can help us identify additional practice gaps. Self-reported changes are perfectly acceptable, and can also help us gather some great new information!
Physicians hopefully will learn something new from a CME activity. What they do with that information is what we should be trying to analyze...self-reported or otherwise.
