#criterion 3

A wonderful peer recently posed this question to the blog:

I think there's still confusion out here about the distinction between how we design an activity and what we're supposed to be trying to measure (coming out of that individual activity).  It's difficult to measure or demonstrate change in competence as the result of an activity where the learners are from a broad community instead of being on your own staff where you can integrate systems changes and follow outcomes over time for example. 

  If I understand correctly, it's fine to design activities primarily to close a knowlege gap, but these activities should be contributing to a program level effort to change competence, performance, or patient outcomes.  The closest I come to trying to measure competence from a one-time knowledge based activity is in asking learners to respond to case presentations before and after the information is delivered (trying to show intention to apply the new knowledge), or asking if they intend to change anything about their practice in light of the new information (which doesn't usualy get me much information). 

  So my question is, for an individual activity, is it sufficient to measure change in knowlege, i.e., a pre-test/post-test?

WOW.  This is a loaded question.  Looking at Criterion 3 "The provider generates activities/educational interventions that are designed to change competence, performance, or patient outcomes as described in its mission statement" and the further clarification on the subject, activities can be designed to change knowledge as long as the overall CME program is designed to change competence, performance or patient outcomes.  

Here's what the ACCME had to say about it in a video with Dr. Kopelow.

Personally, I think there are two issues at play here.  One being that you have to plan CME activities to change competence, performance or patient outcomes, but you don't necessarily have to be successful.  There is no rule that you have to bat 1.000 with each and every CME activity.

Also, as a patient, if I go to my doctor and describe XYZ symptoms, I don't want him/her to say "I know what that is."  I want him/her to say "I know how to treat it" (competence) or "I can treat it" (performance).  Or if they need more information on how to treat it, at least it goes beyond knowing what it is (which is why I'm a big fan of internet point-of-care even though it hasn't caught on yet).

I think we need to get creative when we develop pre/post tests.  Case studies are certainly effective in analyzing changes in competence, but we can also ask more questions that are applicable to practice.  Instead of asking what something is, we can ask how it would be treated, even having more than one right answer.  In addition, having a definitive answer, as opposed to "all of the above" or "none of the above" can help us identify additional practice gaps.  Self-reported changes are perfectly acceptable, and can also help us gather some great new information!

Admittedly, this is one of the easiest ways to demonstrate compliance with Criterion 3, but there is no requirement to have learning objectives.  

One thing to consider- you still must show how your activities are designed to change competence, performance or patient outcomes, and learning objectives are just part of that process.  When creating learning objectives, think about how the learner will actually change their practice (is it attainable?  is it reasonable?  is it measurable? is it even plausible- such as training physicians to treat the zombie apocalypse?).  How will the activity close the professional practice gap?

After completing our research and identifying that the main problem the 'i' has is competition from online articles, we began to look at the best way to do this and answer the brief given to us. The brief was difficult to address it was quite limiting. We were asked to come up with a paper based campaign which could be developed digitally online eventually and put a small budget to big effect.

Below is a simple poster aims to do just that. It's straight to the point, clear and eye-catching. It encourages the use of a newspaper over online articles by subtly suggesting, perhaps comical, uses for a newspaper that cannot be achieved with online articles. As suggested by our investigation it doesn't matter why or how people come to buy/see the 'i' as long as it is in front of them so they can trial it and hopefully like it enough to buy it in future! If someone buys the 'i' to protect a surface while painting, as it only costs 20p, they are likely to flick through it beforehand and potentially become a frequent customer. The repetition of the 'i' logo reinforces the product in viewers' minds and hopefully when next buying a newspaper they will think of 'i'. This poster is also easily adaptable and low cost to produce. As well as displaying around campus, the poster would fit nicely in a magazine or scaled down as a flyer.

When waiting on transport, which students do on a daily basis, people often like something to occupy their mind. This poster shown below, encourages observer interaction. The idea behind this design is that 'i' is the one newspaper that suits all needs. Again, this poster promotes other uses for a newspaper that cannot be achieved through online alternatives and is cost effective.

Also, this design has the potential to be developed digitally to appear online (as required by the brief), for example, as an interactive advertisement  on social networking site that viewers could flick through, with the end result always being 'i'.

Another avenue the flow chart design could develop into is in campus shops. We noticed upon visiting our own campus shop that the ‘I’ is hard to pick out from current displays as it blends in with other papers, especially as the ‘I’ logo is on the left hand side of the front cover and other papers are stacked to overlap it. Our solution to this problem involves guiding customers to the ‘I’ newspaper stand. We propose this is done by providing shops with large stickers to put on their floor. The stickers would act in the same way the flow chart posters but would be adapted to focus on attracting people who are planning to buy a newspaper and encourage them to choose ‘I’. Below is a rough idea of how this would work.

Below is a rough sketch of the typical simple design of the point of sale unit we propose is used.

After discussions with our tutor, we decided our next steps for this project should include constructing a list of both rational & irrational reasons why a typical student would buy a newspaper and put together a persona of such a person.

RATIONAL

Interest in current affairs

Relevant to course work

To read on the commute to uni

To contribute to a conversation

To read a featured article of particular interest to them

To collect printed voucher for relevant offers

To complete included puzzles

Enter included competitions

To find out what's on TV/local events

IRRATIONAL

To appear intelligent

To impress peers

To make a hat

Enable them to sit on a seat when it's raining without getting wet

Mop up a spilled drink

To protect surfaces when painting

To use in a pets litter tray

To make a boat

Keep warm on the street

To make paper chains

Wrap fish & chips

To construct a ransom note

To use instead of wrapping paper

To block light out of windows

To spy on people by cutting out peep holes

To make a paper aeroplane

To draw on

A substitute for toilet roll

To use as an umbrella

After Compiling the list of rational reasons to buy the 'i' against irrational reasons we noticed that there were a n umber of uses for the newspaper that cannot be achieved be reading articles online, such as making paper aeroplanes or wrapping presents. This ties in with the theory that advertisements should push the benefits rather than features when trying to sell a product, in this case, the benefits of being a newspaper would be the other uses for the paper.

To conclude our research we created a persona of what we think a typical student 'i' consumer would be like.

PERSONA

Paul.

24.

Student at Dundee University - Final year studying politics.

Likes football & socialising.

Has a part-time job.

In a long-term relationship but lives with two other male flatmates.

Likes films and occasionally plays computer games.

Good taste - careful with money and saves up for more expensive items.

Takes care of himself and eats well.

Inquisitive nature.

THE PROBLEM:

Day 3

As mentioned in my previous blog, in order to gain a greater understanding of student communities and their views on newspapers we constructed an online survey with the aid of Google Docs. After publishing the survey last week, we have a total of 33 responses.

One of the more interesting but perhaps least surprising results was that 36% of students asked preferred to read news articles online in comparison to just 6% purchasing newspapers (as shown in the picture below).

Another important point that came to light was that although very few students would choose to buy a newspaper, a staggering 67% (2/3 of those who responded) said they would read a newspaper if it was given to them or was lying around. This shows that perhaps it is more important to initially get newspapers in front of students as apposed to encouraging them to buy a paper themselves, relating back to - and supporting - the brief that states "once people trial the product they love it." The pie chart below illustrates the drawn conclusion.

Currently our group is considering the most effective use of a campaign to make the 'i' newspaper popular among students. We have looked at the following three options:

1.To target students directly and encourage them to purchase the paper.

2. To target the student unions, shops, etc. and come up with ways to get them on board and push the product. (Where it's sold, how it's sold, promotions etc.)

3. To find a way of getting students to sample the paper initially then lead up to selling the paper to students.

As mentioned above our research suggests that a combination of the second and third approaches would be most beneficial for the product. We have discussed how it might be easier to convince students who are regular newspaper readers that the 'i' would be beneficial to them but the real challenge lies in convincing students who do not currently read newspapers to start.

Personally, I feel yesterday's task went well, despite having just one day to complete it. Many people work well under pressure and as a result, vast amounts of ideas were generated quickly. I think our team made good use of the time and divided up tasks between groups efficiently. Everyone was very involved for most of the day with only one or two people losing interest towards the end. 

I'm happy with our final outcome and I think it is a successful response to the brief, for the most part. The campaign has the potential to be developed further and so perhaps instead of another one day brief, it would be more beneficial to revisit this task and get the most out of it we can.

Working in such a large group meant it was difficult to know where to start and get the ball rolling but once we did, the volume of such different ideas that could be produced was great.

I was very surprised at how easy it was to decide which campaign we were going to put forward as I was worried so many people would mean many different strong opinions. The voting system proved useful as we could whittle ideas down, exploring each one further as we did, until we reached one almost everyone was happy with. Even those who would not say this campaign was their first choice, could see the benefits of it and agree it had potential.

A persona is the identification of significant fictional characters whose profile gathers up the features of an existing social group. In this way the personas assume the attributes of the groups they represent: from their social and demographic characteristics, to their own needs, desires, habits and cultural backgrounds. Collecting a description of each allows a clear and visible picture of the different kinds of customers within the target market.

In order to gather the relevant information for creating personas of SMASH, our team of advertising and branding students set about constructing a couple of interactive surveys.

Before doing so we came up with the following list of questions we would aim to answer through carrying out the surveys:

Is the person male or female?

What is their marital status?

What is their profession?

Do they have any children?

What age bracket do they fall in to?

Where do they live?

Who do they live with?

Are they the main food shopper of the household?

Are they the main cook at home?

How long on average to they spend preparing a meal in the evening?

How long on average do they spend sitting down to eat a meal?

Our first interactive survey involved asking participants to map out their preferred meal from the given ingredients by drawing a line to connect an picture from each column: side, meat and veg. One of the given side options was potatoes, and the idea was to identify participants who chose this option without specifically mentioning Smash. We then asked the participants who opted for potatoes to rank 5 potato based products – chips, Smash, wedges, microwaveable mash or traditional potatoes from scratch - in order of preference to gauge Smash in relation to some of its competitors.

From this we quickly realised that although most of the participants chose potato as a side, Smash was the lowest ranked of the given products.

Our second interactive survey involved asking participants to draw their preferred meal on a paper plate. Drawing offers a common ground for discussion -  even when participants have different cultural or social backgrounds - so this method proved successful, as participants could animate their choices and thoughts could surface naturally.

After encouraging participants to be involved in shaping our results, we again asked them to rate Smash against its competitors of the relevant food group.

Again, Smash did not prove popular. However, these surveys did reveal that potato is a very popular meal accompaniment among various groups of people.

Two examples of different personas that our group constructed are as follows:

Mikey

Age: 23.

Occupation: Graduate working part-time at both a clothes shop and restaurant while looking for a career relevant to his degree.

Marital Status: Single.

Children: None.

Lives: In a small flat in a city centre with other male graduates.

Interests: Sports, Socialising, Social Drinking.

Relevant Information: Mikey lacks a disposable income and cannot afford to bulk-buy when food shopping. He is very busy juggling multiple jobs and a demanding social life so he mostly eats on the go. He has never properly learned to cook and is still living a typical student lifestyle.

Sarah

Age: 37.

Occupation: Full-time demanding job as a hairdresser where she is on her feet for several hours at a time.

Marital Status: Single.

Children: Two young children. Max, 8, and Sophie, 12.

Lives: In a house in a relatively small town.

Interests: Not a lot of free time but mostly socialising with close friends who pop round to visit her at home.