#formative evaluation

Again, this blog proceeds with its promotion of a consumer government approach to civics education.  To date, this blog has addressed various concerns with such a change including commentary on the needs for the approach, needs of the subject matter, curricular goals and objectives, related teaching strategies, etc. 

Interested readers who have not followed the proceeding postings, but wish to read them, are directed to the first one, “A Practical Turn” (March 19, 2024).  It can be found through the citation here[1] or, along with subsequent postings, the archive feature of the blog.  By reviewing them, readers can consider a multi-faceted rationale for this proposal.

          This posting addresses instructional evaluation.  Evaluation at the instructional level can utilize a variety of techniques.  On a summative basis, evaluation should be criterion reference based.  Competencies on the following concerns need to be spelled out:

Cognitive – knowledge (recall) of basic facts and procedures, application of inquiry skills (modeled after scientific method and other reputable research methodologies), logical deduction skills, and appropriate communication skills.  And …

Affective – non-graded attitudes that relate to dispositions regarding citizen participation regarding governing issues.

A pre-test, that can use a multiple choice or open-ended format, need to be administered to determine:

Pre-requisite knowledge and skills attained (e.g., sixth grade reading and mathematics level proficiencies),

Knowledge of subject matter that instruction will address, and

Relevant value orientations students hold prior to instruction.

As with most courses of study, there would most likely be a final written examination.  That test will provide evidence as to the success of instruction by comparing results with pre-test to identify measured change.  By comparing results between pre- and post-testing, specific areas of deficiencies can be detected by conducting an item analysis.

          On a formative basis, individual units could provide unit tests and quizzes, formulated from lesson objectives, and on student-project work products.  Again, course objectives should guide these reviews.  Informally, teachers can rely on classroom participation and on one-on-one discussions with students to acquire feedback information.

           With those concerns reviewed, there remains only one topic this blogger wishes to address, that is, evaluation of this proposed course of study itself.  The next posting will mark the end of this proposal by looking at curriculum evaluation, asking how one can determine how effective can a course on consumer government, as outlined in this blog, be.

The learning teams at Family Dollar have made a great decision and a great commitment to ISPI Charlotte to provide memberships for 18 or 20 of their employees. To welcome them into the chapter I offered a lunch and learn for them. Since they missed the introductory year of programs on Instructional Design, I did a session entitled, “Instructional Design:  How to Sell the Real Value.”

We had just over 20 people attend—must have been a good lunch.  There seems to be a common theme running through training departments in America. People still have confidence in Instructional Design principles, but making it work in today’s world of tight deadlines and last minute requests is a challenge.  We decided that it was a good idea to “pick your battles” and wait for the right project. By picking the right project, you can be sure to make a good example and show how the process really works at its best. I emphasized just three aspects of instructional design to begin selling the real, practical value of the process:

1. Do the proper amount of analysis, especially task analysis, to define the real behavioral outcomes at the beginning of the project. This helps build consensus among SMEs.

2. Conduct what we call a “Blueprint Meeting” as soon as you have a defined Measurement Strategy and Instructional Strategy with all SMEs, stakeholders, designers—everybody on the project team. This gets everybody on board with consensus building early in the project.

3. Conduct Formative Evaluation. Build a short prototype that exemplifies your Measurement and Instructional strategies and try it out with four to six learners to get their feedback and observe how well it works. Make revisions, complete the course, and then do a pilot with twelve to twenty learners in a more formal setting. Observe, get feedback, and make final revisions.