Innovation & chaos ... in search of optimality

Do my teams agree? Assuring validity and fairness in group assignments through peer assessment

Teachers have long used group projects to improve students’ teamwork skills and future-ready employability. Teammate peer assessment is increasingly adopted to improve the fairness of group assignment grades and to minimise the risk of dysfunctional teamwork behaviour such as freeloading. Learn how to assure that your peer assessment results are fair and valid.

More information

Two workshops: improving learning outcomes from group assignments through teammate peer feedback.

Join Patrick Dodd and me in Auckland to share experience about improving learning outcomes from group assignments through teammate peer feedback. 

We're presenting two experiential workshops as part of Ako Aotearoa Talking Teaching Diverse Learners 2019: Inclusive Teaching, 28-29 November at Unitec Institute of Technology, Auckland, New Zealand.

Full programme and registration at https://lnkd.in/fum-DnB 

Ako Aotearoa is the New Zealand Academy of Tertiary Teaching Excellence.

MP John Bercow interacts vigorously with Harvard Kennedy School students 2019-09-16. I was delighted and inspired to learn from this robust champion of representative parliamentary democracy.

A public address by the Right Honourable John Bercow, Member of Parliament and Speaker of the House of Commons, United Kingdom.

Moderated by Cathryn Cluver Ashbrook, Executive Director, The Project on Europe and the Transatlantic Relationship, Harvard Kennedy School, Institute of Politics, Harvard University, Boston, Massachusetts.

Alternative slideshow access

Monday, September 16, 2019

Digitally-enabled team peer assessment: a fast, fly-past

In situations of group work, how do we 

Raise group members’ employability competencies

Minimise free-riding

Identify dysfunctional team behaviour

Calculate a fair grade or bonus based on contribution, leadership, and teamwork?

We propose that conducting at least two peer assessments, one formative then one summative is the best practice solution for raising individual results, team performance, employability, and minimizing risks (Mellalieu & Dodd, 2019).

The video presents a fast, fly-by of a digitally-enabled solution addressing these issues when they arise from group work in higher education, such as team-based learning (TBL).

An effective solution provides a flexible and comprehensive platform for team coaches and teachers to manage the group-based peer assessment and peer feedback processes. These processes contribute to several positive feedback loops that improve team effectiveness, academic results and employability (Mellalieu, 2019).

Decision criteria

What factors are relevant to making a choice about a preferred platform for managing peer feedback and peer assessment in teams? 

Amongst the criteria we suggest are

Result preview, publication, and update

Speed to deploy

Team and individual dysfunction alerts

Personal result calculation

Teamset management - ability to adjust the composition of one or more teams during the progress of a peer assessment

Standardised qualitative and qualitative rating rubrics

Automated deployment and notifications to team members and teachers

The video illustrates the relevance and importance of these features from the perspective of both a teacher and team member.

Learn more: Best practice in academic team assignments

View our experience-based best practices for peer assessment preparation, diagnosing team health, and team feedback in our recorded webinar

Improving Group Assignments through Peer Feedback and Peer Assessment. 

References

The turbocharging effect of peer feedback on team effectiveness, academic performance and employability in student teams

Teamwork is often used in academic settings to help improve students’ academic performance whilst developing valued career-ready competencies such as teamwork, communications, project management and leadership. These academic and employability benefits do not arise spontaneously just because students are working on a group project. In contrast, the teacher should take steps to encourage these benefits’ development through initiating peer assessment and peer feedback as part of the academic programme. We justified these steps in our three-stage model of the team peer feedback process. Here we examine more closely the mechanism through which these benefits are delivered. Specifically, our Causal Loop Diagram illustrates the reinforcing effects of peer assessment on individual motivation, team performance, and, ultimately, employability (Maani & Cavana, 2000).

Social loafing and the team effectiveness loop

One risk that can arise with student team projects is social loafing or freeloading. The risk of social loafing is first reduced when students know that peer assessment will be used to determine their personal result, a result that may be materially above or below the result awarded to the team by the teacher, the team result (Gibbs, in Sprague, Wilson, & McKenzie, 2019, p. 2). Reducing the risk of social loafing both raises students’ feeling of fairness in the conduct of the teamwork, and raises all team members’ motivation and contribution to the team’s objectives and ultimate academic result. 

Peer feedback, team training and the team effectiveness loop

The quality of peer feedback is also important in raising both students’ motivation and capacity to contribute effort to the team and the development of the competencies needed to work more effectively with the team. These teamwork competencies compound the impact of individual contributions towards the team result. For example, meetings are better planned and executed, quiet students are proactively drawn into conversation, and team members help each other with their learning of course and project concepts (Carr, Herman, Keldsen, Miller, & Wakefield, 2005). Team training, either self-directed by the students, or guided by the teacher, contributes to raising team effectiveness through raising the quality of peer assessment and peer feedback, raising general teamwork competencies, and raising individuals’ motivation for teamwork

Turbocharging the employability loop

A student’s employability is raised as both their academic achievement (personal result) and teamwork competencies are raised through effective peer feedback and improved team results. Ultimately, a positive, reinforcing effect ‘turbocharges’ the entire system as students’ motivation for team work is raised through knowledge of their increased level of employability.

So, what’s the impact?

We find that quality peer assessment and feedback is associated with a 10 to 15 mark increase in a team’s result compared with teams who abdicate their responsibility for accurate, fair, honest and constructive feedback (on a grade scale of 0 to 100 marks). Examples of unconstructive peer feedback include: when a team member rates each of their team members the same; when a team is inconsistent in its ratings; or when a team member’s self-rating differs significantly from the rating received from other team members. Considering the value of feedback, when asked ‘How did other class members most help you achieve your goals?’ 34% of students in an organisation development class stated ‘By giving me honest appraisals of my performance’, the highest mentioned item (Mellalieu & Dodd, 2018).

Assuring quality peer feedback

The teacher plays an important role in ensuring that the quality of peer feedback provided by students to their team members is accurate, fair, honest, and constructive. A quality result is achieved through the teacher rehearsing the students’ use of the peer assessment rubric before the peer assessment survey. Alternatively, early formative use of peer feedback during a group projects helps clarify both the practicalities and consequences of the summative peer assessment process with which the teacher intends to conclude and assess the project.

During the peer assessment, the dashboard of a comprehensive digital peer assessment system will highlight teams and individuals who are at risk of failure, or who abdicated their peer assessment activity, for example, overgenerous, outlier, or inconsistent ratings. Finally, students need a safe space to receive and react constructively to the feedback they have received. 

Further information

Best practices for peer assessment preparation, quality assurance, and receiving feedback are presented in the video of our webinar Improving Group Assignments through Peer Feedback and Peer Assessment. 

References

Better teams from peer assessment and feedback

How does peer assessment and feedback contribute to better team results and raised employability?

Our three stage model of the team peer feedback process suggests four factors are in play: Academic achievement (Team and personal results), Perceptions of fairness, and employability.

Stage 1 Traditional group project In the traditional group project, all team members usually share the same team result, allocated by the teacher to the team’s outputs such as a report or presentation. However, there is often moderate to poor team dynamics, where one team member is a social loafer and/or one team member overachieves or dominates the contribution to the team result. Consequently, we find that teachers suspect that more than half of students dislike group projects. One reason is that students feel the outcomes, measured by the results they gain for their contribution, are not obviously fair.

Stage 2 Summative team peer assessment only In peer assessment, a student’s personal result is adjusted above or below the team result in proportion to a peer rating of their contribution assessed by other team members. According to Gibbs, students are less likely to free ride on group projects if their contributions will be considered in determining grades (Gibbs, 2009 in Sprague, Wilson, & McKenzie, 2019, p. 2). Consequently, introducing peer assessment to group projects will have a small positive impact on the team result and average personal result awarded to team members when compared with the traditional group project. 

However, whilst the sense of fairness amongst most team members is increased when peer assessment is used, there may be a sense of shock or dismay for a student who is rated poorly. Compared with Stage 1, such students are at risk failing the assignment. Consequently, their feeling of fairness is diminished if they have not had the opportunity to learn - well before the end of the group project -  that their contribution was inadequately regarded by their teammates. Furthermore, such a situation could give grounds for an appeal against the fail grade awarded.

Stage 3 Peer assessment with formative feedback We find the most productive outcomes on group projects surface when formative peer assessment is provided to a team early in their group work combined with formative feedback and summative assessment following the group work conclusion.  We find that those team members receiving average peer ratings strive to ‘pull their socks up’ and apply more effort. Receiving developmental feedback through the formative peer assessment process helps these students focus on applying their relative strengths towards the team’s objective, whilst working to manage their weaker areas. Team members also act more proactively to help each other contribute, and ‘check in’ to gain assurance that their contributions are on the right track. Whilst these processes may also apply to the most weakly-rated team member(s), the formative peer assessment certainly gives them fair warning that they are at risk of failing the assignment if their behaviours are not improved significantly. A grounds for appeal against a fail grade is much diminished!

The impact on employability Most importantly, informative feedback early in a group project provides the opportunity for team members to learn and apply important teamwork competencies through the remainder of the project, such as communication, leadership, initiative taking, chairing meetings, and helping others contribute. These are all competencies regarded as absolutely essential for new graduates according to research by the National Association of Colleges and Employers (NACE). The group becomes a learning team. Furthermore, the teamwork competencies help each student improve the team dynamics and academic results achieved in their future studies.

Further reading

Sprague, M., Wilson, K. F., & McKenzie, K. S. (2019). Evaluating the quality of peer and self-evaluations as measures of student contributions to group projects. Higher Education Research & Development, 38(5), 1061–1074. https://doi.org/10.1080/07294360.2019.1615417

 (2018) Employers Rate the Essential Need of the Career Readiness Competencies. Job Outlook 2019. Figure 42, p. 33. Bethlehem, PA. National Association of Colleges and Employers (NACE). Retrieved from https://www.naceweb.org/

The Positive Feedback Loop that Improves Team Effectiveness

A digital platform for peer assessment and peer feedback

At the heart of raising team effectiveness, individual effectiveness and employability lie timely peer assessment and peer feedback during academic teamwork. Peer Assess Pro provides teachers with a digital platform to conduct efficiently all elements of the peer assessment and peer feedback process. Online resources help the student and teacher make best use of the peer feedback learning opportunity. Active warnings notify the teacher of at risk students, dysfunctional teams, or potentially scapegoating behavior.

DOMM Founders Party - Tuesday 12 July, 2016

DOMM Founders’ Day celebrates the journey of six academics at #Unitec Institute of Technology, 1996 - 2016. Collectively, Jeff Marriott (England), Helen Mitchell (Scotland), Greg Wilson (Canada), Asoka Gunaratne (Sri Lanka), Alastair Emerson (The Mainland) and Peter Mellalieu (Wales/Alsace Lorraine) have contributed 100 years of programme leadership and development, teaching, and research with their learning partners (students), industry partners, colleagues, and graduate alumni.

A public celebration for the founders is to be held at Unitec Institute of Technology, Wharekai, 2pm - 3 pm, Tuesday 12 July 2016, 139 Carrington Road, Mt Albert, Auckland. All welcome.

RSVP [email protected]

http://petermellalieu.zenfolio.com/p719487658/slideshow

Over the past twenty years, the department,  currently known as the Department of Management and Marketing (#DOMM), has been variously known also as the School of Applied Management, Department of Management, School of Management and Entrepreneurship (#SME), and has been a member of the Faculty of Business, the Faculty of Creative Industries (sic) and Business (#FCIB) and now the Business and Enterprise Network (BEN).

Related

DOMM Founders' Day 1996-2016 

DOMM Founders' Day celebrates the journey of six academics at #Unitec Institute of Technology, 1996 - 2016. Collectively, Jeff Marriott (England), Helen Mitchell (Scotland), Greg Wilson (Canada), Asoka Gunaratne (Sri Lanka), Alastair Emerson (The Mainland) and Peter Mellalieu (Wales/Alsace Lorraine) have contributed 100 years of programme leadership and development, teaching, and research with their learning partners (students), industry partners, colleagues, and graduate alumni. 

Over the past twenty years, the department,  currently known as the Department of Management and Marketing (#DOMM), has been variously known also as the Department of Management, School of Management and Entrepreneurship (#SME), and has been a member of the Faculty of Business, the Faculty of Creative Industries (sic) and Business (#FCIB) and now the Business and Enterprise Network (BEN). 

Learning Outcomes

Introduce the scope of environmentally sustainable operations management and wealth with green

Identify the global and national drivers for sustainable operations

Discuss exemplary cases of sustainable operations

Identify practical, personal initiatives for embarking on the sustainability journey.

1. The scope of environmentally sustainable operations management and wealth with green

Course textbook

Heizer, J., & Render, B. (2014). Sustainability [Supplement 5]. In Operations Management: Sustainability and Supply Chain Management [Global Edition] (11th ed., pp. 223–239). Pearson Education.

Key sustainability concepts

Corporate social responsibility

Systems view

The commons

Triple bottom line

Design for sustainability

Carbon footprint

Design for dissassembly

Production processes

Logistics and supply chain

Reverse logistics

Life cycle cost of ownership

Life cycle analysis and management (LCA/LCM)

National and international policies, standards, protocols

2. Identify the global and national drivers for sustainable operations

Gilbert, M. (2016, January 19). Our Frightened, Fragmented World in One Davos Map. Retrieved January 21, 2016, from http://www.bloombergview.com/articles/2016-01-19/if-we-can-t-solve-climate-change-other-risks-become-irrelevant-

Rockström, J. (2016, January 20). Can this revolution save our warming planet? Retrieved January 21, 2016, from http://www.weforum.org/agenda/2016/01/revolution-warming-planet

The Global Risks Report 2016. (2016) (11th ed.). Geneva: World Economic Forum. Retrieved from http://www.weforum.org/reports/the-global-risks-report-2016

3. Discuss exemplary cases of sustainable operations

Carpet tile manufacture: Interface Flor and Mission ø

https://www.youtube.com/watch?v=NrsK4Vn1ExY

Image source: Arratia, R. (2014, September 30). The 7 fronts of mount sustainabilityInterface Cut the Fluff. Retrieved May 16, 2016, from http://www.interfacecutthefluff.com/7-fronts-mount-sustainability/

Office chair design: Formway

McLaren, J. (2008). Life Cycle Management - Sustainability and society bridging piece [Case study of Formway Furniture]. Wellington, New Zealand: Landcare Research. Retrieved from http://www.landcareresearch.co.nz/publications/researchpubs/LCM_Briefing.pdf#search=%22mclaren%224. Identify practical, personal initiatives for embarking on the sustainability journey.

Will One Planet Living work for you? 

The ten principles of One Planet Living. (n.d.). Retrieved September 16, 2014, from http://www.oneplanetliving.net/what-is-one-planet-living/the-ten-principles/

One Planet at Unitec

Who We Are - One Planet. (n.d.). Retrieved May 16, 2016, from http://www.oneplanet.unitec.ac.nz/

Postscript

BSNS6350 Wealth with Green assignment

Movie: The Age of Stupid

The Age of Stupid: Trailers: Global Premiere from SPANNER FILMS on Vimeo.

Live to 45 countries , September 22nd 2009

Further readings

Mellalieu, P. J. (2015a). Wealth with Green: Lessons WITH Exemplary Green Enterprise. Presented at the 29th Annual Conference of the Australian and New Zealand Academy of Management: Managing for Peak Performance, Queenstown, NZ: Australian and New Zealand Academy of Management (ANZAM). Retrieved from https://www.academia.edu/19190917/Wealth_with_Green_Lessons_WITH_Exemplary_Green_Enterprise

Mellalieu, P. J. (2015b, December 16). Wealth WITH Green: A professional academic course. Retrieved from http://pogus.tumblr.com/post/118188712763/wealthwithgreen

The Unitec Master of Business Innovation and Entrepreneurship programme (MBIE) employed unique, purpose-designed learning approaches. This album reveals photos from the MBIE orientation, leadership, and team work module, Capabilities for Innovators. Students begin the module facilitated through several outdoor adventure learning cycles. Through the course, the students (learning partners), develop their own abilities to design, lead and facilitate capacity building programmes for their own staff. The module culminates in the learning partners facilitating their own (risk-managed!) learning adventures.

Further information

Mellalieu, P. J. (2014, February 9). Great entrepreneurs are born, then made. Retrieved from

http://pogus.tumblr.com/post/76080929964/can-entrepreneurs-be-taught-peter-mellalieu

Leberman, S., & Mellalieu, P. J. (1996). ALP-DevCo and the Action Learning Programme: A Trojan Horse for Moving from Mystery to Mastery [Training educators to use experiential education using an isomorphically-framed training-products development company]. Presented at the Action Learning, Action Research &  Process Management Professional Conference, (ALARPM), University of Queensland, Brisbane. Retrieved from http://www.academia.edu/1510526/Training_educators_to_use_experiential_education_using_an_isomorphically-framed_training-products_development_company

Captain Stratton’s Fancy, by Peter Warlock.

Sea Fever by John Ireland

'Sea Fever' composed by John Ireland, to text by John Masefield. Performed by Peter Mellalieu at a house concert of the Auckland Lieder Group, Inc, 28 February 2016.