Aug 30, 2014
This tool is designed for the problem of peer-grading/peer-reviewing. Given a set of assignments that need to be graded, we aggregate the grades provided by the peer graders/reviewers. The peer-grading toolkit takes as input a set of orderings provided by the reviewers indicating their preferences over the different assignments. For instance, in the example provided below reviewer 1 rates assignment 1 as being better than assignment 2 which in turn is better than assignment 3. Given these orderings you can use the tool to produce an overall ranking of all assignments as well as an estimate of how reliable each of the different reviewers were.
