JEMS manual main pages: Home | Recent Changes | Page Index | Site Map | QwikiSyntax

This page explain the difference between weight and global weight for the review scores. The scores in the review are used to calculate the final average for the paper. JEMS support scores that weight different for different reviewers.

JEMS calculates the final score for each paper both considering and not considering the global weight. If you define to global weight for scores, global weight should be ignored.

The classical example to use global weights is the reviewer confidence or experience in the area. We wanted reviewers with more confidence to weight more in the final score than reviewers with less confidence. Since more than one score could be used to calculate the weight of a reviewer, with split this value in the global weight. In the example below I didn’t split the global weight because this is not common.

Fake review form configuration:
Score 1: Values: 1 to 5; Weight: 0.25; GWeight: 0 ß E.g., Originality
Score 2: Values: 1 to 3; Weight: 0.25; GWeight: 0 ß E.g., Technical quality
Score 3: Values: 1 to 2; Weight: 0.5; GWeight: 0 ß E.g., Final recommendation
Score 4: Values: 1 to 3; Weight: 0; GWeight: 1 ß E.g., Confidence in the review

Fake scores from reviewer 1:
Score 1: 1
Score 2: 1
Score 3: 1
Score 4: 3
This reviewer score: ( (1/5) * 0.25 + (1/3) * 0.25 + (1/2) * 0.5 ) * 10 = 3.83 ß Global Weight is not used here
This reviewer relative weight: (3/3) * 1 = 1
This reviewer final score: 3.83 * 1 = 3.83

Fake scores from reviewer 2:
Score 1: 5
Score 2: 3
Score 3: 2
Score 4: 1
This reviewer score: ( (5/5) * 0.25 + (3/3) * 0.25 + (2/2) * 0.5 ) * 10 = 10 ß Global Weight is not used here

This reviewer relative weight: (1/3) * 1 = 0.33
This reviewer final score: 10 * 0.33 = 3.3
Score for this paper: (3.83 + 3.3 ) / (1 + 0.33) = 5.36


Essa página em português: como definer pesos dos scores.