Items

Judge A

Judge B

Judge C

Judge D

Judge E

Kappa measure

1

3

3

3

3

3

1*

2

2

3

3

3

3

0.6*

3

3

3

3

3

3

1*

4

3

3

3

3

3

1*

5

3

3

3

3

3

1*

6

3

3

3

3

3

1*

7

3

3

3

3

3

1*

8

3

3

3

3

2

0.6*

9

3

3

3

3

2

0.6*

10

3

3

3

3

3

1*

11

3

3

3

3

3

1*

12

3

3

3

3

2

0.6*

13

3

3

3

3

2

0.6*

14

3

3

3

3

2

0.6*

15

3

3

3

3

3

1*

16

3

3

3

3

3

1*

17

3

3

3

3

3

1*

18

3

3

3

3

3

1*

19

3

3

3

3

3

1*

20

3

3

3

3

3

1*

21

3

3

3

3

3

1*

22

3

3

3

3

3

1*

23

3

3

3

3

3

1*

24

3

3

3

3

3

1*

25

2

3

2

3

3

0.4

26

3

3

3

3

3

1*

27

3

3

3

3

2

0.6*

28

3

3

3

3

3

1*

29

3

3

3

3

3

1*

30

1

1

1

3

2

0.3

31

3

3

3

1

2

0.3

32

3

3

3

3

3

1*

33

3

3

3

3

3

1*

34

3

3

3

3

1

0.6*

35

2

2

3

3

3

0.4

36

3

3

3

3

3

1*

37

3

3

2

3

2

0.4

38

3

3

3

3

3

1*

39

3

3

3

3

3

1*

40

3

3

2

3

3

0.6*