Items

Judge A

Judge B

Judge C

Judge D

Judge E

Kappa measure

1

3

3

3

3

3

1*

2

3

3

3

3

2

0.6*

3

3

3

3

3

3

1*

4

3

3

3

3

3

1*

5

3

3

3

3

3

1*

6

3

3

3

3

3

1*

7

3

3

3

3

2

0.6*

8

3

3

3

3

3

1*

9

3

3

3

3

3

1*

10

2

-

3

3

3

0.5

11

3

-

3

3

3

1*

12

2

-

3

3

3

0.5

13

3

3

3

3

3

1*

14

3

3

3

3

3

1*

15

2

3

3

3

3

0.6*

16

3

3

3

3

2

0.6*

17

3

3

3

3

3

1*

18

3

3

3

3

3

1*

19

3

3

3

3

3

1*

20

3

3

3

3

3

1*

21

3

3

3

3

3

1*

22

3

3

3

3

3

1*

23

3

3

3

3

3

1*

24

3

3

3

3

3

1*

25

3

3

3

3

3

1*

26

3

3

3

3

3

1*

27

3

3

3

3

3

1*

28

3

3

3

3

3

1*

29

3

3

3

3

3

1*

30

2

1

2

3

3

0.2

31

3

3

3

3

3

1*

32

3

3

3

3

3

1*

33

3

3

3

3

3

1*

34

3

3

3

3

2

0.6*

35

3

3

3

3

3

1*

36

3

3

3

3

3

1*

37

3

3

3

3

3

1*

38

3

3

3

3

3

1*

39

3

3

3

3

3

1*

40

3

3

3

3

3

1*