statistics data collection observation

Cohen’s kappa coefficient is a statistic which measures inter-rater agreement for qualitative (categorical) items. It is generally thought to be a more robust measure than simple percent agreement calculation, since k takes into account the agreement occurring by chance. Cohen’s kappa measures the agreement between two raters who each classify N items into C mutually exclusive categories.

Cohen’s kappa coefficient is defined and given by the following function:


${k = \frac{p_0 – p_e}{1-p_e} = 1 – \frac{1-p_o}{1-p_e}}$

Where −

  • ${p_0}$ = relative observed agreement among raters.
  • ${p_e}$ = the hypothetical probability of chance agreement.

${p_0}$ and ${p_e}$ are computed using the observed data to calculate the probabilities of each observer randomly saying each category. If the raters are in complete agreement then ${k}$ = 1. If there is no agreement among the raters other than what would be expected by chance (as given by ${p_e}$), ${k}$ ≤ 0.


Problem Statement:

Suppose that you were analyzing data related to a group of 50 people applying for a grant. Each grant proposal was read by two readers and each reader either said “Yes” or “No” to the proposal. Suppose the disagreement count data were as follows, where A and B are readers, data on the diagonal slanting left shows the count of agreements and the data on the diagonal slanting right, disagreements:

Yes No
A Yes 20 5
No 10 15

Calculate Cohen’s kappa coefficient.


Note that there were 20 proposals that were granted by both reader A and reader B and 15 proposals that were rejected by both readers. Thus, the observed proportionate agreement is

${p_0 = \frac{20+15}{50} = 0.70}$

To calculate ${p_e}$ (the probability of random agreement) we note that:

  • Reader A said “Yes” to 25 applicants and “No” to 25 applicants. Thus reader A said “Yes” 50% of the time.
  • Reader B said “Yes” to 30 applicants and “No” to 20 applicants. Thus reader B said “Yes” 60% of the time.

Using formula P(A and B) = P(A) x P(B) where P is probability of event occuring.

The probability that both of them would say “Yes” randomly is 0.50 x 0.60 = 0.30 and the probability that both of them would say “No” is 0.50 x 0.40 = 0.20. Thus the overall probability of random agreement is ${p_e}$ = 0.3 + 0.2 = 0.5.

So now applying our formula for Cohen’s Kappa we get:

${k = \frac{p_0 – p_e}{1-p_e} = \frac{0.70 – 0.50}{1-0.50} = 0.40}$

Table of Contents
1.statistics adjusted rsquared

2.statistics analysis of variance

3.statistics arithmetic mean

4.statistics arithmetic median

5.statistics arithmetic mode

6.statistics arithmetic range

7.statistics bar graph

8.statistics best point estimation

9.statistics beta distribution

10.statistics binomial distribution

11.statistics blackscholes model

12.statistics boxplots

13.statistics central limit theorem

14.statistics chebyshevs theorem

15.statistics chisquared distribution

16.statistics chi squared table

17.statistics circular permutation

18.statistics cluster sampling

19.statistics cohens kappa coefficient

20.statistics combination

21.statistics combination with replacement

22.statistics comparing plots

23.statistics continuous uniform distribution

24.statistics cumulative frequency

25.statistics coefficient of variation

26.statistics correlation coefficient

27.statistics cumulative plots

28.statistics cumulative poisson distribution

29.statistics data collection

30.statistics data collection questionaire designing

31.statistics data collection observation

32.statistics data collection case study method

33.statistics data patterns

34.statistics deciles statistics

35.statistics dot plot

36.statistics exponential distribution

37.statistics f distribution

38.statistics f test table

39.statistics factorial

40.statistics frequency distribution

41.statistics gamma distribution

42.statistics geometric mean

43.statistics geometric probability distribution

44.statistics goodness of fit

45.statistics grand mean

46.statistics gumbel distribution

47.statistics harmonic mean

48.statistics harmonic number

49.statistics harmonic resonance frequency

50.statistics histograms

51.statistics hypergeometric distribution

52.statistics hypothesis testing

53.statistics interval estimation

54.statistics inverse gamma distribution

55.statistics kolmogorov smirnov test

56.statistics kurtosis

57.statistics laplace distribution

58.statistics linear regression

59.statistics log gamma distribution

60.statistics logistic regression

61.statistics mcnemar test

62.statistics mean deviation

63.statistics means difference

64.statistics multinomial distribution

65.statistics negative binomial distribution

66.statistics normal distribution

67.statistics odd and even permutation

68.statistics one proportion z test

69.statistics outlier function

70.statistics permutation

71.statistics permutation with replacement

72.statistics pie chart

73.statistics poisson distribution

74.statistics pooled variance r

75.statistics power calculator

76.statistics probability

77.statistics probability additive theorem

78.statistics probability multiplicative theorem

79.statistics probability bayes theorem

80.statistics probability density function

81.statistics process capability cp amp process performance pp

82.statistics process sigma

83.statistics quadratic regression equation

84.statistics qualitative data vs quantitative data

85.statistics quartile deviation

86.statistics range rule of thumb

87.statistics rayleigh distribution

88.statistics regression intercept confidence interval

89.statistics relative standard deviation

90.statistics reliability coefficient

91.statistics required sample size

92.statistics residual analysis

93.statistics residual sum of squares

94.statistics root mean square

95.statistics sample planning

96.statistics sampling methods

97.statistics scatterplots

98.statistics shannon wiener diversity index

99.statistics signal to noise ratio

100.statistics simple random sampling

101.statistics skewness

102.statistics standard deviation

103.statistics standard error se

104.statistics standard normal table

105.statistics statistical significance

106.statistics formulas

107.statistics notations

108.statistics stem and leaf plot

109.statistics stratified sampling

110.statistics student t test

111.statistics sum of square

112.statistics tdistribution table

113.statistics ti 83 exponential regression

114.statistics transformations

115.statistics trimmed mean

116.statistics type i amp ii errors

117.statistics variance

118.statistics venn diagram

119.statistics weak law of large numbers

120.statistics z table

121.discuss statistics

Leave a Reply

Your email address will not be published. Required fields are marked *