Purpose
To analyse the
results of an R-index test.
R_index_rank_withMD.xlsx
Background
The R-index applies signal detection theory as an alternative approach to discrimination testing. It is often used where there are a large number of products requiring a ‘yes/no’ response relative to a control or reference product.
It is also applicable where the ‘yes/no’ response is broken down into ‘degrees of certainty’ – e.g., ‘different, I am sure’; ‘different, I am not sure’; ‘different, I do not know and guess’; ‘the same, I do not know and guess’; ‘the same, I am not sure’; ‘the same, I am sure’.
In the above scenario, the R-index results from converting the rating data to a measure related to the area under the ROC (receiver operation characteristics) curve.
Options
- Reference Product: Select which product
should be used as a reference.
- Type of test: Similarity or Difference test
- Difference:
If the test is a similarity test, the threshold that will be considered
non-negligible. That is, H0 is that the difference is greater than
this threshold.
- Significance level: Type-I error
- Order of the Scale: Increase or Decrease.
Should an increase or decrease in scores in the data set result in an increase
in the certainty of difference. By default, a score of 1 means the assessor is
sure the samples are the same, and a score at the top of the scale means the
assessor is sure the samples are different.
- Number of Decimals for Values: Required number
of decimals for values given in the results.
- Number of Decimals for P-Values: Required number
of decimals for any p-values given in the results.
Results and Interpretation
Frequency table
This is a contingency table gives a summary of how many responses given
for each of the scale categories. The products form the rows and there is a column
for each level of the scale.
It is a useful
visual look see. It is also the basis for calculating the R-index
using the traditional psychophysics principles from Thurstone.
R-index table
The second tab is
the R-Index table showing the estimated R-Index for each product. The following
values are given:
- N. The number of times each sample was
evaluated.
- An estimate of the R-index. R-index is
expressed as a proportion. It
is equivalent to the probability of correctly choosing one of two samples in a
paired comparison, but is calculated from theoretical paired comparisons
between each possible signal/noise pairing.
- The critical value Z (at the significance
threshold).
- The p-value to conclude whether each product
was significantly different from the reference product.
Interpretation
At a basic
interpretation level, the p-values are used to establish if each product is
statistically different from (or similar to) the reference product.
The key point to
note with the R-index is that it is based on Thurstonian scaling and the law of
comparative judgement.
R index values can
be analysed using parametric statistics e.g. ANOVA (as can d’).
Significance of
R-index can be determined from critical value tables for ‘n’ responses.
‘Equivalent’ d’ can
be determined for r-index values from published tables.
- The R-index is
calculated manually.
References
- Bi & O’Mahony (2007) Updated and extended table for testing the Significance of the r-index Journal of Sensory Studies 22 713–720.
- Lawless & Heymann (1998) Sensory Evaluation of Foods Chapter 5 Discrimination Theories and advanced topics 140-159.
- Macmillan N & Creelman C D (2004) Detection Theory A User’s Guide. Lawrence Arlbaum Associates Inc.
- O’Mahony M (1992) Understanding Discrimination Tests: A user friendly treatment of response bias, rating and ranking R-index tests and their relationship to signal detection. Journal of Sensory Studies 7 1-47.
- O’Mahony M & Rousseau B (2002) Discrimination testing: a few ideas, old and new. Food Quality and Preference 14 157-164.
- Rousseau B (2011) Measuring product similarities: Are two indices, R-Index and d′, interchangeable? IFT Annual Meeting, New Orleans, USA
- Brown J (1974) Recognition assessment by rating and ranking. Brit J Psycho 65 13-22.
- Lee H-S, van Hout D Quantification of Sensory & Food quality: The R-Index Analysis Journal of Food Science 2009.
- Bi, J. (2006). Statistical Analyses for R-Index. Journal of Sensory Studies, 21, 584-600.
- Lawless, H.T. and Heymann, H. (2010). Sensory Evaluation of Food – Principles and Practices. Springer.
- O’Mahony, M. (1986). Sensory Evaluation of Food – Statistical Methods and Procedures. CRC Press.