Estimating reproducibility is often wrongly thought of as basic science. Although it has a significant clinical relevance, its importance is underestimated. It was Al-exander Pope in 1732 who was frst to understand th...Estimating reproducibility is often wrongly thought of as basic science. Although it has a significant clinical relevance, its importance is underestimated. It was Al-exander Pope in 1732 who was frst to understand the value of reproducibility, with his famous comment “Who shall decide when doctors disagree？”. Pope’s question concerns the medical doctors’ opinion on a patient’s status, which from a statistical point of view may be considered a categorical variable. However, the same question may be posed for continuous quantitative variables. Reproducibility is complementary to variabil-ity： the larger the variability, the lower the reproduc-ibility, and vice versa. Thus, we can think at them as interchangeable, even thought statistical methods have been developed for the estimation of variability. The question now is “Why do we need to know the repro-ducibility of measurements？ ”. The most important and simplest answer is that we need to know how reliable a measured value or a subjective judgment is before tak-ing clinical decisions based on this measurement/judg-ment. Integrating this knowledge in clinical practice is a key aspect of evidence-based medicine.展开更多
Aims Vegetation sampling employing observers is prone to both inter-observer and intra-observer error.Three types of errors are common:(i)overlooking error(i.e.not observing species actually present),(ii)misidentifica...Aims Vegetation sampling employing observers is prone to both inter-observer and intra-observer error.Three types of errors are common:(i)overlooking error(i.e.not observing species actually present),(ii)misidentification error(i.e.not correctly identifying species)and(iii)estimation error(i.e.not accurately estimating abundance).I conducted a literature review of 59 articles that provided quantitative estimates or statistical inferences regarding observer error in vegetation studies.Important FindingsAlmost all studies(92%)that tested for a statistically significant effect of observer error found at least one significant comparison.In surveys of species composition,mean pseudoturnover(the percentage of species overlooked by one observer but not another)was 10-30%.Species misidentification rates were on the order of 5-10%.The mean coefficient of variation(CV)among observers in surveys of vegetation cover was often several hundred%for species with low cover,although CVs of 25-50%were more representative of species with mean covers of>50%.A variety of metrics and indices(including commonly used diversity indices)and multivariate data analysis techniques(including ordinations and classifications)were found to be sensitive to observer error.Sources of error commonly include both characteristics of the vegetation(e.g.small size of populations,rarity,morphology,phenology)and attributes of the observers(e.g.mental fatigue,personal biases,differences in experience,physical stress).The use of multiple observers,additional training including active feedback approaches,and continual evaluation and calibration among observers are recommended as strategies to reduce observer error in vegetation surveys.展开更多
文摘Estimating reproducibility is often wrongly thought of as basic science. Although it has a significant clinical relevance, its importance is underestimated. It was Al-exander Pope in 1732 who was frst to understand the value of reproducibility, with his famous comment “Who shall decide when doctors disagree？”. Pope’s question concerns the medical doctors’ opinion on a patient’s status, which from a statistical point of view may be considered a categorical variable. However, the same question may be posed for continuous quantitative variables. Reproducibility is complementary to variabil-ity： the larger the variability, the lower the reproduc-ibility, and vice versa. Thus, we can think at them as interchangeable, even thought statistical methods have been developed for the estimation of variability. The question now is “Why do we need to know the repro-ducibility of measurements？ ”. The most important and simplest answer is that we need to know how reliable a measured value or a subjective judgment is before tak-ing clinical decisions based on this measurement/judg-ment. Integrating this knowledge in clinical practice is a key aspect of evidence-based medicine.
文摘Aims Vegetation sampling employing observers is prone to both inter-observer and intra-observer error.Three types of errors are common:(i)overlooking error(i.e.not observing species actually present),(ii)misidentification error(i.e.not correctly identifying species)and(iii)estimation error(i.e.not accurately estimating abundance).I conducted a literature review of 59 articles that provided quantitative estimates or statistical inferences regarding observer error in vegetation studies.Important FindingsAlmost all studies(92%)that tested for a statistically significant effect of observer error found at least one significant comparison.In surveys of species composition,mean pseudoturnover(the percentage of species overlooked by one observer but not another)was 10-30%.Species misidentification rates were on the order of 5-10%.The mean coefficient of variation(CV)among observers in surveys of vegetation cover was often several hundred%for species with low cover,although CVs of 25-50%were more representative of species with mean covers of>50%.A variety of metrics and indices(including commonly used diversity indices)and multivariate data analysis techniques(including ordinations and classifications)were found to be sensitive to observer error.Sources of error commonly include both characteristics of the vegetation(e.g.small size of populations,rarity,morphology,phenology)and attributes of the observers(e.g.mental fatigue,personal biases,differences in experience,physical stress).The use of multiple observers,additional training including active feedback approaches,and continual evaluation and calibration among observers are recommended as strategies to reduce observer error in vegetation surveys.
