BACKGROUND Hepatocellular carcinoma is the most common primary liver malignancy.From the results of previous studies,Liver Imaging Reporting and Data System(LIRADS)on contrast-enhanced ultrasound(CEUS)has shown satisf...BACKGROUND Hepatocellular carcinoma is the most common primary liver malignancy.From the results of previous studies,Liver Imaging Reporting and Data System(LIRADS)on contrast-enhanced ultrasound(CEUS)has shown satisfactory diagnostic value.However,a unified conclusion on the interobserver stability of this innovative ultrasound imaging has not been determined.The present metaanalysis examined the interobserver agreement of CEUS LI-RADS to provide some reference for subsequent related research.AIM To evaluate the interobserver agreement of LI-RADS on CEUS and analyze the sources of heterogeneity between studies.METHODS Relevant papers on the subject of interobserver agreement on CEUS LI-RADS published before March 1,2020 in China and other countries were analyzed.The studies were filtered,and the diagnostic criteria were evaluated.The selected references were analyzed using the“meta”and“metafor”packages of R software version 3.6.2.RESULTS Eight studies were ultimately included in the present analysis.Meta-analysis results revealed that the summary Kappa value of included studies was 0.76[95%confidence interval,0.67-0.83],which shows substantial agreement.Higgins I2 statistics also confirmed the substantial heterogeneity(I2=91.30%,95%confidence interval,85.3%-94.9%,P<0.01).Meta-regression identified the variables,including the method of patient enrollment,method of consistency testing,and patient race,which explained the substantial study heterogeneity.CONCLUSION CEUS LI-RADS demonstrated overall substantial interobserver agreement,but heterogeneous results between studies were also obvious.Further clinical investigations should consider a modified recommendation about the experimental design.展开更多
AIM: To investigate a classification of endocytoscopy (ECS) images in Barrett’s esophagus (BE) and evaluate its diagnostic performance and interobserver variability.
Background:Bacterial infections are common in patients with decompensated cirrhosis,largely owing to bacterial translocation and cirrhosis-associated immune dysfunction.This study aims to determine the reliability for...Background:Bacterial infections are common in patients with decompensated cirrhosis,largely owing to bacterial translocation and cirrhosis-associated immune dysfunction.This study aims to determine the reliability for classifying infections in patients with decompensated cirrhosis based on the Centers for Disease Control and Prevention(CDC)criteria.Methods:The patients with decompensated cirrhosis with suspicious infection in a registered prospective cohort of cirrhosis from May 1,2014 to February 25,2015 in the ward of First Affiliated Hospital of Zhejiang University were retrospectively identified.Agreement assessment was conducted focusing on site of infection,the possibility of infection,and pathogens of infection on both system level and specific diagnosis level.A subgroup analysis was performed based on with/without acute-on-chronic liver failure(ACLF).Results:A total of 402 infectious episodes among 351 patients were enrolled for consistency analysis.The overall agreement for site of infection was 94%(378/402)(k=0.90,95%CI 0.86–0.94)on system level and 86%(346/402)(k=0.84,95%CI 0.80–0.88)on specific diagnosis level.On possibility of infection,the overall agreement was 81%(306/378)(weighted k=0.71,95%CI 0.65–0.77),with 84%(224/267)(weighted k=0.75,95%CI 0.63–0.87)in patients with ACLF and 80%(70/88)(weighted k=0.68,95%CI 0.60–0.76)in patients without ACLF,respectively.On pathogens of infection,the overall agreement was 72%(60/83)(k=0.70,95%CI 0.60–0.80)among most frequent infections.Conclusion:The agreement for classifying infections in patients with decompensated cirrhosis based on CDC criteria is acceptable overall,suggesting that it can be a useful tool for clinical management in patients with decompensated cirrhosis with suspicious infections.展开更多
Nonalcoholic fatty liver disease(NAFLD), a hepatic manifestation of metabolic syndrome, is the most common chronic liver disease, and the prevalence is rapidly increasing worldwide. Nonalcoholic steatohepatitis(NASH),...Nonalcoholic fatty liver disease(NAFLD), a hepatic manifestation of metabolic syndrome, is the most common chronic liver disease, and the prevalence is rapidly increasing worldwide. Nonalcoholic steatohepatitis(NASH), the severe form of NAFLD, can progress to liver cirrhosis and hepatocellular carcinoma(HCC). Although noninvasive clinical scores and image-based diagnosis for NAFLD have improved, histopathological evaluation of biopsy specimens remains the gold standard for diagnosing NAFLD/NASH. Steatosis, lobular inflammation, and hepatocellular ballooning are all necessary components for the diagnosis of NASH; fibrosis is also typically observed. Other histopathological abnormalities commonly observed in NASH include hepatocellular glycogenated nuclei, lipogranulomas, and acidophil bodies. The characteristics of pediatric NAFLD/NASH differ from adult NAFLD/NASH. Specifically, steatosis and portal inflammation are more severe in pediatric NAFLD, while intralobular inflammation and perisinusoidal fibrosis are milder. Although interobserver agreement for evaluating the extent of steatosis and fibrosis is high, agreement is low for intralobular and portal inflammation. A recently reported histological variant of HCC, steatohepatitic HCC(SH-HCC), showsfeatures that resemble non-neoplastic steatohepatitis,and is thought to be strongly associated with underlying NASH.In this report,we review the histopathological features of NAFLD/NASH.展开更多
BACKGROUND Accurate target volume delineation is the premise for the implementation of precise radiotherapy.Inadequate target volume delineation may diminish tumor control or increase toxicity.Although several clinica...BACKGROUND Accurate target volume delineation is the premise for the implementation of precise radiotherapy.Inadequate target volume delineation may diminish tumor control or increase toxicity.Although several clinical target volume(CTV)delineation guidelines for rectal cancer have been published in recent years,significant interobserver variation(IOV)in CTV delineation still exists among radiation oncologists.However,proper education may serve as a bridge that connects complex guidelines with clinical practice.AIM To examine whether an education program could improve the accuracy and consistency of preoperative radiotherapy CTV delineation for rectal cancer.METHODS The study consisted of a baseline target volume delineation,a 150-min education intervention,and a follow-up evaluation.A 42-year-old man diagnosed with stage IIIC(T3N2bM0)rectal adenocarcinoma was selected for target volume delineation.CTVs obtained before and after the program were compared.Dice similarity coefficient(DSC),inclusiveness index(IncI),conformal index(CI),and relative volume difference[ΔV(%)]were analyzed to quantitatively evaluate the disparities between the participants’delineation and the standard CTV.Maximum volume ratio(MVR)and coefficient of variation(CV)were calculated to assess the IOV.Qualitative analysis included four common controversies in CTV delineation concerning the upper boundary of the target volume,external iliac area,groin area,and ischiorectal fossa.RESULTS Of the 18 radiation oncologists from 10 provinces in China,13 completed two sets of CTVs.In quantitative analysis,the average CTV volume decreased from 809.82 cm3 to 705.21 cm3(P=0.001)after the education program.Regarding the indices for geometric comparison,the mean DSC,IncI,and CI increased significantly,whileΔV(%)decreased remarkably,indicating improved agreement between participants’delineation and the standard CTV.Moreover,an 11.80%reduction in MVR and 18.19%reduction in CV were noted,demonstrating a smaller IOV in delineation after the education program.Regarding qualitative analysis,the greatest variations in baseline were observed at the external iliac area and ischiorectal fossa;61.54%(8/13)and 53.85%(7/13)of the participants unnecessarily delineated the external iliac area and the ischiorectal fossa,respectively.However,the education program reduced these variations.CONCLUSION Wide variations in CTV delineation for rectal cancer are present among radiation oncologists in China's Mainland.A well-structured education program could improve delineation accuracy and reduce IOVs.展开更多
AIM: To compare the interpretation of probe-based confocal laser endomicroscopy (pCLE) findings between endoscopists and gastrointestinal (GI)-pathologists.
To establish consensual definitions of anoperineal lesions of Crohn’s(APLOC)disease and assess interobserver agreement on their diagnosis between experts.METHODSA database of digitally recorded pictures of APLOC was ...To establish consensual definitions of anoperineal lesions of Crohn’s(APLOC)disease and assess interobserver agreement on their diagnosis between experts.METHODSA database of digitally recorded pictures of APLOC was examined by a coordinating group who selected two series of 20 pictures illustrating the various aspects of APLOC.A reading group comprised:eight experts from the SociétéNationale Française de Colo Proctologie group of study and research in proctology and one academic dermatologist.All members of the coordinating and reading groups participated in dedicated meetings.The coordinating group initially conducted a literature review to analyse verbatim descriptions used to evaluate APLOC.The study included two phases:establishment of consensual definitions using a formal consensus method and later assessment of interobserver agreement on the diagnosis of APLOC using photos of APLOC,a standardised questionnaire and Fleiss’s kappa test or descriptive statistics.RESULTSTerms used in literature to evaluate visible APLOC did not include precise definitions or reference to definitions.Most of the expert reports on the first set of photos agreed with the main diagnosis but their verbatim reporting contained substantial variation.The definitions of ulceration(entity,depth,extension),anal skin tags(entity,inflammatory activity,ulcerated aspect),fistula(complexity,quality of drainage,inflammatory activity of external openings),perianal skin lesions(abscess,papules,edema,erythema)and anoperineal scars were validated.For fistulae,they decided to follow the American Gastroenterology Association’s guidelines definitions.The diagnosis of ulceration(κ=0.70),fistulae(κ=0.75),inflammatory activity of external fistula openings(86.6%agreement),abscesses(84.6%agreement)and erythema(100%agreement)achieved a substantial degree of interobserver reproducibility.CONCLUSIONThis study constructed consensual definitions of APLOC and their characteristics and showed that experts have a fair level of interobserver agreement when using most of the definitions.展开更多
Estimating reproducibility is often wrongly thought of as basic science. Although it has a significant clinical relevance, its importance is underestimated. It was Al-exander Pope in 1732 who was frst to understand th...Estimating reproducibility is often wrongly thought of as basic science. Although it has a significant clinical relevance, its importance is underestimated. It was Al-exander Pope in 1732 who was frst to understand the value of reproducibility, with his famous comment “Who shall decide when doctors disagree？”. Pope’s question concerns the medical doctors’ opinion on a patient’s status, which from a statistical point of view may be considered a categorical variable. However, the same question may be posed for continuous quantitative variables. Reproducibility is complementary to variabil-ity： the larger the variability, the lower the reproduc-ibility, and vice versa. Thus, we can think at them as interchangeable, even thought statistical methods have been developed for the estimation of variability. The question now is “Why do we need to know the repro-ducibility of measurements？ ”. The most important and simplest answer is that we need to know how reliable a measured value or a subjective judgment is before tak-ing clinical decisions based on this measurement/judg-ment. Integrating this knowledge in clinical practice is a key aspect of evidence-based medicine.展开更多
Digital pathology is a major revolution in pathology and is changing the clinical routine for pathologists. We work on providing a computer aided diagnosis system that automatically and robustly provides the pathologi...Digital pathology is a major revolution in pathology and is changing the clinical routine for pathologists. We work on providing a computer aided diagnosis system that automatically and robustly provides the pathologist with a second opinion for many diagnosis tasks. However, inter-observer variability prevents thorough validation of any proposed technique for any specific problems. In this work, we study the variability and reliability of proliferation rate estimation from digital pathology images for breast cancer proliferation rate estimation. We also study the robustness of our recently proposed method CAD system for PRE estimation. Three statistical significance tests showed that our automated CAD system was as reliable as the expert pathologist in both brown and blue nuclei estimation on a dataset of 100 images.展开更多
Aims Vegetation sampling employing observers is prone to both inter-observer and intra-observer error.Three types of errors are common:(i)overlooking error(i.e.not observing species actually present),(ii)misidentifica...Aims Vegetation sampling employing observers is prone to both inter-observer and intra-observer error.Three types of errors are common:(i)overlooking error(i.e.not observing species actually present),(ii)misidentification error(i.e.not correctly identifying species)and(iii)estimation error(i.e.not accurately estimating abundance).I conducted a literature review of 59 articles that provided quantitative estimates or statistical inferences regarding observer error in vegetation studies.Important FindingsAlmost all studies(92%)that tested for a statistically significant effect of observer error found at least one significant comparison.In surveys of species composition,mean pseudoturnover(the percentage of species overlooked by one observer but not another)was 10-30%.Species misidentification rates were on the order of 5-10%.The mean coefficient of variation(CV)among observers in surveys of vegetation cover was often several hundred%for species with low cover,although CVs of 25-50%were more representative of species with mean covers of>50%.A variety of metrics and indices(including commonly used diversity indices)and multivariate data analysis techniques(including ordinations and classifications)were found to be sensitive to observer error.Sources of error commonly include both characteristics of the vegetation(e.g.small size of populations,rarity,morphology,phenology)and attributes of the observers(e.g.mental fatigue,personal biases,differences in experience,physical stress).The use of multiple observers,additional training including active feedback approaches,and continual evaluation and calibration among observers are recommended as strategies to reduce observer error in vegetation surveys.展开更多
基金Supported by Health Commission of Hubei Province,China No.WJ2019M077 and No.WJ2019H227Natural Science Foundation of Hubei Province,China No.2019CFB286and Science and Technology Bureau of Shihezi,China No.2019ZH11.
文摘BACKGROUND Hepatocellular carcinoma is the most common primary liver malignancy.From the results of previous studies,Liver Imaging Reporting and Data System(LIRADS)on contrast-enhanced ultrasound(CEUS)has shown satisfactory diagnostic value.However,a unified conclusion on the interobserver stability of this innovative ultrasound imaging has not been determined.The present metaanalysis examined the interobserver agreement of CEUS LI-RADS to provide some reference for subsequent related research.AIM To evaluate the interobserver agreement of LI-RADS on CEUS and analyze the sources of heterogeneity between studies.METHODS Relevant papers on the subject of interobserver agreement on CEUS LI-RADS published before March 1,2020 in China and other countries were analyzed.The studies were filtered,and the diagnostic criteria were evaluated.The selected references were analyzed using the“meta”and“metafor”packages of R software version 3.6.2.RESULTS Eight studies were ultimately included in the present analysis.Meta-analysis results revealed that the summary Kappa value of included studies was 0.76[95%confidence interval,0.67-0.83],which shows substantial agreement.Higgins I2 statistics also confirmed the substantial heterogeneity(I2=91.30%,95%confidence interval,85.3%-94.9%,P<0.01).Meta-regression identified the variables,including the method of patient enrollment,method of consistency testing,and patient race,which explained the substantial study heterogeneity.CONCLUSION CEUS LI-RADS demonstrated overall substantial interobserver agreement,but heterogeneous results between studies were also obvious.Further clinical investigations should consider a modified recommendation about the experimental design.
文摘AIM: To investigate a classification of endocytoscopy (ECS) images in Barrett’s esophagus (BE) and evaluate its diagnostic performance and interobserver variability.
基金This work was supported by Chinese National Natural Science Foundation(No.81870425)the 13-5 State S&T Projects of China(2018ZX10302-206)and Fundamental Research Funds for the Central Universities.
文摘Background:Bacterial infections are common in patients with decompensated cirrhosis,largely owing to bacterial translocation and cirrhosis-associated immune dysfunction.This study aims to determine the reliability for classifying infections in patients with decompensated cirrhosis based on the Centers for Disease Control and Prevention(CDC)criteria.Methods:The patients with decompensated cirrhosis with suspicious infection in a registered prospective cohort of cirrhosis from May 1,2014 to February 25,2015 in the ward of First Affiliated Hospital of Zhejiang University were retrospectively identified.Agreement assessment was conducted focusing on site of infection,the possibility of infection,and pathogens of infection on both system level and specific diagnosis level.A subgroup analysis was performed based on with/without acute-on-chronic liver failure(ACLF).Results:A total of 402 infectious episodes among 351 patients were enrolled for consistency analysis.The overall agreement for site of infection was 94%(378/402)(k=0.90,95%CI 0.86–0.94)on system level and 86%(346/402)(k=0.84,95%CI 0.80–0.88)on specific diagnosis level.On possibility of infection,the overall agreement was 81%(306/378)(weighted k=0.71,95%CI 0.65–0.77),with 84%(224/267)(weighted k=0.75,95%CI 0.63–0.87)in patients with ACLF and 80%(70/88)(weighted k=0.68,95%CI 0.60–0.76)in patients without ACLF,respectively.On pathogens of infection,the overall agreement was 72%(60/83)(k=0.70,95%CI 0.60–0.80)among most frequent infections.Conclusion:The agreement for classifying infections in patients with decompensated cirrhosis based on CDC criteria is acceptable overall,suggesting that it can be a useful tool for clinical management in patients with decompensated cirrhosis with suspicious infections.
文摘Nonalcoholic fatty liver disease(NAFLD), a hepatic manifestation of metabolic syndrome, is the most common chronic liver disease, and the prevalence is rapidly increasing worldwide. Nonalcoholic steatohepatitis(NASH), the severe form of NAFLD, can progress to liver cirrhosis and hepatocellular carcinoma(HCC). Although noninvasive clinical scores and image-based diagnosis for NAFLD have improved, histopathological evaluation of biopsy specimens remains the gold standard for diagnosing NAFLD/NASH. Steatosis, lobular inflammation, and hepatocellular ballooning are all necessary components for the diagnosis of NASH; fibrosis is also typically observed. Other histopathological abnormalities commonly observed in NASH include hepatocellular glycogenated nuclei, lipogranulomas, and acidophil bodies. The characteristics of pediatric NAFLD/NASH differ from adult NAFLD/NASH. Specifically, steatosis and portal inflammation are more severe in pediatric NAFLD, while intralobular inflammation and perisinusoidal fibrosis are milder. Although interobserver agreement for evaluating the extent of steatosis and fibrosis is high, agreement is low for intralobular and portal inflammation. A recently reported histological variant of HCC, steatohepatitic HCC(SH-HCC), showsfeatures that resemble non-neoplastic steatohepatitis,and is thought to be strongly associated with underlying NASH.In this report,we review the histopathological features of NAFLD/NASH.
基金Supported by the Beijing Municipal Science&Technology Commission,No.Z181100001718192the Capital’s Funds for Health Improvement and Research,No.2020-2-1027 and No.2020-1-4021+1 种基金the National Natural Science Foundation,No.82073333the Beijing Natural Science Foundation,No.1212011.
文摘BACKGROUND Accurate target volume delineation is the premise for the implementation of precise radiotherapy.Inadequate target volume delineation may diminish tumor control or increase toxicity.Although several clinical target volume(CTV)delineation guidelines for rectal cancer have been published in recent years,significant interobserver variation(IOV)in CTV delineation still exists among radiation oncologists.However,proper education may serve as a bridge that connects complex guidelines with clinical practice.AIM To examine whether an education program could improve the accuracy and consistency of preoperative radiotherapy CTV delineation for rectal cancer.METHODS The study consisted of a baseline target volume delineation,a 150-min education intervention,and a follow-up evaluation.A 42-year-old man diagnosed with stage IIIC(T3N2bM0)rectal adenocarcinoma was selected for target volume delineation.CTVs obtained before and after the program were compared.Dice similarity coefficient(DSC),inclusiveness index(IncI),conformal index(CI),and relative volume difference[ΔV(%)]were analyzed to quantitatively evaluate the disparities between the participants’delineation and the standard CTV.Maximum volume ratio(MVR)and coefficient of variation(CV)were calculated to assess the IOV.Qualitative analysis included four common controversies in CTV delineation concerning the upper boundary of the target volume,external iliac area,groin area,and ischiorectal fossa.RESULTS Of the 18 radiation oncologists from 10 provinces in China,13 completed two sets of CTVs.In quantitative analysis,the average CTV volume decreased from 809.82 cm3 to 705.21 cm3(P=0.001)after the education program.Regarding the indices for geometric comparison,the mean DSC,IncI,and CI increased significantly,whileΔV(%)decreased remarkably,indicating improved agreement between participants’delineation and the standard CTV.Moreover,an 11.80%reduction in MVR and 18.19%reduction in CV were noted,demonstrating a smaller IOV in delineation after the education program.Regarding qualitative analysis,the greatest variations in baseline were observed at the external iliac area and ischiorectal fossa;61.54%(8/13)and 53.85%(7/13)of the participants unnecessarily delineated the external iliac area and the ischiorectal fossa,respectively.However,the education program reduced these variations.CONCLUSION Wide variations in CTV delineation for rectal cancer are present among radiation oncologists in China's Mainland.A well-structured education program could improve delineation accuracy and reduce IOVs.
文摘AIM: To compare the interpretation of probe-based confocal laser endomicroscopy (pCLE) findings between endoscopists and gastrointestinal (GI)-pathologists.
文摘To establish consensual definitions of anoperineal lesions of Crohn’s(APLOC)disease and assess interobserver agreement on their diagnosis between experts.METHODSA database of digitally recorded pictures of APLOC was examined by a coordinating group who selected two series of 20 pictures illustrating the various aspects of APLOC.A reading group comprised:eight experts from the SociétéNationale Française de Colo Proctologie group of study and research in proctology and one academic dermatologist.All members of the coordinating and reading groups participated in dedicated meetings.The coordinating group initially conducted a literature review to analyse verbatim descriptions used to evaluate APLOC.The study included two phases:establishment of consensual definitions using a formal consensus method and later assessment of interobserver agreement on the diagnosis of APLOC using photos of APLOC,a standardised questionnaire and Fleiss’s kappa test or descriptive statistics.RESULTSTerms used in literature to evaluate visible APLOC did not include precise definitions or reference to definitions.Most of the expert reports on the first set of photos agreed with the main diagnosis but their verbatim reporting contained substantial variation.The definitions of ulceration(entity,depth,extension),anal skin tags(entity,inflammatory activity,ulcerated aspect),fistula(complexity,quality of drainage,inflammatory activity of external openings),perianal skin lesions(abscess,papules,edema,erythema)and anoperineal scars were validated.For fistulae,they decided to follow the American Gastroenterology Association’s guidelines definitions.The diagnosis of ulceration(κ=0.70),fistulae(κ=0.75),inflammatory activity of external fistula openings(86.6%agreement),abscesses(84.6%agreement)and erythema(100%agreement)achieved a substantial degree of interobserver reproducibility.CONCLUSIONThis study constructed consensual definitions of APLOC and their characteristics and showed that experts have a fair level of interobserver agreement when using most of the definitions.
文摘Estimating reproducibility is often wrongly thought of as basic science. Although it has a significant clinical relevance, its importance is underestimated. It was Al-exander Pope in 1732 who was frst to understand the value of reproducibility, with his famous comment “Who shall decide when doctors disagree？”. Pope’s question concerns the medical doctors’ opinion on a patient’s status, which from a statistical point of view may be considered a categorical variable. However, the same question may be posed for continuous quantitative variables. Reproducibility is complementary to variabil-ity： the larger the variability, the lower the reproduc-ibility, and vice versa. Thus, we can think at them as interchangeable, even thought statistical methods have been developed for the estimation of variability. The question now is “Why do we need to know the repro-ducibility of measurements？ ”. The most important and simplest answer is that we need to know how reliable a measured value or a subjective judgment is before tak-ing clinical decisions based on this measurement/judg-ment. Integrating this knowledge in clinical practice is a key aspect of evidence-based medicine.
文摘Digital pathology is a major revolution in pathology and is changing the clinical routine for pathologists. We work on providing a computer aided diagnosis system that automatically and robustly provides the pathologist with a second opinion for many diagnosis tasks. However, inter-observer variability prevents thorough validation of any proposed technique for any specific problems. In this work, we study the variability and reliability of proliferation rate estimation from digital pathology images for breast cancer proliferation rate estimation. We also study the robustness of our recently proposed method CAD system for PRE estimation. Three statistical significance tests showed that our automated CAD system was as reliable as the expert pathologist in both brown and blue nuclei estimation on a dataset of 100 images.
文摘Aims Vegetation sampling employing observers is prone to both inter-observer and intra-observer error.Three types of errors are common:(i)overlooking error(i.e.not observing species actually present),(ii)misidentification error(i.e.not correctly identifying species)and(iii)estimation error(i.e.not accurately estimating abundance).I conducted a literature review of 59 articles that provided quantitative estimates or statistical inferences regarding observer error in vegetation studies.Important FindingsAlmost all studies(92%)that tested for a statistically significant effect of observer error found at least one significant comparison.In surveys of species composition,mean pseudoturnover(the percentage of species overlooked by one observer but not another)was 10-30%.Species misidentification rates were on the order of 5-10%.The mean coefficient of variation(CV)among observers in surveys of vegetation cover was often several hundred%for species with low cover,although CVs of 25-50%were more representative of species with mean covers of>50%.A variety of metrics and indices(including commonly used diversity indices)and multivariate data analysis techniques(including ordinations and classifications)were found to be sensitive to observer error.Sources of error commonly include both characteristics of the vegetation(e.g.small size of populations,rarity,morphology,phenology)and attributes of the observers(e.g.mental fatigue,personal biases,differences in experience,physical stress).The use of multiple observers,additional training including active feedback approaches,and continual evaluation and calibration among observers are recommended as strategies to reduce observer error in vegetation surveys.
