Generative Artificial Intelligence(GenAI)systems have achieved remarkable capabilities across text,code,and image generation;however,their outputs remain prone to errors,hallucinations,and biases.Users often overtrust...Generative Artificial Intelligence(GenAI)systems have achieved remarkable capabilities across text,code,and image generation;however,their outputs remain prone to errors,hallucinations,and biases.Users often overtrust these outputs due to limited transparency,which can lead to misuse and decision errors.This study addresses the challenge of calibrating trust in GenAI through a human centered testing framework enhanced with adaptive explainability.We introduce a methodology that adjusts explanations dynamically according to user expertise,model output confidence,and contextual risk factors,providing guidance that is informative but not overwhelming.The framework was evaluated using outputs from OpenAI’s Generative Pretrained Transformer 4(GPT-4)for text and code generation and Stable Diffusion,a deep generative image model,for image synthesis.The evaluation covered text,code,and visual modalities.A dataset of 5000 GenAI outputs was created and reviewed by a diverse participant group of 360 individuals categorized by expertise level.Results show that adaptive explanations improve error detection rates,reduce the mean squared trust calibration error,and maintain efficient decision making compared with both static and no explanation conditions.Theframework increased error detection by up to 16% across expertise levels,a gain that can provide practical benefits in high stakes fields.For example,in healthcare it may help identify diagnostic errors earlier,and in law it may prevent reliance on flawed evidence in judicial work.These improvements highlight the framework’s potential to make Artificial Intelligence(AI)deployment safer and more accountable.Visual analyses,including trust accuracy plots,reliability diagrams,and misconception maps,show that the adaptive approach reduces overtrust and reveals patterns of misunderstanding across modalities.Statistical results confirmthe robustness of thesefindings across novice,intermediate,and expert users.The study offers insights for designing explanations that balance completeness and simplicity to improve trust calibration and cognitive load.The approach has implications for safe and transparent GenAI deployment and can inform both AI interface design and policy development for responsible AI use.展开更多
文摘Generative Artificial Intelligence(GenAI)systems have achieved remarkable capabilities across text,code,and image generation;however,their outputs remain prone to errors,hallucinations,and biases.Users often overtrust these outputs due to limited transparency,which can lead to misuse and decision errors.This study addresses the challenge of calibrating trust in GenAI through a human centered testing framework enhanced with adaptive explainability.We introduce a methodology that adjusts explanations dynamically according to user expertise,model output confidence,and contextual risk factors,providing guidance that is informative but not overwhelming.The framework was evaluated using outputs from OpenAI’s Generative Pretrained Transformer 4(GPT-4)for text and code generation and Stable Diffusion,a deep generative image model,for image synthesis.The evaluation covered text,code,and visual modalities.A dataset of 5000 GenAI outputs was created and reviewed by a diverse participant group of 360 individuals categorized by expertise level.Results show that adaptive explanations improve error detection rates,reduce the mean squared trust calibration error,and maintain efficient decision making compared with both static and no explanation conditions.Theframework increased error detection by up to 16% across expertise levels,a gain that can provide practical benefits in high stakes fields.For example,in healthcare it may help identify diagnostic errors earlier,and in law it may prevent reliance on flawed evidence in judicial work.These improvements highlight the framework’s potential to make Artificial Intelligence(AI)deployment safer and more accountable.Visual analyses,including trust accuracy plots,reliability diagrams,and misconception maps,show that the adaptive approach reduces overtrust and reveals patterns of misunderstanding across modalities.Statistical results confirmthe robustness of thesefindings across novice,intermediate,and expert users.The study offers insights for designing explanations that balance completeness and simplicity to improve trust calibration and cognitive load.The approach has implications for safe and transparent GenAI deployment and can inform both AI interface design and policy development for responsible AI use.
