Generation of good-quality distractors is a key and time-consuming task associated withmultiple-choice questions(MCQs),one of the assessment items that have dominated the educational field for years.Recent advances in...Generation of good-quality distractors is a key and time-consuming task associated withmultiple-choice questions(MCQs),one of the assessment items that have dominated the educational field for years.Recent advances in language models and architectures present an opportunity for helping teachers to generate and update these elements to the required speed and scale of widespread increase in online education.This study focuses on a text-to-text approach for joints generation of distractors for MCQs,where the context,question and correct answer are used as input,while the set of distractors corresponds to the output,allowing the generation of three distractors in a singlemodel inference.By fine-tuning FlanT5 models and LongT5 with TGlobal attention using a RACE-based dataset,the potential of this approach is explored,demonstrating an improvement in the BLEU and ROUGE-L metrics when compared to previous works and a GPT-3.5 baseline.Additionally,BERTScore is introduced in the evaluation,showing that the fine-tuned models generate distractors semantically close to the reference,but the GPT-3.5 baseline still outperforms in this area.A tendency toward duplicating distractors is noted,although models fine-tuned with Low-Rank Adaptation(LoRA)and 4-bit quantization showcased a significant reduction in duplicated distractors.展开更多
With the increasing growth of online news,fake electronic news detection has become one of the most important paradigms of modern research.Traditional electronic news detection techniques are generally based on contex...With the increasing growth of online news,fake electronic news detection has become one of the most important paradigms of modern research.Traditional electronic news detection techniques are generally based on contextual understanding,sequential dependencies,and/or data imbalance.This makes distinction between genuine and fabricated news a challenging task.To address this problem,we propose a novel hybrid architecture,T5-SA-LSTM,which synergistically integrates the T5 Transformer for semantically rich contextual embedding with the Self-Attentionenhanced(SA)Long Short-Term Memory(LSTM).The LSTM is trained using the Adam optimizer,which provides faster and more stable convergence compared to the Stochastic Gradient Descend(SGD)and Root Mean Square Propagation(RMSProp).The WELFake and FakeNewsPrediction datasets are used,which consist of labeled news articles having fake and real news samples.Tokenization and Synthetic Minority Over-sampling Technique(SMOTE)methods are used for data preprocessing to ensure linguistic normalization and class imbalance.The incorporation of the Self-Attention(SA)mechanism enables the model to highlight critical words and phrases,thereby enhancing predictive accuracy.The proposed model is evaluated using accuracy,precision,recall(sensitivity),and F1-score as performance metrics.The model achieved 99%accuracy on the WELFake dataset and 96.5%accuracy on the FakeNewsPrediction dataset.It outperformed the competitive schemes such as T5-SA-LSTM(RMSProp),T5-SA-LSTM(SGD)and some other models.展开更多
基金supported by the Universidad de Alcalá(UAH)under Grant PIUAH21/IA-010Comunidad Autonóma de Madrid under Grant CM/JIN/2021-034.
文摘Generation of good-quality distractors is a key and time-consuming task associated withmultiple-choice questions(MCQs),one of the assessment items that have dominated the educational field for years.Recent advances in language models and architectures present an opportunity for helping teachers to generate and update these elements to the required speed and scale of widespread increase in online education.This study focuses on a text-to-text approach for joints generation of distractors for MCQs,where the context,question and correct answer are used as input,while the set of distractors corresponds to the output,allowing the generation of three distractors in a singlemodel inference.By fine-tuning FlanT5 models and LongT5 with TGlobal attention using a RACE-based dataset,the potential of this approach is explored,demonstrating an improvement in the BLEU and ROUGE-L metrics when compared to previous works and a GPT-3.5 baseline.Additionally,BERTScore is introduced in the evaluation,showing that the fine-tuned models generate distractors semantically close to the reference,but the GPT-3.5 baseline still outperforms in this area.A tendency toward duplicating distractors is noted,although models fine-tuned with Low-Rank Adaptation(LoRA)and 4-bit quantization showcased a significant reduction in duplicated distractors.
基金supported by Princess Nourah bint Abdulrahman University Researchers Supporting Project number(PNURSP2025R195)Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia.
文摘With the increasing growth of online news,fake electronic news detection has become one of the most important paradigms of modern research.Traditional electronic news detection techniques are generally based on contextual understanding,sequential dependencies,and/or data imbalance.This makes distinction between genuine and fabricated news a challenging task.To address this problem,we propose a novel hybrid architecture,T5-SA-LSTM,which synergistically integrates the T5 Transformer for semantically rich contextual embedding with the Self-Attentionenhanced(SA)Long Short-Term Memory(LSTM).The LSTM is trained using the Adam optimizer,which provides faster and more stable convergence compared to the Stochastic Gradient Descend(SGD)and Root Mean Square Propagation(RMSProp).The WELFake and FakeNewsPrediction datasets are used,which consist of labeled news articles having fake and real news samples.Tokenization and Synthetic Minority Over-sampling Technique(SMOTE)methods are used for data preprocessing to ensure linguistic normalization and class imbalance.The incorporation of the Self-Attention(SA)mechanism enables the model to highlight critical words and phrases,thereby enhancing predictive accuracy.The proposed model is evaluated using accuracy,precision,recall(sensitivity),and F1-score as performance metrics.The model achieved 99%accuracy on the WELFake dataset and 96.5%accuracy on the FakeNewsPrediction dataset.It outperformed the competitive schemes such as T5-SA-LSTM(RMSProp),T5-SA-LSTM(SGD)and some other models.
