Visual question answering(VQA)is a multimodal task,involving a deep understanding of the image scene and the question’s meaning and capturing the relevant correlations between both modalities to infer the appropriate...Visual question answering(VQA)is a multimodal task,involving a deep understanding of the image scene and the question’s meaning and capturing the relevant correlations between both modalities to infer the appropriate answer.In this paper,we propose a VQA system intended to answer yes/no questions about real-world images,in Arabic.To support a robust VQA system,we work in two directions:(1)Using deep neural networks to semantically represent the given image and question in a fine-grainedmanner,namely ResNet-152 and Gated Recurrent Units(GRU).(2)Studying the role of the utilizedmultimodal bilinear pooling fusion technique in the trade-o.between the model complexity and the overall model performance.Some fusion techniques could significantly increase the model complexity,which seriously limits their applicability for VQA models.So far,there is no evidence of how efficient these multimodal bilinear pooling fusion techniques are for VQA systems dedicated to yes/no questions.Hence,a comparative analysis is conducted between eight bilinear pooling fusion techniques,in terms of their ability to reduce themodel complexity and improve themodel performance in this case of VQA systems.Experiments indicate that these multimodal bilinear pooling fusion techniques have improved the VQA model’s performance,until reaching the best performance of 89.25%.Further,experiments have proven that the number of answers in the developed VQA system is a critical factor that a.ects the effectiveness of these multimodal bilinear pooling techniques in achieving their main objective of reducing the model complexity.The Multimodal Local Perception Bilinear Pooling(MLPB)technique has shown the best balance between the model complexity and its performance,for VQA systems designed to answer yes/no questions.展开更多
In Chinese question answering system, because there is more semantic relation in questions than that in query words, the precision can be improved by expanding query while using natural language questions to retrieve ...In Chinese question answering system, because there is more semantic relation in questions than that in query words, the precision can be improved by expanding query while using natural language questions to retrieve documents. This paper proposes a new approach to query expansion based on semantics and statistics Firstly automatic relevance feedback method is used to generate a candidate expansion word set. Then the expanded query words are selected from the set based on the semantic similarity and seman- tic relevancy between the candidate words and the original words. Experiments show the new approach is effective for Web retrieval and out-performs the conventional expansion approaches.展开更多
Question answering systems offer a friendly interface for human beings to interact with massive online information. It is time consuming for users to retrieve useful medical information with search engines among massi...Question answering systems offer a friendly interface for human beings to interact with massive online information. It is time consuming for users to retrieve useful medical information with search engines among massive online websites. An effort is made to build a Chinese Question Answering System in Medical Domain(CQASMD) to provide useful medical information for users. A large medical knowledge base with more than 300 thousand medical terms and their descriptions is firstly constructed to store the structured medical knowledge data, and classified with the FastText model. Furthermore, a Word2Vec model is adopted to capture the semantic meanings of words, and the questions and answers are processed with sentence embedding to capture semantic context information. Users' questions are firstly classified and processed into a sentence vector and a matching algorithm is adopted to match the most similar question. After querying the constructed medical knowledge base, the corresponding answers to previous questions are responded to users. The architecture and flowchart of CQASMD is proposed, which will play an important role in self disease diagnosis and treatment.展开更多
Aiming at the lack of professional knowledge to guide apparel recommendation,an apparel recommendation method based on image design expert knowledge has been proposed.Then,apparel recommendation knowledge graphs have ...Aiming at the lack of professional knowledge to guide apparel recommendation,an apparel recommendation method based on image design expert knowledge has been proposed.Then,apparel recommendation knowledge graphs have been created and a apparel recommendation question and answer(Q&A)system has been designed and implemented.The question templates in the apparel recommendation domain were defined,the task of recognizing the named entities of question sentences was completed by the Bi-directional encoder representations from transformer-Bi-directional long short-term memory-conditional random field(BERT-BiLSTM-CRF)model,and the question template with the highest matching degree to the user’s question was obtained by using term frequency-inverse document frequency(TF-IDF)algorithm.The corresponding cypher graph database query statement was generated to retrieve the knowledge graph for answers,and iFLYTEK’s voice application programming interface(API)was called to implement the Q&A.The experimental results have shown that the Q&A system has a high accuracy rate and application value in the field of apparel recommendations.展开更多
In this work, a best answer recommendation model is proposed for a Question Answering (QA) system. A Community Question Answering System was subsequently developed based on the model. The system applies Brouwer Fixed ...In this work, a best answer recommendation model is proposed for a Question Answering (QA) system. A Community Question Answering System was subsequently developed based on the model. The system applies Brouwer Fixed Point Theorem to prove the existence of the desired voter scoring function and Normalized Google Distance (NGD) to show closeness between words before an answer is suggested to users. Answers are ranked according to their Fixed-Point Score (FPS) for each question. Thereafter, the highest scored answer is chosen as the FPS Best Answer (BA). For each question asked by user, the system applies NGD to check if similar or related questions with the best answer had been asked and stored in the database. When similar or related questions with the best answer are not found in the database, Brouwer Fixed point is used to calculate the best answer from the pool of answers on a question then the best answer is stored in the NGD data-table for recommendation purpose. The system was implemented using PHP scripting language, MySQL for database management, JQuery, and Apache. The system was evaluated using standard metrics: Reciprocal Rank, Mean Reciprocal Rank (MRR) and Discounted Cumulative Gain (DCG). The system eliminated longer waiting time faced by askers in a community question answering system. The developed system can be used for research and learning purposes.展开更多
Automatic Question Answer System(QAS)is a kind of high-powered software system based on Internet.Its key technology is the interrelated technology based on natural language understanding,including the construction of ...Automatic Question Answer System(QAS)is a kind of high-powered software system based on Internet.Its key technology is the interrelated technology based on natural language understanding,including the construction of knowledge base and corpus,the Word Segmentation and POS Tagging of text,the Grammatical Analysis and Semantic Analysis of sentences etc.This thesis dissertated mainly the denotation of knowledge-information based on semantic network in QAS,the stochastic syntax-parse model named LSF of knowledge-information in QAS,the structure and constitution of QAS.And the LSF model's parameters were exercised,which proved that they were feasible.At the same time,through "the limited-domain QAS" which was exploited for banks by us,these technologies were proved effective and propagable.展开更多
Nowadays, the computer is increasingly popular, and college examination is developing in the direction of traditional examination means to automation and intelligence ones gradually, all these make it inevitable to co...Nowadays, the computer is increasingly popular, and college examination is developing in the direction of traditional examination means to automation and intelligence ones gradually, all these make it inevitable to construct question bank for courses, and to generate test paper using computers. This paper uses the Delphi technique, to make improvements to existing components, combining with VBA programming, and use of SQL Server to implement the question bank management and test paper auto-generation system, which could generate test paper in Word Document. A large number of tests show that the software is running stably and system features are functioning correctly on Windows 2000/XP/2003 platform with Office XP/2003 environment.展开更多
Editors Yang Wang,Xi'an Jiaotong University Dongbo Shi,Shanghai Jiaotong University Ye Sun,University College London Zhesi Shen,National Science Library,CAS Topic of the Special Issue What are the top questions to...Editors Yang Wang,Xi'an Jiaotong University Dongbo Shi,Shanghai Jiaotong University Ye Sun,University College London Zhesi Shen,National Science Library,CAS Topic of the Special Issue What are the top questions towards better science and innovation and the required data to answer these questions?展开更多
Editors Yang Wang,Xi'an Jiaotong University Dongbo Shi,Shanghai Jiaotong University Ye Sun,University College London Zhesi Shen,National Science Library,CASTopic of the Special Issue What are the top questions tow...Editors Yang Wang,Xi'an Jiaotong University Dongbo Shi,Shanghai Jiaotong University Ye Sun,University College London Zhesi Shen,National Science Library,CASTopic of the Special Issue What are the top questions towards better science and innovation and the required data to answer these questions?展开更多
Medical visual question answering(MedVQA)faces unique challenges due to the high precision required for images and the specialized nature of the questions.These challenges include insufficient feature extraction capab...Medical visual question answering(MedVQA)faces unique challenges due to the high precision required for images and the specialized nature of the questions.These challenges include insufficient feature extraction capabilities,a lack of textual priors,and incomplete information fusion and interaction.This paper proposes an enhanced bootstrapping language-image pre-training(BLIP)model for MedVQA based on multimodal feature augmentation and triple-path collaborative attention(FCA-BLIP)to address these issues.First,FCA-BLIP employs a unified bootstrap multimodal model architecture that integrates ResNet and bidirectional encoder representations from Transformer(BERT)models to enhance feature extraction capabilities.It enables a more precise analysis of the details in images and questions.Next,the pre-trained BLIP model is used to extract features from image-text sample pairs.The model can understand the semantic relationships and shared information between images and text.Finally,a novel attention structure is developed to fuse the multimodal feature vectors,thereby improving the alignment accuracy between modalities.Experimental results demonstrate that the proposed method performs well in clinical visual question-answering tasks.For the MedVQA task of staging diabetic macular edema in fundus imaging,the proposed method outperforms the existing major models in several performance metrics.展开更多
The consultation intention of emergency decision-makers in urban rail transit(URT)is input into the emergency knowledge base in the form of domain questions to obtain emergency decision support services.This approach ...The consultation intention of emergency decision-makers in urban rail transit(URT)is input into the emergency knowledge base in the form of domain questions to obtain emergency decision support services.This approach facilitates the rapid collection of complete knowledge and rules to form effective decisions.However,the current structured degree of the URT emergency knowledge base remains low,and the domain questions lack labeled datasets,resulting in a large deviation between the consultation outcomes and the intended objectives.To address this issue,this paper proposes a question intention recognition model for the URT emergency domain,leveraging knowledge graph(KG)and data enhancement technology.First,a structured storage of emergency cases and emergency plans is realized based on KG.Subsequently,a comprehensive question template is developed,and the labeled dataset of emergency domain questions in URT is generated through the KG.Lastly,data enhancement is applied by prompt learning and the NLP Chinese Data Augmentation(NLPCDA)tool,and the intention recognition model combining Generalized Auto-regression Pre-training for Language Understanding(XLNet)and Recurrent Convolutional Neural Network for Text Classification(TextRCNN)is constructed.Word embeddings are generated by XLNet,context information is further captured using Bidirectional Long Short-Term Memory Neural Network(BiLSTM),and salient features are extracted with Convolutional Neural Network(CNN).Experimental results demonstrate that the proposed model can enhance the clarity of classification and the identification of domain questions,thereby providing supportive knowledge for emergency decision-making in URT.展开更多
To improve question answering （QA） performance based on real-world web data sets,a new set of question classes and a general answer re-ranking model are defined.With pre-defined dictionary and grammatical analysis,t...To improve question answering （QA） performance based on real-world web data sets,a new set of question classes and a general answer re-ranking model are defined.With pre-defined dictionary and grammatical analysis,the question classifier draws both semantic and grammatical information into information retrieval and machine learning methods in the form of various training features,including the question word,the main verb of the question,the dependency structure,the position of the main auxiliary verb,the main noun of the question,the top hypernym of the main noun,etc.Then the QA query results are re-ranked by question class information.Experiments show that the questions in real-world web data sets can be accurately classified by the classifier,and the QA results after re-ranking can be obviously improved.It is proved that with both semantic and grammatical information,applications such as QA, built upon real-world web data sets, can be improved,thus showing better performance.展开更多
Questions which were conventionally designed to check reading comprehension can also be used to enhance understanding. Different types of questions can be designed to achieve different purposes in reading. While desig...Questions which were conventionally designed to check reading comprehension can also be used to enhance understanding. Different types of questions can be designed to achieve different purposes in reading. While designing questions, teachers should take into consideration such points as the language used, manner of presentation and types of questions etc. Moreover, once questions have been designed, it’s essential for teachers to think about the techniques for the use of these questions. If appro- priately used, questions contribute greatly to students’ understanding of what they’ re reading by helping to explore the meaning that language conveys, in addition to developing proper reading skills. Therefore, teachers should be able to teach reading with well-designed questions so that the ultimate goal of understanding the text is likely to be achieved.展开更多
文摘Visual question answering(VQA)is a multimodal task,involving a deep understanding of the image scene and the question’s meaning and capturing the relevant correlations between both modalities to infer the appropriate answer.In this paper,we propose a VQA system intended to answer yes/no questions about real-world images,in Arabic.To support a robust VQA system,we work in two directions:(1)Using deep neural networks to semantically represent the given image and question in a fine-grainedmanner,namely ResNet-152 and Gated Recurrent Units(GRU).(2)Studying the role of the utilizedmultimodal bilinear pooling fusion technique in the trade-o.between the model complexity and the overall model performance.Some fusion techniques could significantly increase the model complexity,which seriously limits their applicability for VQA models.So far,there is no evidence of how efficient these multimodal bilinear pooling fusion techniques are for VQA systems dedicated to yes/no questions.Hence,a comparative analysis is conducted between eight bilinear pooling fusion techniques,in terms of their ability to reduce themodel complexity and improve themodel performance in this case of VQA systems.Experiments indicate that these multimodal bilinear pooling fusion techniques have improved the VQA model’s performance,until reaching the best performance of 89.25%.Further,experiments have proven that the number of answers in the developed VQA system is a critical factor that a.ects the effectiveness of these multimodal bilinear pooling techniques in achieving their main objective of reducing the model complexity.The Multimodal Local Perception Bilinear Pooling(MLPB)technique has shown the best balance between the model complexity and its performance,for VQA systems designed to answer yes/no questions.
基金the Specialized Research Program Fundthe Doctoral Program of Higher Education of China (20050007023)the Natural Science Foundation of Shandong Province(Y2004G04)
文摘In Chinese question answering system, because there is more semantic relation in questions than that in query words, the precision can be improved by expanding query while using natural language questions to retrieve documents. This paper proposes a new approach to query expansion based on semantics and statistics Firstly automatic relevance feedback method is used to generate a candidate expansion word set. Then the expanded query words are selected from the set based on the semantic similarity and seman- tic relevancy between the candidate words and the original words. Experiments show the new approach is effective for Web retrieval and out-performs the conventional expansion approaches.
基金the National Natural Science Foundation of China(No.61303094)the Program of Science and Technology Commission of Shanghai Municipality(Nos.16511102400 and 16111107801)the Innovation Program of Shanghai Municipal Education Commission(No.14YZ024)
文摘Question answering systems offer a friendly interface for human beings to interact with massive online information. It is time consuming for users to retrieve useful medical information with search engines among massive online websites. An effort is made to build a Chinese Question Answering System in Medical Domain(CQASMD) to provide useful medical information for users. A large medical knowledge base with more than 300 thousand medical terms and their descriptions is firstly constructed to store the structured medical knowledge data, and classified with the FastText model. Furthermore, a Word2Vec model is adopted to capture the semantic meanings of words, and the questions and answers are processed with sentence embedding to capture semantic context information. Users' questions are firstly classified and processed into a sentence vector and a matching algorithm is adopted to match the most similar question. After querying the constructed medical knowledge base, the corresponding answers to previous questions are responded to users. The architecture and flowchart of CQASMD is proposed, which will play an important role in self disease diagnosis and treatment.
文摘Aiming at the lack of professional knowledge to guide apparel recommendation,an apparel recommendation method based on image design expert knowledge has been proposed.Then,apparel recommendation knowledge graphs have been created and a apparel recommendation question and answer(Q&A)system has been designed and implemented.The question templates in the apparel recommendation domain were defined,the task of recognizing the named entities of question sentences was completed by the Bi-directional encoder representations from transformer-Bi-directional long short-term memory-conditional random field(BERT-BiLSTM-CRF)model,and the question template with the highest matching degree to the user’s question was obtained by using term frequency-inverse document frequency(TF-IDF)algorithm.The corresponding cypher graph database query statement was generated to retrieve the knowledge graph for answers,and iFLYTEK’s voice application programming interface(API)was called to implement the Q&A.The experimental results have shown that the Q&A system has a high accuracy rate and application value in the field of apparel recommendations.
文摘In this work, a best answer recommendation model is proposed for a Question Answering (QA) system. A Community Question Answering System was subsequently developed based on the model. The system applies Brouwer Fixed Point Theorem to prove the existence of the desired voter scoring function and Normalized Google Distance (NGD) to show closeness between words before an answer is suggested to users. Answers are ranked according to their Fixed-Point Score (FPS) for each question. Thereafter, the highest scored answer is chosen as the FPS Best Answer (BA). For each question asked by user, the system applies NGD to check if similar or related questions with the best answer had been asked and stored in the database. When similar or related questions with the best answer are not found in the database, Brouwer Fixed point is used to calculate the best answer from the pool of answers on a question then the best answer is stored in the NGD data-table for recommendation purpose. The system was implemented using PHP scripting language, MySQL for database management, JQuery, and Apache. The system was evaluated using standard metrics: Reciprocal Rank, Mean Reciprocal Rank (MRR) and Discounted Cumulative Gain (DCG). The system eliminated longer waiting time faced by askers in a community question answering system. The developed system can be used for research and learning purposes.
基金Sponsored by the National Natural Science Foundation of China(Grant No.60305009)the Ph.D Degree Teacher Foundation of North China Electric Power University(Grant No.H0585).
文摘Automatic Question Answer System(QAS)is a kind of high-powered software system based on Internet.Its key technology is the interrelated technology based on natural language understanding,including the construction of knowledge base and corpus,the Word Segmentation and POS Tagging of text,the Grammatical Analysis and Semantic Analysis of sentences etc.This thesis dissertated mainly the denotation of knowledge-information based on semantic network in QAS,the stochastic syntax-parse model named LSF of knowledge-information in QAS,the structure and constitution of QAS.And the LSF model's parameters were exercised,which proved that they were feasible.At the same time,through "the limited-domain QAS" which was exploited for banks by us,these technologies were proved effective and propagable.
文摘Nowadays, the computer is increasingly popular, and college examination is developing in the direction of traditional examination means to automation and intelligence ones gradually, all these make it inevitable to construct question bank for courses, and to generate test paper using computers. This paper uses the Delphi technique, to make improvements to existing components, combining with VBA programming, and use of SQL Server to implement the question bank management and test paper auto-generation system, which could generate test paper in Word Document. A large number of tests show that the software is running stably and system features are functioning correctly on Windows 2000/XP/2003 platform with Office XP/2003 environment.
文摘Editors Yang Wang,Xi'an Jiaotong University Dongbo Shi,Shanghai Jiaotong University Ye Sun,University College London Zhesi Shen,National Science Library,CAS Topic of the Special Issue What are the top questions towards better science and innovation and the required data to answer these questions?
文摘Editors Yang Wang,Xi'an Jiaotong University Dongbo Shi,Shanghai Jiaotong University Ye Sun,University College London Zhesi Shen,National Science Library,CASTopic of the Special Issue What are the top questions towards better science and innovation and the required data to answer these questions?
基金Supported by the Program for Liaoning Excellent Talents in University(No.LR15045)the Liaoning Provincial Science and Technology Department Applied Basic Research Plan(No.101300243).
文摘Medical visual question answering(MedVQA)faces unique challenges due to the high precision required for images and the specialized nature of the questions.These challenges include insufficient feature extraction capabilities,a lack of textual priors,and incomplete information fusion and interaction.This paper proposes an enhanced bootstrapping language-image pre-training(BLIP)model for MedVQA based on multimodal feature augmentation and triple-path collaborative attention(FCA-BLIP)to address these issues.First,FCA-BLIP employs a unified bootstrap multimodal model architecture that integrates ResNet and bidirectional encoder representations from Transformer(BERT)models to enhance feature extraction capabilities.It enables a more precise analysis of the details in images and questions.Next,the pre-trained BLIP model is used to extract features from image-text sample pairs.The model can understand the semantic relationships and shared information between images and text.Finally,a novel attention structure is developed to fuse the multimodal feature vectors,thereby improving the alignment accuracy between modalities.Experimental results demonstrate that the proposed method performs well in clinical visual question-answering tasks.For the MedVQA task of staging diabetic macular edema in fundus imaging,the proposed method outperforms the existing major models in several performance metrics.
基金supported in part by the National Natural Science Foundation of China.The funding numbers 62433005,62272036,62132003,and 62173167.
文摘The consultation intention of emergency decision-makers in urban rail transit(URT)is input into the emergency knowledge base in the form of domain questions to obtain emergency decision support services.This approach facilitates the rapid collection of complete knowledge and rules to form effective decisions.However,the current structured degree of the URT emergency knowledge base remains low,and the domain questions lack labeled datasets,resulting in a large deviation between the consultation outcomes and the intended objectives.To address this issue,this paper proposes a question intention recognition model for the URT emergency domain,leveraging knowledge graph(KG)and data enhancement technology.First,a structured storage of emergency cases and emergency plans is realized based on KG.Subsequently,a comprehensive question template is developed,and the labeled dataset of emergency domain questions in URT is generated through the KG.Lastly,data enhancement is applied by prompt learning and the NLP Chinese Data Augmentation(NLPCDA)tool,and the intention recognition model combining Generalized Auto-regression Pre-training for Language Understanding(XLNet)and Recurrent Convolutional Neural Network for Text Classification(TextRCNN)is constructed.Word embeddings are generated by XLNet,context information is further captured using Bidirectional Long Short-Term Memory Neural Network(BiLSTM),and salient features are extracted with Convolutional Neural Network(CNN).Experimental results demonstrate that the proposed model can enhance the clarity of classification and the identification of domain questions,thereby providing supportive knowledge for emergency decision-making in URT.
基金Microsoft Research Asia Internet Services in Academic Research Fund(No.FY07-RES-OPP-116)the Science and Technology Development Program of Tianjin(No.06YFGZGX05900)
文摘To improve question answering （QA） performance based on real-world web data sets,a new set of question classes and a general answer re-ranking model are defined.With pre-defined dictionary and grammatical analysis,the question classifier draws both semantic and grammatical information into information retrieval and machine learning methods in the form of various training features,including the question word,the main verb of the question,the dependency structure,the position of the main auxiliary verb,the main noun of the question,the top hypernym of the main noun,etc.Then the QA query results are re-ranked by question class information.Experiments show that the questions in real-world web data sets can be accurately classified by the classifier,and the QA results after re-ranking can be obviously improved.It is proved that with both semantic and grammatical information,applications such as QA, built upon real-world web data sets, can be improved,thus showing better performance.
文摘Questions which were conventionally designed to check reading comprehension can also be used to enhance understanding. Different types of questions can be designed to achieve different purposes in reading. While designing questions, teachers should take into consideration such points as the language used, manner of presentation and types of questions etc. Moreover, once questions have been designed, it’s essential for teachers to think about the techniques for the use of these questions. If appro- priately used, questions contribute greatly to students’ understanding of what they’ re reading by helping to explore the meaning that language conveys, in addition to developing proper reading skills. Therefore, teachers should be able to teach reading with well-designed questions so that the ultimate goal of understanding the text is likely to be achieved.
