This paper proposed a new method of semi-automatic extraction for semantic structures from unlabelled corpora in specific domains. The approach is statistical in nature. The extracted structures can be used for shallo...This paper proposed a new method of semi-automatic extraction for semantic structures from unlabelled corpora in specific domains. The approach is statistical in nature. The extracted structures can be used for shallow parsing and semantic labeling. By iteratively extracting new words and clustering words, we get an inital semantic lexicon that groups words of the same semantic meaning together as a class. After that, a bootstrapping algorithm is adopted to extract semantic structures. Then the semantic structures are used to extract new展开更多
Introduction:Traditional dietary surveys are timeconsuming,and manual recording may lead to omissions.Improvement during data collection is essential to enhance accuracy of nutritional surveys.In recent years,large la...Introduction:Traditional dietary surveys are timeconsuming,and manual recording may lead to omissions.Improvement during data collection is essential to enhance accuracy of nutritional surveys.In recent years,large language models(LLMs)have been rapidly developed,which can provide text-processing functions and assist investigators in conducting dietary surveys.Methods:Thirty-eight participants from 15 families in the Huangpu and Jiading districts of Shanghai were selected.A standardized 24-hour dietary recall protocol was conducted using an intelligent recording pen that simultaneously captured audio data.These recordings were then transcribed into text.After preprocessing,we used GLM-4 for prompt engineering and chain-of-thought for collaborative reasoning,output structured data,and analyzed its integrity and consistency.Model performance was evaluated using precision and F1 scores.Results:The overall integrity rate of the LLMbased structured data reached 92.5%,and the overall consistency rate compared with manual recording was 86%.The LLM can accurately and completely recognize the names of ingredients and dining and production locations during the transcription.The LLM achieved 94%precision and an F1 score of 89.7%for the full dataset.Conclusion:LLM-based text recognition and structured data extraction can serve as effective auxiliary tools to improve efficiency and accuracy in traditional dietary surveys.With the rapid advancement of artificial intelligence,more accurate and efficient auxiliary tools can be developed for more precise and efficient data collection in nutrition research.展开更多
A log is a text message that is generated in various services,frameworks,and programs.The majority of log data mining tasks rely on log parsing as the first step,which transforms raw logs into formatted log templates....A log is a text message that is generated in various services,frameworks,and programs.The majority of log data mining tasks rely on log parsing as the first step,which transforms raw logs into formatted log templates.Existing log parsing approaches often fail to effectively handle the trade-off between parsing quality and performance.In view of this,in this paper,we present Multi-Layer Parser(ML-Parser),an online log parser that runs in a streaming manner.Specifically,we present a multi-layer structure in log parsing to strike a balance between efficiency and effectiveness.Coarse-grained tokenization and a fast similarity measure are applied for efficiency while fine-grained tokenization and an accurate similarity measure are used for effectiveness.In experiments,we compare ML-Parser with two existing online log parsing approaches,Drain and Spell,on ten real-world datasets,five labeled and five unlabeled.On the five labeled datasets,we use the proportion of correctly parsed logs to measure the accuracy,and ML-Parser achieves the highest accuracy on four datasets.On the whole ten datasets,we use Loss metric to measure the parsing quality.ML-Parse achieves the highest quality on seven out of the ten datasets while maintaining relatively high efficiency.展开更多
文摘This paper proposed a new method of semi-automatic extraction for semantic structures from unlabelled corpora in specific domains. The approach is statistical in nature. The extracted structures can be used for shallow parsing and semantic labeling. By iteratively extracting new words and clustering words, we get an inital semantic lexicon that groups words of the same semantic meaning together as a class. After that, a bootstrapping algorithm is adopted to extract semantic structures. Then the semantic structures are used to extract new
基金Supported by the Ministry of Finance of the People’s Republic of China from 2022 to 2024(grant number 102393220020070000016).
文摘Introduction:Traditional dietary surveys are timeconsuming,and manual recording may lead to omissions.Improvement during data collection is essential to enhance accuracy of nutritional surveys.In recent years,large language models(LLMs)have been rapidly developed,which can provide text-processing functions and assist investigators in conducting dietary surveys.Methods:Thirty-eight participants from 15 families in the Huangpu and Jiading districts of Shanghai were selected.A standardized 24-hour dietary recall protocol was conducted using an intelligent recording pen that simultaneously captured audio data.These recordings were then transcribed into text.After preprocessing,we used GLM-4 for prompt engineering and chain-of-thought for collaborative reasoning,output structured data,and analyzed its integrity and consistency.Model performance was evaluated using precision and F1 scores.Results:The overall integrity rate of the LLMbased structured data reached 92.5%,and the overall consistency rate compared with manual recording was 86%.The LLM can accurately and completely recognize the names of ingredients and dining and production locations during the transcription.The LLM achieved 94%precision and an F1 score of 89.7%for the full dataset.Conclusion:LLM-based text recognition and structured data extraction can serve as effective auxiliary tools to improve efficiency and accuracy in traditional dietary surveys.With the rapid advancement of artificial intelligence,more accurate and efficient auxiliary tools can be developed for more precise and efficient data collection in nutrition research.
基金the National Natural Science Foundation of China under Grant No.61672163.
文摘A log is a text message that is generated in various services,frameworks,and programs.The majority of log data mining tasks rely on log parsing as the first step,which transforms raw logs into formatted log templates.Existing log parsing approaches often fail to effectively handle the trade-off between parsing quality and performance.In view of this,in this paper,we present Multi-Layer Parser(ML-Parser),an online log parser that runs in a streaming manner.Specifically,we present a multi-layer structure in log parsing to strike a balance between efficiency and effectiveness.Coarse-grained tokenization and a fast similarity measure are applied for efficiency while fine-grained tokenization and an accurate similarity measure are used for effectiveness.In experiments,we compare ML-Parser with two existing online log parsing approaches,Drain and Spell,on ten real-world datasets,five labeled and five unlabeled.On the five labeled datasets,we use the proportion of correctly parsed logs to measure the accuracy,and ML-Parser achieves the highest accuracy on four datasets.On the whole ten datasets,we use Loss metric to measure the parsing quality.ML-Parse achieves the highest quality on seven out of the ten datasets while maintaining relatively high efficiency.
