Aiming at the problems of incomplete characterization of text relations,poor guidance of potential representations,and low quality of model generation in the field of controllable long text generation,this paper propo...Aiming at the problems of incomplete characterization of text relations,poor guidance of potential representations,and low quality of model generation in the field of controllable long text generation,this paper proposes a new GSPT-CVAE model(Graph Structured Processing,Single Vector,and Potential Attention Com-puting Transformer-Based Conditioned Variational Autoencoder model).The model obtains a more comprehensive representation of textual relations by graph-structured processing of the input text,and at the same time obtains a single vector representation by weighted merging of the vector sequences after graph-structured processing to get an effective potential representation.In the process of potential representation guiding text generation,the model adopts a combination of traditional embedding and potential attention calculation to give full play to the guiding role of potential representation for generating text,to improve the controllability and effectiveness of text generation.The experimental results show that the model has excellent representation learning ability and can learn rich and useful textual relationship representations.The model also achieves satisfactory results in the effectiveness and controllability of text generation and can generate long texts that match the given constraints.The ROUGE-1 F1 score of this model is 0.243,the ROUGE-2 F1 score is 0.041,the ROUGE-L F1 score is 0.22,and the PPL-Word score is 34.303,which gives the GSPT-CVAE model a certain advantage over the baseline model.Meanwhile,this paper compares this model with the state-of-the-art generative models T5,GPT-4,Llama2,and so on,and the experimental results show that the GSPT-CVAE model has a certain competitiveness.展开更多
Generation-based linguistic steganography is a popular research area of information hiding.The text generative steganographic method based on conditional probability coding is the direction that researchers have recen...Generation-based linguistic steganography is a popular research area of information hiding.The text generative steganographic method based on conditional probability coding is the direction that researchers have recently paid attention to.However,in the course of our experiment,we found that the secret information hiding in the text tends to destroy the statistical distribution characteristics of the original text,which indicates that this method has the problem of the obvious reduction of text quality when the embedding rate increases,and that the topic of generated texts is uncontrollable,so there is still room for improvement in concealment.In this paper,we propose a topic-controlled steganography method which is guided by graph-to-text generation.The proposed model can automatically generate steganographic texts carrying secret messages from knowledge graphs,and the topic of the generated texts is controllable.We also provide a graph path coding method with corresponding detailed algorithms for graph-to-text generation.Different from traditional linguistic steganography methods,we encode the secret information during graph path coding rather than using conditional probability.We test our method in different aspects and compare it with other text generative steganographic methods.The experimental results show that the model proposed in this paper can effectively improve the quality of the generated text and significantly improve the concealment of steganographic text.展开更多
Recently,generative artificial intelligence(GenAI)has developed into a new form of technology that can create copy,image,audio,and video content and adapt it to individual preferences on every channel and moment autom...Recently,generative artificial intelligence(GenAI)has developed into a new form of technology that can create copy,image,audio,and video content and adapt it to individual preferences on every channel and moment automatically.But most fail at proof-of-concept,as the pipelines needed to govern data,generate it controllably,deliver it,and do causal evaluation are absent or poorly aligned.This paper puts forward a practical end-to-end framework concerning personalized advertising driven by GenAI,which combines representation learning,constrained generation,and experimentation into a single operating cycle.First,we pick a modular architecture:profiles and contexts go into controllable large language and diffusion models that yield brand-safe assets under deterministic conditioning,which are chosen via a contextual bandit and vetted by policy and equality guardrails.Second,we give a measurement stack going from straightforward A/B/n tests to doubly-robust uplift modeling,making it possible to find out diverse treatment effects that are good to use in business metrics(incremental conversions and profit).Third,we operationalize latency budgets,humans in the loop,red teams,safety filters,and post-deployment monitoring with clear escalation paths.We focus throughout the paper on reproducibility,privacy(consent,privacy,differential privacy,on-device inference),and on GDPR/CCPA-like governance specifications.We end on our actionable blueprint,algorithmic choices,sample prompts,KPIs,and step-wise rollout to achieve trustworthy performance upgrades without putting creative quality,fairness,or compliance to the test.展开更多
Purpose:A text generation based multidisciplinary problem identification method is proposed,which does not rely on a large amount of data annotation.Design/methodology/approach:The proposed method first identifies the...Purpose:A text generation based multidisciplinary problem identification method is proposed,which does not rely on a large amount of data annotation.Design/methodology/approach:The proposed method first identifies the research objective types and disciplinary labels of papers using a text classification technique;second,it generates abstractive titles for each paper based on abstract and research objective types using a generative pre-trained language model;third,it extracts problem phrases from generated titles according to regular expression rules;fourth,it creates problem relation networks and identifies the same problems by exploiting a weighted community detection algorithm;finally,it identifies multidisciplinary problems based on the disciplinary labels of papers.Findings:Experiments in the“Carbon Peaking and Carbon Neutrality”field show that the proposed method can effectively identify multidisciplinary research problems.The disciplinary distribution of the identified problems is consistent with our understanding of multidisciplinary collaboration in the field.Research limitations:It is necessary to use the proposed method in other multidisciplinary fields to validate its effectiveness.Practical implications:Multidisciplinary problem identification helps to gather multidisciplinary forces to solve complex real-world problems for the governments,fund valuable multidisciplinary problems for research management authorities,and borrow ideas from other disciplines for researchers.Originality/value:This approach proposes a novel multidisciplinary problem identification method based on text generation,which identifies multidisciplinary problems based on generative abstractive titles of papers without data annotation required by standard sequence labeling techniques.展开更多
Currently,the amount of sports news is increasing,given the number of sports available.As a result,manually writing sports news requires high labor costs to achieve the intended efficiency.Therefore,it is necessary to...Currently,the amount of sports news is increasing,given the number of sports available.As a result,manually writing sports news requires high labor costs to achieve the intended efficiency.Therefore,it is necessary to develop the automatic generation of sports news.Most available news gen-eration methods mainly rely on real-time commentary sentences,which have the following limitations:(1)unable to select suitable commentary sentences for news generation,and(2)the generated sports news could not accurately describe game events.Therefore,this study proposes a sports news generation with text-editing model(SNG-TE)is proposed to generate sports news,which includes selector and rewriter modules.Within the study context,a weight adjustment mechanism in the selector module is designed to improve the hit rate of important sentences.Furthermore,the text-editing model is introduced in the rewriter module to ensure that the generated news sentences can cor-rectly describe the game events.The annotation and generation experiments are designed to evaluate the developed model.The study results have shown that in the annotation experiment,the accuracy of the sentence annotated by the selector increased by about 8%compared with other methods.Moreover,in the generation experiment,the sports news generated by the rewriter achieved a 49.66 ROUGE-1 score and 21.47 ROUGE-2,both of which are better than the available models.Additionally,the proposed model saved about 15 times the consumption of time.Hence,the proposed model provides better performance in both accuracy and efficiency,which is very suitable for the automatic generation of sports news.展开更多
Digitization,informatization,and Internet penetration have led to a significant rise in cross-border e-commerce(CBEC),attracting considerable interest from academia,government,and industry.This study employed a novel ...Digitization,informatization,and Internet penetration have led to a significant rise in cross-border e-commerce(CBEC),attracting considerable interest from academia,government,and industry.This study employed a novel method combining automatic text generation technology and traditional bibliometric analysis to summarize and categorize the research on CBEC evolution from 2000 to 2021.Articles were selected and examined with a focus on four dimensions:customer,risk,supply chain,and platform.Contradictions in these dimensions were found to result in two major obstacles to CBEC development,namely,dataset sharing and platform scalability.These obstacles prevent research on cross-border platforms from moving beyond theory-based studies.Further research needs to examine how soft computing can be used to accelerate and remodel the global trade ecosystem.展开更多
Generating diverse and factual text is challenging and is receiving increasing attention.By sampling from the latent space,variational autoencoder-based models have recently enhanced the diversity of generated text.Ho...Generating diverse and factual text is challenging and is receiving increasing attention.By sampling from the latent space,variational autoencoder-based models have recently enhanced the diversity of generated text.However,existing research predominantly depends on summarizationmodels to offer paragraph-level semantic information for enhancing factual correctness.The challenge lies in effectively generating factual text using sentence-level variational autoencoder-based models.In this paper,a novel model called fact-aware conditional variational autoencoder is proposed to balance the factual correctness and diversity of generated text.Specifically,our model encodes the input sentences and uses them as facts to build a conditional variational autoencoder network.By training a conditional variational autoencoder network,the model is enabled to generate text based on input facts.Building upon this foundation,the input text is passed to the discriminator along with the generated text.By employing adversarial training,the model is encouraged to generate text that is indistinguishable to the discriminator,thereby enhancing the quality of the generated text.To further improve the factual correctness,inspired by the natural language inference system,the entailment recognition task is introduced to be trained together with the discriminator via multi-task learning.Moreover,based on the entailment recognition results,a penalty term is further proposed to reconstruct the loss of our model,forcing the generator to generate text consistent with the facts.Experimental results demonstrate that compared with competitivemodels,ourmodel has achieved substantial improvements in both the quality and factual correctness of the text,despite only sacrificing a small amount of diversity.Furthermore,when considering a comprehensive evaluation of diversity and quality metrics,our model has also demonstrated the best performance.展开更多
With the wider growth of web-based documents,the necessity of automatic document clustering and text summarization is increased.Here,document summarization that is extracting the essential task with appropriate inform...With the wider growth of web-based documents,the necessity of automatic document clustering and text summarization is increased.Here,document summarization that is extracting the essential task with appropriate information,removal of unnecessary data and providing the data in a cohesive and coherent manner is determined to be a most confronting task.In this research,a novel intelligent model for document clustering is designed with graph model and Fuzzy based association rule generation(gFAR).Initially,the graph model is used to map the relationship among the data(multi-source)followed by the establishment of document clustering with the generation of association rule using the fuzzy concept.This method shows benefit in redundancy elimination by mapping the relevant document using graph model and reduces the time consumption and improves the accuracy using the association rule generation with fuzzy.This framework is provided in an interpretable way for document clustering.It iteratively reduces the error rate during relationship mapping among the data(clusters)with the assistance of weighted document content.Also,this model represents the significance of data features with class discrimination.It is also helpful in measuring the significance of the features during the data clustering process.The simulation is done with MATLAB 2016b environment and evaluated with the empirical standards like Relative Risk Patterns(RRP),ROUGE score,and Discrimination Information Measure(DMI)respectively.Here,DailyMail and DUC 2004 dataset is used to extract the empirical results.The proposed gFAR model gives better trade-off while compared with various prevailing approaches.展开更多
To address the difficulty of training high-quality models in some specific domains due to the lack of fine-grained annotation resources, we propose in this paper a knowledge-integrated cross-domain data generation met...To address the difficulty of training high-quality models in some specific domains due to the lack of fine-grained annotation resources, we propose in this paper a knowledge-integrated cross-domain data generation method for unsupervised domain adaptation tasks. Specifically, we extract domain features, lexical and syntactic knowledge from source-domain and target-domain data, and use a masking model with an extended masking strategy and a re-masking strategy to obtain domain-specific data that remove domain-specific features. Finally, we improve the sequence generation model BART and use it to generate high-quality target domain data for the task of aspect and opinion co-extraction from the target domain. Experiments were performed on three conventional English datasets from different domains, and our method generates more accurate and diverse target domain data with the best results compared to previous methods.展开更多
In this study,we explore the potential of Multiway Transformers for text-to-image generation to achieve performance improvements through a concise and efficient decoupled model design and the inference efficiency prov...In this study,we explore the potential of Multiway Transformers for text-to-image generation to achieve performance improvements through a concise and efficient decoupled model design and the inference efficiency provided by bidirectional encoding.We propose a method for improving the image tokenizer using pretrained Vision Transformers.Next,we employ bidirectional Multiway Transformers to restore the masked visual tokens combined with the unmasked text tokens.On the MS-COCO benchmark,our Multiway Transformers outperform vanilla Transformers,achieving superior FID scores and confirming the efficacy of the modality-specific parameter computation design.Ablation studies reveal that the fusion of visual and text tokens in bidirectional encoding contributes to improved model performance.Additionally,our proposed tokenizer outperforms VQGAN in image reconstruction quality and enhances the text-to-image generation results.By incorporating the additional CC-3M dataset for intermediate finetuning on our model with 688M parameters,we achieve competitive results with a finetuned FID score of 4.98 on MS-COCO.展开更多
As an important subject of natural language generation,Controllable Text Generation(CTG)focuses on integrating additional constraints and controls while generating texts and has attracted a lot of attention.Existing c...As an important subject of natural language generation,Controllable Text Generation(CTG)focuses on integrating additional constraints and controls while generating texts and has attracted a lot of attention.Existing controllable text generation approaches mainly capture the statistical association implied within training texts,but generated texts lack causality consideration.This paper intends to review recent CTG approaches from a causal perspective.Firstly,according to previous research on basic types of CTG models,it is discovered that their essence is to obtain the association,and then four kinds of challenges caused by absence of causality are introduced.Next,this paper reviews the improvements to address these challenges from four aspects,namely representation disentanglement,causal inference,knowledge enhancement and multi-aspect CTG respectively.Additionally,this paper inspects existing evaluations of CTG,especially evaluations for causality of CTG.Finally,this review discusses some future research directions for the causality improvement of CTG and makes a conclusion.展开更多
Steganography based on generative adversarial networks(GANs)has become a hot topic among researchers.Due to GANs being unsuitable for text fields with discrete characteristics,researchers have proposed GANbased stegan...Steganography based on generative adversarial networks(GANs)has become a hot topic among researchers.Due to GANs being unsuitable for text fields with discrete characteristics,researchers have proposed GANbased steganography methods that are less dependent on text.In this paper,we propose a new method of generative lyrics steganography based on GANs,called GAN-GLS.The proposed method uses the GAN model and the largescale lyrics corpus to construct and train a lyrics generator.In this method,the GAN uses a previously generated line of a lyric as the input sentence in order to generate the next line of the lyric.Using a strategy based on the penalty mechanism in training,the GAN model generates non-repetitive and diverse lyrics.The secret information is then processed according to the data characteristics of the generated lyrics in order to hide information.Unlike other text generation-based linguistic steganographic methods,our method changes the way that multiple generated candidate items are selected as the candidate groups in order to encode the conditional probability distribution.The experimental results demonstrate that our method can generate highquality lyrics as stego-texts.Moreover,compared with other similar methods,the proposed method achieves good performance in terms of imperceptibility,embedding rate,effectiveness,extraction success rate and security.展开更多
In the realm of Multi-Label Text Classification(MLTC),the dual challenges of extracting rich semantic features from text and discerning inter-label relationships have spurred innovative approaches.Many studies in sema...In the realm of Multi-Label Text Classification(MLTC),the dual challenges of extracting rich semantic features from text and discerning inter-label relationships have spurred innovative approaches.Many studies in semantic feature extraction have turned to external knowledge to augment the model’s grasp of textual content,often overlooking intrinsic textual cues such as label statistical features.In contrast,these endogenous insights naturally align with the classification task.In our paper,to complement this focus on intrinsic knowledge,we introduce a novel Gate-Attention mechanism.This mechanism adeptly integrates statistical features from the text itself into the semantic fabric,enhancing the model’s capacity to understand and represent the data.Additionally,to address the intricate task of mining label correlations,we propose a Dual-end enhancement mechanism.This mechanism effectively mitigates the challenges of information loss and erroneous transmission inherent in traditional long short term memory propagation.We conducted an extensive battery of experiments on the AAPD and RCV1-2 datasets.These experiments serve the dual purpose of confirming the efficacy of both the Gate-Attention mechanism and the Dual-end enhancement mechanism.Our final model unequivocally outperforms the baseline model,attesting to its robustness.These findings emphatically underscore the imperativeness of taking into account not just external knowledge but also the inherent intricacies of textual data when crafting potent MLTC models.展开更多
In recent years,deep neural network has achieved great success in solving many natural language processing tasks.Particularly,substantial progress has been made on neural text generation,which takes the linguistic and...In recent years,deep neural network has achieved great success in solving many natural language processing tasks.Particularly,substantial progress has been made on neural text generation,which takes the linguistic and non-linguistic input,and generates natural language text.This survey aims to provide an up-to-date synthesis of core tasks in neural text generation and the architectures adopted to handle these tasks,and draw attention to the challenges in neural text generation.We first outline the mainstream neural text generation frameworks,and then introduce datasets,advanced models and challenges of four core text generation tasks in detail,including AMR-to-text generation,data-to-text generation,and two text-to-text generation tasks(i.e.,text summarization and paraphrase generation).Finally,we present future research directions for neural text generation.This survey can be used as a guide and reference for researchers and practitioners in this area.展开更多
Text generation is an essential research area in artificial intelligence(AI)technology and natural language processing and provides key technical support for the rapid development of AI-generated content(AIGC).It is b...Text generation is an essential research area in artificial intelligence(AI)technology and natural language processing and provides key technical support for the rapid development of AI-generated content(AIGC).It is based on technologies such as natural language processing,machine learning,and deep learning,which enable learning language rules through training models to automatically generate text that meets grammatical and semantic requirements.In this paper,we sort and systematically summarize the main research progress in text generation and review recent text generation papers,focusing on presenting a detailed understanding of the technical models.In addition,several typical text generation application systems are presented.Finally,we address some challenges and future directions in AI text generation.We conclude that improving the quality,quantity,interactivity,and adaptability of generated text can help fundamentally advance AI text generation development.展开更多
Algorithms for numeric data classification have been applied for text classification. Usually the vector space model is used to represent text collections. The characteristics of this representation such as sparsity a...Algorithms for numeric data classification have been applied for text classification. Usually the vector space model is used to represent text collections. The characteristics of this representation such as sparsity and high dimensionality sometimes impair the quality of general-purpose classifiers. Networks can be used to represent text collections, avoiding the high sparsity and allowing to model relationships among different objects that compose a text collection. Such network- based representations can improve the quality of the classification results. One of the simplest ways to represent textual collections by a network is through a bipartite heterogeneous network, which is composed of objects that represent the documents connected to objects that represent the terms. Heterogeneous bipartite networks do not require computation of similarities or relations among the objects and can be used to model any type of text collection. Due to the advantages of representing text collections through bipartite heterogeneous networks, in this article we present a text classifier which builds a classification model using the structure of a bipartite heterogeneous network. Such an algorithm, referred to as IMBHN （Inductive Model Based on Bipartite Heterogeneous Network）, induces a classification model assigning weights to objects that represent the terms for each class of the text collection. An empirical evaluation using a large amount of text collections from different domains shows that the proposed IMBHN algorithm produces significantly better results than k-NN, C4.5, SVM, and Naive Bayes algorithms.展开更多
Chinese medicine(CM)diagnosis intellectualization is one of the hotspots in the research of CM modernization.The traditional CM intelligent diagnosis models transform the CM diagnosis issues into classification issues...Chinese medicine(CM)diagnosis intellectualization is one of the hotspots in the research of CM modernization.The traditional CM intelligent diagnosis models transform the CM diagnosis issues into classification issues,however,it is difficult to solve the problems such as excessive or similar categories.With the development of natural language processing techniques,text generation technique has become increasingly mature.In this study,we aimed to establish the CM diagnosis generation model by transforming the CM diagnosis issues into text generation issues.The semantic context characteristic learning capacity was enhanced referring to Bidirectional Long Short-Term Memory(BILSTM)with Transformer as the backbone network.Meanwhile,the CM diagnosis generation model Knowledge Graph Enhanced Transformer(KGET)was established by introducing the knowledge in medical field to enhance the inferential capability.The KGET model was established based on 566 CM case texts,and was compared with the classic text generation models including Long Short-Term Memory sequence-to-sequence(LSTM-seq2seq),Bidirectional and Auto-Regression Transformer(BART),and Chinese Pre-trained Unbalanced Transformer(CPT),so as to analyze the model manifestations.Finally,the ablation experiments were performed to explore the influence of the optimized part on the KGET model.The results of Bilingual Evaluation Understudy(BLEU),Recall-Oriented Understudy for Gisting Evaluation 1(ROUGE1),ROUGE2 and Edit distance of KGET model were 45.85,73.93,54.59 and 7.12,respectively in this study.Compared with LSTM-seq2seq,BART and CPT models,the KGET model was higher in BLEU,ROUGE1 and ROUGE2 by 6.00–17.09,1.65–9.39 and 0.51–17.62,respectively,and lower in Edit distance by 0.47–3.21.The ablation experiment results revealed that introduction of BILSTM model and prior knowledge could significantly increase the model performance.Additionally,the manual assessment indicated that the CM diagnosis results of the KGET model used in this study were highly consistent with the practical diagnosis results.In conclusion,text generation technology can be effectively applied to CM diagnostic modeling.It can effectively avoid the problem of poor diagnostic performance caused by excessive and similar categories in traditional CM diagnostic classification models.CM diagnostic text generation technology has broad application prospects in the future.展开更多
The sources of meanings of literary works can be reviewed from various angles, one of which should be hermeneutics. Since the mid-20~ century, Western hermeneutics developed two important types of theories： one was t...The sources of meanings of literary works can be reviewed from various angles, one of which should be hermeneutics. Since the mid-20~ century, Western hermeneutics developed two important types of theories： one was the ontological hermeneutics from Martin Heidegger to Hans-Georg Gadamer, orientated towards the reader-centered theory in terms of the view of meanings; the other was the ＂hermeneutics as the general methodology of the Geisteswissenschaften＂ represented by Italian philosopher Emilio Betti, which affirmed that the author （subject） was one of the important sources of meanings of works. Due to intricate reasons, the latter exerted less influence, and Gadamer＇s ontological hermeneutics firmly stuck to a mainstay position in European and American academia. Since the 1990s, the circle of literature in China has also been greatly influenced by the hermeneutic theories from Heidegger and Gadamer to reception aesthetics, and Betti＇s impact almost can be neglected. Consequently, the view of meanings of literary works from the perspective of the reader-centered theory has been widely accepted, and the role of the author to endow initial meanings to his works was belittled and even denied. Since Zhang Jiang advanced the theory of ＂imposed interpretation＂ in 2014, academia has begun to reflect upon the one-sidedness of the reader-centered theory （relativism and subjectivism）, and pick up the significance, which cannot be denied, of the author＇s meaning in the generation of meanings of literary works. In reference to Betti＇s hermeneutic train of thought and methodology, this essay, based on the practice of literary creation, concludes that the meaning of a literary work is created by the author and readers together in their interaction, and constantly generated in the dynamic process of the three factors of author, literary texts and readers, rather than by the author alone, or by readers solely.展开更多
Hierarchical multi-granularity image classification is a challenging task that aims to tag each given image with multiple granularity labels simultaneously.Existing methods tend to overlook that different image region...Hierarchical multi-granularity image classification is a challenging task that aims to tag each given image with multiple granularity labels simultaneously.Existing methods tend to overlook that different image regions contribute differently to label prediction at different granularities,and also insufficiently consider relationships between the hierarchical multi-granularity labels.We introduce a sequence-to-sequence mechanism to overcome these two problems and propose a multi-granularity sequence generation(MGSG)approach for the hierarchical multi-granularity image classification task.Specifically,we introduce a transformer architecture to encode the image into visual representation sequences.Next,we traverse the taxonomic tree and organize the multi-granularity labels into sequences,and vectorize them and add positional information.The proposed multi-granularity sequence generation method builds a decoder that takes visual representation sequences and semantic label embedding as inputs,and outputs the predicted multi-granularity label sequence.The decoder models dependencies and correlations between multi-granularity labels through a masked multi-head self-attention mechanism,and relates visual information to the semantic label information through a crossmodality attention mechanism.In this way,the proposed method preserves the relationships between labels at different granularity levels and takes into account the influence of different image regions on labels with different granularities.Evaluations on six public benchmarks qualitatively and quantitatively demonstrate the advantages of the proposed method.Our project is available at https://github.com/liuxindazz/mgs.展开更多
Aiming at complex and changeable factors such as speech theme and environment,which make it difficult for a speaker to prepare the speech text in a short time,this paper proposes a speech generation and demonstration s...Aiming at complex and changeable factors such as speech theme and environment,which make it difficult for a speaker to prepare the speech text in a short time,this paper proposes a speech generation and demonstration system based on deep learning.This system is based on the Deep Learning Development Framework(PyTorch),trained through the theory of GPT-2 and the open source pretrained model,to generate multiple speeches according to the topics given by users,and the system generates thefinal speech and corresponding voice demon-stration audio through text modification,speech synthesis and other technologies to help users quickly obtain the target document and audio.Experiments show that the text generated by this model is smooth and easy to use,which helps shorten the preparation time of speakers and improves the confidence of the impromptu speaker.In addition,the paper explores the application prospects of text generation and has certain reference value.展开更多
文摘Aiming at the problems of incomplete characterization of text relations,poor guidance of potential representations,and low quality of model generation in the field of controllable long text generation,this paper proposes a new GSPT-CVAE model(Graph Structured Processing,Single Vector,and Potential Attention Com-puting Transformer-Based Conditioned Variational Autoencoder model).The model obtains a more comprehensive representation of textual relations by graph-structured processing of the input text,and at the same time obtains a single vector representation by weighted merging of the vector sequences after graph-structured processing to get an effective potential representation.In the process of potential representation guiding text generation,the model adopts a combination of traditional embedding and potential attention calculation to give full play to the guiding role of potential representation for generating text,to improve the controllability and effectiveness of text generation.The experimental results show that the model has excellent representation learning ability and can learn rich and useful textual relationship representations.The model also achieves satisfactory results in the effectiveness and controllability of text generation and can generate long texts that match the given constraints.The ROUGE-1 F1 score of this model is 0.243,the ROUGE-2 F1 score is 0.041,the ROUGE-L F1 score is 0.22,and the PPL-Word score is 34.303,which gives the GSPT-CVAE model a certain advantage over the baseline model.Meanwhile,this paper compares this model with the state-of-the-art generative models T5,GPT-4,Llama2,and so on,and the experimental results show that the GSPT-CVAE model has a certain competitiveness.
基金supported in part by the National Natural Science Foundation of China [62102136]the 2020 Opening Fund for Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering [2020SDSJ06]the Construction Fund for Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering [2019ZYYD007].
文摘Generation-based linguistic steganography is a popular research area of information hiding.The text generative steganographic method based on conditional probability coding is the direction that researchers have recently paid attention to.However,in the course of our experiment,we found that the secret information hiding in the text tends to destroy the statistical distribution characteristics of the original text,which indicates that this method has the problem of the obvious reduction of text quality when the embedding rate increases,and that the topic of generated texts is uncontrollable,so there is still room for improvement in concealment.In this paper,we propose a topic-controlled steganography method which is guided by graph-to-text generation.The proposed model can automatically generate steganographic texts carrying secret messages from knowledge graphs,and the topic of the generated texts is controllable.We also provide a graph path coding method with corresponding detailed algorithms for graph-to-text generation.Different from traditional linguistic steganography methods,we encode the secret information during graph path coding rather than using conditional probability.We test our method in different aspects and compare it with other text generative steganographic methods.The experimental results show that the model proposed in this paper can effectively improve the quality of the generated text and significantly improve the concealment of steganographic text.
文摘Recently,generative artificial intelligence(GenAI)has developed into a new form of technology that can create copy,image,audio,and video content and adapt it to individual preferences on every channel and moment automatically.But most fail at proof-of-concept,as the pipelines needed to govern data,generate it controllably,deliver it,and do causal evaluation are absent or poorly aligned.This paper puts forward a practical end-to-end framework concerning personalized advertising driven by GenAI,which combines representation learning,constrained generation,and experimentation into a single operating cycle.First,we pick a modular architecture:profiles and contexts go into controllable large language and diffusion models that yield brand-safe assets under deterministic conditioning,which are chosen via a contextual bandit and vetted by policy and equality guardrails.Second,we give a measurement stack going from straightforward A/B/n tests to doubly-robust uplift modeling,making it possible to find out diverse treatment effects that are good to use in business metrics(incremental conversions and profit).Third,we operationalize latency budgets,humans in the loop,red teams,safety filters,and post-deployment monitoring with clear escalation paths.We focus throughout the paper on reproducibility,privacy(consent,privacy,differential privacy,on-device inference),and on GDPR/CCPA-like governance specifications.We end on our actionable blueprint,algorithmic choices,sample prompts,KPIs,and step-wise rollout to achieve trustworthy performance upgrades without putting creative quality,fairness,or compliance to the test.
基金supported by the General Projects of ISTIC Innovation Foundation“Problem innovation solution mining based on text generation model”(MS2024-03).
文摘Purpose:A text generation based multidisciplinary problem identification method is proposed,which does not rely on a large amount of data annotation.Design/methodology/approach:The proposed method first identifies the research objective types and disciplinary labels of papers using a text classification technique;second,it generates abstractive titles for each paper based on abstract and research objective types using a generative pre-trained language model;third,it extracts problem phrases from generated titles according to regular expression rules;fourth,it creates problem relation networks and identifies the same problems by exploiting a weighted community detection algorithm;finally,it identifies multidisciplinary problems based on the disciplinary labels of papers.Findings:Experiments in the“Carbon Peaking and Carbon Neutrality”field show that the proposed method can effectively identify multidisciplinary research problems.The disciplinary distribution of the identified problems is consistent with our understanding of multidisciplinary collaboration in the field.Research limitations:It is necessary to use the proposed method in other multidisciplinary fields to validate its effectiveness.Practical implications:Multidisciplinary problem identification helps to gather multidisciplinary forces to solve complex real-world problems for the governments,fund valuable multidisciplinary problems for research management authorities,and borrow ideas from other disciplines for researchers.Originality/value:This approach proposes a novel multidisciplinary problem identification method based on text generation,which identifies multidisciplinary problems based on generative abstractive titles of papers without data annotation required by standard sequence labeling techniques.
基金funded by the Research Project of Natural Science at Anhui Universities in 2021,Research on relation extraction of emergency plan knowledge graph based on deep embedding clustering(No.KJ2021A0994).
文摘Currently,the amount of sports news is increasing,given the number of sports available.As a result,manually writing sports news requires high labor costs to achieve the intended efficiency.Therefore,it is necessary to develop the automatic generation of sports news.Most available news gen-eration methods mainly rely on real-time commentary sentences,which have the following limitations:(1)unable to select suitable commentary sentences for news generation,and(2)the generated sports news could not accurately describe game events.Therefore,this study proposes a sports news generation with text-editing model(SNG-TE)is proposed to generate sports news,which includes selector and rewriter modules.Within the study context,a weight adjustment mechanism in the selector module is designed to improve the hit rate of important sentences.Furthermore,the text-editing model is introduced in the rewriter module to ensure that the generated news sentences can cor-rectly describe the game events.The annotation and generation experiments are designed to evaluate the developed model.The study results have shown that in the annotation experiment,the accuracy of the sentence annotated by the selector increased by about 8%compared with other methods.Moreover,in the generation experiment,the sports news generated by the rewriter achieved a 49.66 ROUGE-1 score and 21.47 ROUGE-2,both of which are better than the available models.Additionally,the proposed model saved about 15 times the consumption of time.Hence,the proposed model provides better performance in both accuracy and efficiency,which is very suitable for the automatic generation of sports news.
基金supported by the National Natural Science Foundation(NSFC)Programs of China(Grant Nos.:72011540408 and 72032006)the National Research Foundation of Korea(Grant No.:NRF-2020K2A9A2A06069972)the support of the Youth Innovation Team of Shaanxi Universities“Big data and Business Intelligent Innovation Team”and Shaanxi Superiority Funding Project for Scientific and Technological Activities of Overseas Scholars(Grant No.:2018017).
文摘Digitization,informatization,and Internet penetration have led to a significant rise in cross-border e-commerce(CBEC),attracting considerable interest from academia,government,and industry.This study employed a novel method combining automatic text generation technology and traditional bibliometric analysis to summarize and categorize the research on CBEC evolution from 2000 to 2021.Articles were selected and examined with a focus on four dimensions:customer,risk,supply chain,and platform.Contradictions in these dimensions were found to result in two major obstacles to CBEC development,namely,dataset sharing and platform scalability.These obstacles prevent research on cross-border platforms from moving beyond theory-based studies.Further research needs to examine how soft computing can be used to accelerate and remodel the global trade ecosystem.
基金supported by the Science and Technology Department of Sichuan Province(No.2021YFG0156).
文摘Generating diverse and factual text is challenging and is receiving increasing attention.By sampling from the latent space,variational autoencoder-based models have recently enhanced the diversity of generated text.However,existing research predominantly depends on summarizationmodels to offer paragraph-level semantic information for enhancing factual correctness.The challenge lies in effectively generating factual text using sentence-level variational autoencoder-based models.In this paper,a novel model called fact-aware conditional variational autoencoder is proposed to balance the factual correctness and diversity of generated text.Specifically,our model encodes the input sentences and uses them as facts to build a conditional variational autoencoder network.By training a conditional variational autoencoder network,the model is enabled to generate text based on input facts.Building upon this foundation,the input text is passed to the discriminator along with the generated text.By employing adversarial training,the model is encouraged to generate text that is indistinguishable to the discriminator,thereby enhancing the quality of the generated text.To further improve the factual correctness,inspired by the natural language inference system,the entailment recognition task is introduced to be trained together with the discriminator via multi-task learning.Moreover,based on the entailment recognition results,a penalty term is further proposed to reconstruct the loss of our model,forcing the generator to generate text consistent with the facts.Experimental results demonstrate that compared with competitivemodels,ourmodel has achieved substantial improvements in both the quality and factual correctness of the text,despite only sacrificing a small amount of diversity.Furthermore,when considering a comprehensive evaluation of diversity and quality metrics,our model has also demonstrated the best performance.
文摘With the wider growth of web-based documents,the necessity of automatic document clustering and text summarization is increased.Here,document summarization that is extracting the essential task with appropriate information,removal of unnecessary data and providing the data in a cohesive and coherent manner is determined to be a most confronting task.In this research,a novel intelligent model for document clustering is designed with graph model and Fuzzy based association rule generation(gFAR).Initially,the graph model is used to map the relationship among the data(multi-source)followed by the establishment of document clustering with the generation of association rule using the fuzzy concept.This method shows benefit in redundancy elimination by mapping the relevant document using graph model and reduces the time consumption and improves the accuracy using the association rule generation with fuzzy.This framework is provided in an interpretable way for document clustering.It iteratively reduces the error rate during relationship mapping among the data(clusters)with the assistance of weighted document content.Also,this model represents the significance of data features with class discrimination.It is also helpful in measuring the significance of the features during the data clustering process.The simulation is done with MATLAB 2016b environment and evaluated with the empirical standards like Relative Risk Patterns(RRP),ROUGE score,and Discrimination Information Measure(DMI)respectively.Here,DailyMail and DUC 2004 dataset is used to extract the empirical results.The proposed gFAR model gives better trade-off while compared with various prevailing approaches.
文摘To address the difficulty of training high-quality models in some specific domains due to the lack of fine-grained annotation resources, we propose in this paper a knowledge-integrated cross-domain data generation method for unsupervised domain adaptation tasks. Specifically, we extract domain features, lexical and syntactic knowledge from source-domain and target-domain data, and use a masking model with an extended masking strategy and a re-masking strategy to obtain domain-specific data that remove domain-specific features. Finally, we improve the sequence generation model BART and use it to generate high-quality target domain data for the task of aspect and opinion co-extraction from the target domain. Experiments were performed on three conventional English datasets from different domains, and our method generates more accurate and diverse target domain data with the best results compared to previous methods.
文摘In this study,we explore the potential of Multiway Transformers for text-to-image generation to achieve performance improvements through a concise and efficient decoupled model design and the inference efficiency provided by bidirectional encoding.We propose a method for improving the image tokenizer using pretrained Vision Transformers.Next,we employ bidirectional Multiway Transformers to restore the masked visual tokens combined with the unmasked text tokens.On the MS-COCO benchmark,our Multiway Transformers outperform vanilla Transformers,achieving superior FID scores and confirming the efficacy of the modality-specific parameter computation design.Ablation studies reveal that the fusion of visual and text tokens in bidirectional encoding contributes to improved model performance.Additionally,our proposed tokenizer outperforms VQGAN in image reconstruction quality and enhances the text-to-image generation results.By incorporating the additional CC-3M dataset for intermediate finetuning on our model with 688M parameters,we achieve competitive results with a finetuned FID score of 4.98 on MS-COCO.
基金supported in part by the National Key Research and Development Program of China(2022YFB4501704)in part by the Shanghai Science and Technology Innovation Action Plan Project(22511100700).
文摘As an important subject of natural language generation,Controllable Text Generation(CTG)focuses on integrating additional constraints and controls while generating texts and has attracted a lot of attention.Existing controllable text generation approaches mainly capture the statistical association implied within training texts,but generated texts lack causality consideration.This paper intends to review recent CTG approaches from a causal perspective.Firstly,according to previous research on basic types of CTG models,it is discovered that their essence is to obtain the association,and then four kinds of challenges caused by absence of causality are introduced.Next,this paper reviews the improvements to address these challenges from four aspects,namely representation disentanglement,causal inference,knowledge enhancement and multi-aspect CTG respectively.Additionally,this paper inspects existing evaluations of CTG,especially evaluations for causality of CTG.Finally,this review discusses some future research directions for the causality improvement of CTG and makes a conclusion.
基金This work was supported in part by the National Natural Science Foundation of China under Grant 61872134,61672222,author Y.L.Liu,http://www.nsfc.gov.cn/in part by Science and Technology Development Center of the Ministry of Education under Grant 2019J01020,author Y.L.Liu,http://www.moe.gov.cn/+1 种基金in part by Science and Technology Project of Transport Department of Hunan Province under Grant 201935,author Y.L.Liu,http://jtt.hunan.gov.cn/Science and Technology Program of Changsha City under Grant kh200519,kq2004021,author Y.L.Liu,http://kjj.changsha.gov.cn/.
文摘Steganography based on generative adversarial networks(GANs)has become a hot topic among researchers.Due to GANs being unsuitable for text fields with discrete characteristics,researchers have proposed GANbased steganography methods that are less dependent on text.In this paper,we propose a new method of generative lyrics steganography based on GANs,called GAN-GLS.The proposed method uses the GAN model and the largescale lyrics corpus to construct and train a lyrics generator.In this method,the GAN uses a previously generated line of a lyric as the input sentence in order to generate the next line of the lyric.Using a strategy based on the penalty mechanism in training,the GAN model generates non-repetitive and diverse lyrics.The secret information is then processed according to the data characteristics of the generated lyrics in order to hide information.Unlike other text generation-based linguistic steganographic methods,our method changes the way that multiple generated candidate items are selected as the candidate groups in order to encode the conditional probability distribution.The experimental results demonstrate that our method can generate highquality lyrics as stego-texts.Moreover,compared with other similar methods,the proposed method achieves good performance in terms of imperceptibility,embedding rate,effectiveness,extraction success rate and security.
基金supported by National Natural Science Foundation of China(NSFC)(Grant Nos.62162022,62162024)the Key Research and Development Program of Hainan Province(Grant Nos.ZDYF2020040,ZDYF2021GXJS003)+2 种基金the Major Science and Technology Project of Hainan Province(Grant No.ZDKJ2020012)Hainan Provincial Natural Science Foundation of China(Grant Nos.620MS021,621QN211)Science and Technology Development Center of the Ministry of Education Industry-University-Research Innovation Fund(2021JQR017).
文摘In the realm of Multi-Label Text Classification(MLTC),the dual challenges of extracting rich semantic features from text and discerning inter-label relationships have spurred innovative approaches.Many studies in semantic feature extraction have turned to external knowledge to augment the model’s grasp of textual content,often overlooking intrinsic textual cues such as label statistical features.In contrast,these endogenous insights naturally align with the classification task.In our paper,to complement this focus on intrinsic knowledge,we introduce a novel Gate-Attention mechanism.This mechanism adeptly integrates statistical features from the text itself into the semantic fabric,enhancing the model’s capacity to understand and represent the data.Additionally,to address the intricate task of mining label correlations,we propose a Dual-end enhancement mechanism.This mechanism effectively mitigates the challenges of information loss and erroneous transmission inherent in traditional long short term memory propagation.We conducted an extensive battery of experiments on the AAPD and RCV1-2 datasets.These experiments serve the dual purpose of confirming the efficacy of both the Gate-Attention mechanism and the Dual-end enhancement mechanism.Our final model unequivocally outperforms the baseline model,attesting to its robustness.These findings emphatically underscore the imperativeness of taking into account not just external knowledge but also the inherent intricacies of textual data when crafting potent MLTC models.
基金the National Natural Science Foundation of China(Grant No.61772036)the Key Laboratory of Science,Technology and Standard in Press Industry(Key Laboratory of Intelligent Press Media Technology)。
文摘In recent years,deep neural network has achieved great success in solving many natural language processing tasks.Particularly,substantial progress has been made on neural text generation,which takes the linguistic and non-linguistic input,and generates natural language text.This survey aims to provide an up-to-date synthesis of core tasks in neural text generation and the architectures adopted to handle these tasks,and draw attention to the challenges in neural text generation.We first outline the mainstream neural text generation frameworks,and then introduce datasets,advanced models and challenges of four core text generation tasks in detail,including AMR-to-text generation,data-to-text generation,and two text-to-text generation tasks(i.e.,text summarization and paraphrase generation).Finally,we present future research directions for neural text generation.This survey can be used as a guide and reference for researchers and practitioners in this area.
基金Project supported by the National Natural Science Foundation of China(No.62272100)the Consulting Project of Chinese Academy of Engineering(No.2023-XY-09)+1 种基金the Major Project of the National Social Science Fund of China(No.21ZD11)the Fundamental Research Funds for the Central Universities,China。
文摘Text generation is an essential research area in artificial intelligence(AI)technology and natural language processing and provides key technical support for the rapid development of AI-generated content(AIGC).It is based on technologies such as natural language processing,machine learning,and deep learning,which enable learning language rules through training models to automatically generate text that meets grammatical and semantic requirements.In this paper,we sort and systematically summarize the main research progress in text generation and review recent text generation papers,focusing on presenting a detailed understanding of the technical models.In addition,several typical text generation application systems are presented.Finally,we address some challenges and future directions in AI text generation.We conclude that improving the quality,quantity,interactivity,and adaptability of generated text can help fundamentally advance AI text generation development.
基金supported by So Paulo Research Foundation(FAPESP)of Brasil under Grant Nos.2011/12823-6,2011/23689-9,and 2011/19850-9
文摘Algorithms for numeric data classification have been applied for text classification. Usually the vector space model is used to represent text collections. The characteristics of this representation such as sparsity and high dimensionality sometimes impair the quality of general-purpose classifiers. Networks can be used to represent text collections, avoiding the high sparsity and allowing to model relationships among different objects that compose a text collection. Such network- based representations can improve the quality of the classification results. One of the simplest ways to represent textual collections by a network is through a bipartite heterogeneous network, which is composed of objects that represent the documents connected to objects that represent the terms. Heterogeneous bipartite networks do not require computation of similarities or relations among the objects and can be used to model any type of text collection. Due to the advantages of representing text collections through bipartite heterogeneous networks, in this article we present a text classifier which builds a classification model using the structure of a bipartite heterogeneous network. Such an algorithm, referred to as IMBHN （Inductive Model Based on Bipartite Heterogeneous Network）, induces a classification model assigning weights to objects that represent the terms for each class of the text collection. An empirical evaluation using a large amount of text collections from different domains shows that the proposed IMBHN algorithm produces significantly better results than k-NN, C4.5, SVM, and Naive Bayes algorithms.
基金Supported by the National Natural Science Foundation of China(No.82174276 and 82074580)the Key Research and Development Program of Jiangsu Province(No.BE2022712)+2 种基金China Postdoctoral Foundation(No.2021M701674)Postdoctoral Research Program of Jiangsu Province(No.2021K457C)Qinglan Project of Jiangsu Universities 2021。
文摘Chinese medicine(CM)diagnosis intellectualization is one of the hotspots in the research of CM modernization.The traditional CM intelligent diagnosis models transform the CM diagnosis issues into classification issues,however,it is difficult to solve the problems such as excessive or similar categories.With the development of natural language processing techniques,text generation technique has become increasingly mature.In this study,we aimed to establish the CM diagnosis generation model by transforming the CM diagnosis issues into text generation issues.The semantic context characteristic learning capacity was enhanced referring to Bidirectional Long Short-Term Memory(BILSTM)with Transformer as the backbone network.Meanwhile,the CM diagnosis generation model Knowledge Graph Enhanced Transformer(KGET)was established by introducing the knowledge in medical field to enhance the inferential capability.The KGET model was established based on 566 CM case texts,and was compared with the classic text generation models including Long Short-Term Memory sequence-to-sequence(LSTM-seq2seq),Bidirectional and Auto-Regression Transformer(BART),and Chinese Pre-trained Unbalanced Transformer(CPT),so as to analyze the model manifestations.Finally,the ablation experiments were performed to explore the influence of the optimized part on the KGET model.The results of Bilingual Evaluation Understudy(BLEU),Recall-Oriented Understudy for Gisting Evaluation 1(ROUGE1),ROUGE2 and Edit distance of KGET model were 45.85,73.93,54.59 and 7.12,respectively in this study.Compared with LSTM-seq2seq,BART and CPT models,the KGET model was higher in BLEU,ROUGE1 and ROUGE2 by 6.00–17.09,1.65–9.39 and 0.51–17.62,respectively,and lower in Edit distance by 0.47–3.21.The ablation experiment results revealed that introduction of BILSTM model and prior knowledge could significantly increase the model performance.Additionally,the manual assessment indicated that the CM diagnosis results of the KGET model used in this study were highly consistent with the practical diagnosis results.In conclusion,text generation technology can be effectively applied to CM diagnostic modeling.It can effectively avoid the problem of poor diagnostic performance caused by excessive and similar categories in traditional CM diagnostic classification models.CM diagnostic text generation technology has broad application prospects in the future.
文摘The sources of meanings of literary works can be reviewed from various angles, one of which should be hermeneutics. Since the mid-20~ century, Western hermeneutics developed two important types of theories： one was the ontological hermeneutics from Martin Heidegger to Hans-Georg Gadamer, orientated towards the reader-centered theory in terms of the view of meanings; the other was the ＂hermeneutics as the general methodology of the Geisteswissenschaften＂ represented by Italian philosopher Emilio Betti, which affirmed that the author （subject） was one of the important sources of meanings of works. Due to intricate reasons, the latter exerted less influence, and Gadamer＇s ontological hermeneutics firmly stuck to a mainstay position in European and American academia. Since the 1990s, the circle of literature in China has also been greatly influenced by the hermeneutic theories from Heidegger and Gadamer to reception aesthetics, and Betti＇s impact almost can be neglected. Consequently, the view of meanings of literary works from the perspective of the reader-centered theory has been widely accepted, and the role of the author to endow initial meanings to his works was belittled and even denied. Since Zhang Jiang advanced the theory of ＂imposed interpretation＂ in 2014, academia has begun to reflect upon the one-sidedness of the reader-centered theory （relativism and subjectivism）, and pick up the significance, which cannot be denied, of the author＇s meaning in the generation of meanings of literary works. In reference to Betti＇s hermeneutic train of thought and methodology, this essay, based on the practice of literary creation, concludes that the meaning of a literary work is created by the author and readers together in their interaction, and constantly generated in the dynamic process of the three factors of author, literary texts and readers, rather than by the author alone, or by readers solely.
基金supported by National Key R&D Program of China(2019YFC1521102)the National Natural Science Foundation of China(61932003)Beijing Science and Technology Plan(Z221100007722004).
文摘Hierarchical multi-granularity image classification is a challenging task that aims to tag each given image with multiple granularity labels simultaneously.Existing methods tend to overlook that different image regions contribute differently to label prediction at different granularities,and also insufficiently consider relationships between the hierarchical multi-granularity labels.We introduce a sequence-to-sequence mechanism to overcome these two problems and propose a multi-granularity sequence generation(MGSG)approach for the hierarchical multi-granularity image classification task.Specifically,we introduce a transformer architecture to encode the image into visual representation sequences.Next,we traverse the taxonomic tree and organize the multi-granularity labels into sequences,and vectorize them and add positional information.The proposed multi-granularity sequence generation method builds a decoder that takes visual representation sequences and semantic label embedding as inputs,and outputs the predicted multi-granularity label sequence.The decoder models dependencies and correlations between multi-granularity labels through a masked multi-head self-attention mechanism,and relates visual information to the semantic label information through a crossmodality attention mechanism.In this way,the proposed method preserves the relationships between labels at different granularity levels and takes into account the influence of different image regions on labels with different granularities.Evaluations on six public benchmarks qualitatively and quantitatively demonstrate the advantages of the proposed method.Our project is available at https://github.com/liuxindazz/mgs.
文摘Aiming at complex and changeable factors such as speech theme and environment,which make it difficult for a speaker to prepare the speech text in a short time,this paper proposes a speech generation and demonstration system based on deep learning.This system is based on the Deep Learning Development Framework(PyTorch),trained through the theory of GPT-2 and the open source pretrained model,to generate multiple speeches according to the topics given by users,and the system generates thefinal speech and corresponding voice demon-stration audio through text modification,speech synthesis and other technologies to help users quickly obtain the target document and audio.Experiments show that the text generated by this model is smooth and easy to use,which helps shorten the preparation time of speakers and improves the confidence of the impromptu speaker.In addition,the paper explores the application prospects of text generation and has certain reference value.
