Deep neural networks(DNNs)are effective in solving both forward and inverse problems for nonlinear partial differential equations(PDEs).However,conventional DNNs are not effective in handling problems such as delay di...Deep neural networks(DNNs)are effective in solving both forward and inverse problems for nonlinear partial differential equations(PDEs).However,conventional DNNs are not effective in handling problems such as delay differential equations(DDEs)and delay integrodifferential equations(DIDEs)with constant delays,primarily due to their low regularity at delayinduced breaking points.In this paper,a DNN method that combines multi-task learning(MTL)which is proposed to solve both the forward and inverse problems of DIDEs.The core idea of this approach is to divide the original equation into multiple tasks based on the delay,using auxiliary outputs to represent the integral terms,followed by the use of MTL to seamlessly incorporate the properties at the breaking points into the loss function.Furthermore,given the increased training dificulty associated with multiple tasks and outputs,we employ a sequential training scheme to reduce training complexity and provide reference solutions for subsequent tasks.This approach significantly enhances the approximation accuracy of solving DIDEs with DNNs,as demonstrated by comparisons with traditional DNN methods.We validate the effectiveness of this method through several numerical experiments,test various parameter sharing structures in MTL and compare the testing results of these structures.Finally,this method is implemented to solve the inverse problem of nonlinear DIDE and the results show that the unknown parameters of DIDE can be discovered with sparse or noisy data.展开更多
Aerosol optical depth(AOD)and fine particulate matter with a diameter of less than or equal to 2.5μm(PM_(2.5))play crucial roles in air quality,human health,and climate change.However,the complex correlation of AOD–...Aerosol optical depth(AOD)and fine particulate matter with a diameter of less than or equal to 2.5μm(PM_(2.5))play crucial roles in air quality,human health,and climate change.However,the complex correlation of AOD–PM_(2.5)and the limitations of existing algorithms pose a significant challenge in realizing the accurate joint retrieval of these two parameters at the same location.On this point,a multi-task learning(MTL)model,which enables the joint retrieval of PM_(2.5)concentration and AOD,is proposed and applied on the top-of-the-atmosphere reflectance data gathered by the Fengyun-4A Advanced Geosynchronous Radiation Imager(FY-4A AGRI),and compared to that of two single-task learning models—namely,Random Forest(RF)and Deep Neural Network(DNN).Specifically,MTL achieves a coefficient of determination(R^(2))of 0.88 and a root-mean-square error(RMSE)of 0.10 in AOD retrieval.In comparison to RF,the R^(2)increases by 0.04,the RMSE decreases by 0.02,and the percentage of retrieval results falling within the expected error range(Within-EE)rises by 5.55%.The R^(2)and RMSE of PM_(2.5)retrieval by MTL are 0.84 and 13.76μg m~(-3)respectively.Compared with RF,the R^(2)increases by 0.06,the RMSE decreases by 4.55μg m~(-3),and the Within-EE increases by 7.28%.Additionally,compared to DNN,MTL shows an increase of 0.01 in R^(2)and a decrease of 0.02 in RMSE in AOD retrieval,with a corresponding increase of 2.89%in Within-EE.For PM_(2.5)retrieval,MTL exhibits an increase of 0.05 in R^(2),a decrease of 1.76μg m~(-3)in RMSE,and an increase of 6.83%in Within-EE.The evaluation suggests that MTL is able to provide simultaneously improved AOD and PM_(2.5)retrievals,demonstrating a significant advantage in efficiently capturing the spatial distribution of PM_(2.5)concentration and AOD.展开更多
Accurate forecasting of blast furnace gas(BFG)production is an essential prerequisite for reasonable energy scheduling and management to reduce carbon emissions.Coupling forecasting between BFG generation and consumpt...Accurate forecasting of blast furnace gas(BFG)production is an essential prerequisite for reasonable energy scheduling and management to reduce carbon emissions.Coupling forecasting between BFG generation and consumption dynamics was taken as the research object.A multi-task learning(MTL)method for BFG forecasting was proposed,which integrated a coupling correlation coefficient(CCC)and an inverted transformer structure.The CCC method could enhance key information extraction by establishing relationships between multiple prediction targets and relevant factors,while MTL effectively captured the inherent correlations between BFG generation and consumption.Finally,a real-world case study was conducted to compare the proposed model with four benchmark models.Results indicated significant reductions in average mean absolute percentage error by 33.37%,achieving 1.92%,with a computational time of 76 s.The sensitivity analysis of hyperparameters such as learning rate,batch size,and units of the long short-term memory layer highlights the importance of hyperparameter tuning.展开更多
Predicting the health status of stroke patients at different stages of the disease is a critical clinical task.The onset and development of stroke are affected by an array of factors,encompassing genetic predispositio...Predicting the health status of stroke patients at different stages of the disease is a critical clinical task.The onset and development of stroke are affected by an array of factors,encompassing genetic predisposition,environmental exposure,unhealthy lifestyle habits,and existing medical conditions.Although existing machine learning-based methods for predicting stroke patients’health status have made significant progress,limitations remain in terms of prediction accuracy,model explainability,and system optimization.This paper proposes a multi-task learning approach based on Explainable Artificial Intelligence(XAI)for predicting the health status of stroke patients.First,we design a comprehensive multi-task learning framework that utilizes the task correlation of predicting various health status indicators in patients,enabling the parallel prediction of multiple health indicators.Second,we develop a multi-task Area Under Curve(AUC)optimization algorithm based on adaptive low-rank representation,which removes irrelevant information from the model structure to enhance the performance of multi-task AUC optimization.Additionally,the model’s explainability is analyzed through the stability analysis of SHAP values.Experimental results demonstrate that our approach outperforms comparison algorithms in key prognostic metrics F1 score and Efficiency.展开更多
As the group-buying model shows significant progress in attracting new users,enhancing user engagement,and increasing platform profitability,providing personalized recommendations for group-buying users has emerged as...As the group-buying model shows significant progress in attracting new users,enhancing user engagement,and increasing platform profitability,providing personalized recommendations for group-buying users has emerged as a new challenge in the field of recommendation systems.This paper introduces a group-buying recommendation model based on multi-head attention mechanisms and multi-task learning,termed the Multi-head Attention Mechanisms and Multi-task Learning Group-Buying Recommendation(MAMGBR)model,specifically designed to optimize group-buying recommendations on e-commerce platforms.The core dataset of this study comes from the Chinese maternal and infant e-commerce platform“Beibei,”encompassing approximately 430,000 successful groupbuying actions and over 120,000 users.Themodel focuses on twomain tasks:recommending items for group organizers(Task Ⅰ)and recommending participants for a given group-buying event(Task Ⅱ).In model evaluation,MAMGBR achieves an MRR@10 of 0.7696 for Task I,marking a 20.23%improvement over baseline models.Furthermore,in Task II,where complex interaction patterns prevail,MAMGBR utilizes auxiliary loss functions to effectively model the multifaceted roles of users,items,and participants,leading to a 24.08%increase in MRR@100 under a 1:99 sample ratio.Experimental results show that compared to benchmark models,such as NGCF and EATNN,MAMGBR’s integration ofmulti-head attentionmechanisms,expert networks,and gating mechanisms enables more accurate modeling of user preferences and social associations within group-buying scenarios,significantly enhancing recommendation accuracy and platform group-buying success rates.展开更多
The accurate prediction of drug absorption,distribution,metabolism,excretion,and toxicity(ADMET)properties represents a crucial step in early drug development for reducing failure risk.Current deep learning approaches...The accurate prediction of drug absorption,distribution,metabolism,excretion,and toxicity(ADMET)properties represents a crucial step in early drug development for reducing failure risk.Current deep learning approaches face challenges with data sparsity and information loss due to single-molecule representation limitations and isolated predictive tasks.This research proposes molecular properties prediction with parallel-view and collaborative learning(MolP-PC),a multi-view fusion and multi-task deep learning framework that integrates 1D molecular fingerprints(MFs),2D molecular graphs,and 3D geometric representations,incorporating an attention-gated fusion mechanism and multi-task adaptive learning strategy for precise ADMET property predictions.Experimental results demonstrate that MolP-PC achieves optimal performance in 27 of 54 tasks,with its multi-task learning(MTL)mechanism significantly enhancing predictive performance on small-scale datasets and surpassing single-task models in 41 of 54 tasks.Additional ablation studies and interpretability analyses confirm the significance of multi-view fusion in capturing multi-dimensional molecular information and enhancing model generalization.A case study examining the anticancer compound Oroxylin A demonstrates MolP-PC’s effective generalization in predicting key pharmacokinetic parameters such as half-life(T0.5)and clearance(CL),indicating its practical utility in drug modeling.However,the model exhibits a tendency to underestimate volume of distribution(VD),indicating potential for improvement in analyzing compounds with high tissue distribution.This study presents an efficient and interpretable approach for ADMET property prediction,establishing a novel framework for molecular optimization and risk assessment in drug development.展开更多
Tropical cyclones(TCs)are one of the most serious types of natural disasters,and accurate TC activity predictions are key to disaster prevention and mitigation.Recently,TC track predictions have made significant progr...Tropical cyclones(TCs)are one of the most serious types of natural disasters,and accurate TC activity predictions are key to disaster prevention and mitigation.Recently,TC track predictions have made significant progress,but the ability to predict their intensity is obviously lagging behind.At present,research on TC intensity prediction takes atmospheric reanalysis data as the research object and mines the relationship between TC-related environmental factors and intensity through deep learning.However,reanalysis data are non-real-time in nature,which does not meet the requirements for operational forecasting applications.Therefore,a TC intensity prediction model named TC-Rolling is proposed,which can simultaneously extract the degree of symmetry for strong TC convective cloud and convection intensity,and fuse the deviation-angle variance with satellite images to construct the correlation between TC convection structure and intensity.For TCs'complex dynamic processes,a convolutional neural network(CNN)is used to learn their temporal and spatial features.For real-time intensity estimation,multi-task learning acts as an implicit time-series enhancement.The model is designed with a rolling strategy that aims to moderate the long-term dependent decay problem and improve accuracy for short-term intensity predictions.Since multiple tasks are correlated,the loss function of 12 h and 24 h are corrected.After testing on a sample of TCs in the Northwest Pacific,with a 4.48 kt root-mean-square error(RMSE)of 6 h intensity prediction,5.78 kt for 12 h,and 13.94 kt for 24 h,TC records from official agencies are used to assess the validity of TC-Rolling.展开更多
The utilization of millimeter-wave frequencies and cognitive radio(CR)are promising ways to increase the spectral efficiency of wireless communication systems.However,conventional CR spectrum sensing techniques entail...The utilization of millimeter-wave frequencies and cognitive radio(CR)are promising ways to increase the spectral efficiency of wireless communication systems.However,conventional CR spectrum sensing techniques entail sampling the received signal at a Nyquist rate,and they are not viable for wideband signals due to their high cost.This paper expounds on how sub-Nyquist sampling in conjunction with deep learning can be leveraged to remove this limitation.To this end,we propose a multi-task learning(MTL)framework using convolutional neural networks for the joint inference of the underlying narrowband signal number,their modulation scheme,and their location in a wideband spectrum.We demonstrate the effectiveness of the proposed framework for real-world millimeter-wave wideband signals collected by physical devices,exhibiting a 91.7% accuracy in the joint inference task when considering up to two narrowband signals over a wideband spectrum.Ultimately,the proposed data-driven approach enables on-the-fly wideband spectrum sensing,combining accuracy,and computational efficiency,which are indispensable for CR and opportunistic networking.展开更多
In recent years,self-supervised learning has achieved great success in areas such as computer vision and natural language processing because it can mine supervised signals from unlabeled data and reduce the reliance o...In recent years,self-supervised learning has achieved great success in areas such as computer vision and natural language processing because it can mine supervised signals from unlabeled data and reduce the reliance on manual labels.However,the currently generated self-supervised signals are either neighbor discrimination or self-discrimination,and there is no model to integrate neighbor discrimination and self-discrimination.Based on this,this paper proposes Fu-Rec that integrates neighbor-discrimination contrastive learning and self-discrimination contrastive learning,which consists of three modules:(1)neighbor-discrimination contrastive learning,(2)selfdiscrimination contrastive learning,and(3)recommendation module.The neighbor-discrimination contrastive learning and selfdiscrimination contrastive learning tasks are used as auxiliary tasks to assist the recommendation task.The Fu-Rec model effectively utilizes the respective advantages of neighbor-discrimination and self-discrimination to consider the information of the user’s neighbors as well as the user and the item itself for the recommendation,which results in better performance of the recommendation module.Experimental results on several public datasets demonstrate the effectiveness of the Fu-Rec proposed in this paper.展开更多
Pre-stack seismic inversion is an effective way to investigate the characteristics of hydrocarbon-bearing reservoirs.Multi-parameter application is the key to identifying reservoir lithology and fluid in pre-stack inv...Pre-stack seismic inversion is an effective way to investigate the characteristics of hydrocarbon-bearing reservoirs.Multi-parameter application is the key to identifying reservoir lithology and fluid in pre-stack inversion.However,multi-parameter inversion may bring coupling effects on the parameters and destabilize the inversion.In addition,the lateral recognition accuracy of geological structures receives great attention.To address these challenges,a multi-task learning network considering the angle-gather difference is proposed in this work.The deep learning network is usually assumed as a black box and it is unclear what it can learn.However,the introduction of angle-gather difference can force the deep learning network to focus on the lateral differences,thus improving the lateral accuracy of the prediction profile.The proposed deep learning network includes input and output blocks.First,angle gathers and the angle-gather difference are fed into two separate input blocks with Res Net architecture and Unet architecture,respectively.Then,three elastic parameters,including P-and S-wave velocities and density,are simultaneously predicted based on the idea of multi-task learning by using three separate output blocks with the same convolutional network layers.Experimental and field data tests demonstrate the effectiveness of the proposed method in improving the prediction accuracy of seismic elastic parameters.展开更多
The global nuclear mass based on the macroscopic-microscopic model was studied by applying a newly designed multi-task learning artificial neural network(MTL-ANN). First, the reported nuclear binding energies of 2095 ...The global nuclear mass based on the macroscopic-microscopic model was studied by applying a newly designed multi-task learning artificial neural network(MTL-ANN). First, the reported nuclear binding energies of 2095 nuclei(Z ≥ 8, N ≥ 8) released in the latest Atomic Mass Evaluation AME2020 and the deviations between the fitting result of the liquid drop model(LDM)and data from AME2020 for each nucleus were obtained.To compensate for the deviations and investigate the possible ignored physics in the LDM, the MTL-ANN method was introduced in the model. Compared to the single-task learning(STL) method, this new network has a powerful ability to simultaneously learn multi-nuclear properties,such as the binding energies and single neutron and proton separation energies. Moreover, it is highly effective in reducing the risk of overfitting and achieving better predictions. Consequently, good predictions can be obtained using this nuclear mass model for both the training and validation datasets and for the testing dataset. In detail, the global root mean square(RMS) of the binding energy is effectively reduced from approximately 2.4 MeV of LDM to the current 0.2 MeV, and the RMS of Sn, Spcan also reach approximately 0.2 MeV. Moreover, compared to STL, for the training and validation sets, 3-9% improvement can be achieved with the binding energy, and 20-30% improvement for S_(n), S_(p);for the testing sets, the reduction in deviations can even reach 30-40%, which significantly illustrates the advantage of the current MTL.展开更多
Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new...Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new tasks and datasets,which is inevitably time-consuming.This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components,three-type seismic damage,and four-type deterioration states.The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head,task-specific recognition subnetwork.The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures.The multi-head,task-specific recognition subnetwork consists of three individual self-attention pipelines,each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task.A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one.Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity.The results show that the proposed method can simultaneously recognize different structural components,seismic damage,and deterioration states,and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.展开更多
There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them.However,the volume of internet data is beyond the...There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them.However,the volume of internet data is beyond the processing capabilities of the current internet infrastructure.Therefore,engineering works using technology to organize and analyze information and extract useful information are interesting in both industry and academia.The goal of this paper is to explore the entity relationship based on deep learning,introduce semantic knowledge by using the prepared language model,develop an advanced entity relationship information extraction method by combining Robustly Optimized BERT Approach(RoBERTa)and multi-task learning,and combine the intelligent characters in the field of linguistic,called Robustly Optimized BERT Approach+Multi-Task Learning(RoBERTa+MTL).To improve the effectiveness of model interaction,multi-task teaching is used to implement the observation information of auxiliary tasks.Experimental results show that our method has achieved an accuracy of 88.95 entity relationship extraction,and a further it has achieved 86.35%of accuracy after being combined with multi-task learning.展开更多
Convective storms and lightning are among the most important weather phenomena that are challenging to forecast.In this study,a novel multi-task learning(MTL)encoder-decoder U-net neural network was developed to forec...Convective storms and lightning are among the most important weather phenomena that are challenging to forecast.In this study,a novel multi-task learning(MTL)encoder-decoder U-net neural network was developed to forecast convective storms and lightning with lead times for up to 90 min,using GOES-16 geostationary satellite infrared brightness temperatures(IRBTs),lightning flashes from Geostationary Lightning Mapper(GLM),and vertically integrated liquid(VIL)from Next Generation Weather Radar(NEXRAD).To cope with the heavily skewed distribution of lightning data,a spatiotemporal exponent-weighted loss function and log-transformed lightning normalization approach were developed.The effects of MTL,single-task learning(STL),and IRBTs as auxiliary input features on convection and lightning nowcasting were investigated.The results showed that normalizing the heavily skew-distributed lightning data along with a log-transformation dramatically outperforms the min-max normalization method for nowcasting an intense lightning event.The MTL model significantly outperformed the STL model for both lightning nowcasting and VIL nowcasting,particularly for intense lightning events.The MTL also helped delay the lightning forecast performance decay with the lead times.Furthermore,incorporating satellite IRBTs as auxiliary input features substantially improved lightning nowcasting,but produced little difference in VIL forecasting.Finally,the MTL model performed better for forecasting both lightning and the VIL of organized convective storms than for isolated cells.展开更多
Sea state bias(SSB)is an important component of errors for the radar altimeter measurements of sea surface height(SSH).However,existing SSB estimation methods are almost all based on single-task learning(STL),where on...Sea state bias(SSB)is an important component of errors for the radar altimeter measurements of sea surface height(SSH).However,existing SSB estimation methods are almost all based on single-task learning(STL),where one model is built on the data from only one radar altimeter.In this paper,taking account of the data from multiple radar altimeters available,we introduced a multi-task learning method,called trace-norm regularized multi-task learning(TNR-MTL),for SSB estimation.Corresponding to each individual task,TNR-MLT involves only three parameters.Hence,it is easy to implement.More importantly,the convergence of TNR-MLT is theoretically guaranteed.Compared with the commonly used STL models,TNR-MTL can effectively utilize the shared information between data from multiple altimeters.During the training of TNR-MTL,we used the JASON-2 and JASON-3 cycle data to solve two correlated SSB estimation tasks.Then the optimal model was selected to estimate SSB on the JASON-2 and the HY-270-71 cycle intersection data.For the JSAON-2 cycle intersection data,the corrected variance(M)has been reduced by 0.60 cm^2 compared to the geophysical data records(GDR);while for the HY-2 cycle intersection data,M has been reduced by 1.30 cm^2 compared to GDR.Therefore,TNR-MTL is proved to be effective for the SSB estimation tasks.展开更多
With the rapid development of the mobile Internet,users generate massive data in different forms in social network every day,and different characteristics of users are reflected by these social media data.How to integ...With the rapid development of the mobile Internet,users generate massive data in different forms in social network every day,and different characteristics of users are reflected by these social media data.How to integrate multiple heterogeneous information and establish user profiles from multiple perspectives plays an important role in providing personalized services,marketing,and recommendation systems.In this paper,we propose Multi-source&Multi-task Learning for User Profiles in Social Network which integrates multiple social data sources and contains a multi-task learning framework to simultaneously predict various attributes of a user.Firstly,we design their own feature extraction models for multiple heterogeneous data sources.Secondly,we design a shared layer to fuse multiple heterogeneous data sources as general shared representation for multi-task learning.Thirdly,we design each task’s own unique presentation layer for discriminant output of specific-task.Finally,we design a weighted loss function to improve the learning efficiency and prediction accuracy of each task.Our experimental results on more than 5000 Sina Weibo users demonstrate that our approach outperforms state-of-the-art baselines for inferring gender,age and region of social media users.展开更多
Buiding data-driven models using machine learning methods has gradually become a common approach for studying reservoir parameters.Among these methods,deep learning methods are highly effective.From the perspective of...Buiding data-driven models using machine learning methods has gradually become a common approach for studying reservoir parameters.Among these methods,deep learning methods are highly effective.From the perspective of multi-task learning,this paper uses six types of logging data—acoustic logging(AC),gamma ray(GR),compensated neutron porosity(CNL),density(DEN),deep and shallow lateral resistivity(LLD)and shallow lateral resistivity(LLS)—that are inputs and three reservoir parameters that are outputs to build a porosity saturation permeability network(PSP-Net)that can predict porosity,saturation,and permeability values simultaneously.These logging data are obtained from 108 training wells in a medium₋low permeability oilfield block in the western district of China.PSP-Net method adopts a serial structure to realize transfer learning of reservoir-parameter characteristics.Compared with other existing methods at the stage of academic exploration to simulating industrial applications,the proposed method overcomes the disadvantages inherent in single-task learning reservoir-parameter prediction models,including easily overfitting and heavy model-training workload.Additionally,the proposed method demonstrates good anti-overfitting and generalization capabilities,integrating professional knowledge and experience.In 37 test wells,compared with the existing method,the proposed method exhibited an average error reduction of 10.44%,27.79%,and 28.83%from porosity,saturation,permeability calculation.The prediction and actual permeabilities are within one order of magnitude.The training on PSP-Net are simpler and more convenient than other single-task learning methods discussed in this paper.Furthermore,the findings of this paper can help in the re-examination of old oilfield wells and the completion of logging data.展开更多
Aspect-based sentiment analysis(ABSA)is a fine-grained process.Its fundamental subtasks are aspect termextraction(ATE)and aspect polarity classification(APC),and these subtasks are dependent and closely related.Howeve...Aspect-based sentiment analysis(ABSA)is a fine-grained process.Its fundamental subtasks are aspect termextraction(ATE)and aspect polarity classification(APC),and these subtasks are dependent and closely related.However,most existing works on Arabic ABSA content separately address them,assume that aspect terms are preidentified,or use a pipeline model.Pipeline solutions design different models for each task,and the output from the ATE model is used as the input to the APC model,which may result in error propagation among different steps because APC is affected by ATE error.These methods are impractical for real-world scenarios where the ATE task is the base task for APC,and its result impacts the accuracy of APC.Thus,in this study,we focused on a multi-task learning model for Arabic ATE and APC in which the model is jointly trained on two subtasks simultaneously in a singlemodel.This paper integrates themulti-task model,namely Local Cotext Foucse-Aspect Term Extraction and Polarity classification(LCF-ATEPC)and Arabic Bidirectional Encoder Representation from Transformers(AraBERT)as a shred layer for Arabic contextual text representation.The LCF-ATEPC model is based on a multi-head selfattention and local context focus mechanism(LCF)to capture the interactive information between an aspect and its context.Moreover,data augmentation techniques are proposed based on state-of-the-art augmentation techniques(word embedding substitution with constraints and contextual embedding(AraBERT))to increase the diversity of the training dataset.This paper examined the effect of data augmentation on the multi-task model for Arabic ABSA.Extensive experiments were conducted on the original and combined datasets(merging the original and augmented datasets).Experimental results demonstrate that the proposed Multi-task model outperformed existing APC techniques.Superior results were obtained by AraBERT and LCF-ATEPC with fusion layer(AR-LCF-ATEPC-Fusion)and the proposed data augmentation word embedding-based method(FastText)on the combined dataset.展开更多
Prevailing linguistic steganalysis approaches focus on learning sensitive features to distinguish a particular category of steganographic texts from non-steganographic texts,by performing binary classification.While i...Prevailing linguistic steganalysis approaches focus on learning sensitive features to distinguish a particular category of steganographic texts from non-steganographic texts,by performing binary classification.While it remains an unsolved problem and poses a significant threat to the security of cyberspace when various categories of non-steganographic or steganographic texts coexist.In this paper,we propose a general linguistic steganalysis framework named LS-MTL,which introduces the idea of multi-task learning to deal with the classification of various categories of steganographic and non-steganographic texts.LS-MTL captures sensitive linguistic features from multiple related linguistic steganalysis tasks and can concurrently handle diverse tasks with a constructed model.In the proposed framework,convolutional neural networks(CNNs)are utilized as private base models to extract sensitive features for each steganalysis task.Besides,a shared CNN is built to capture potential interaction information and share linguistic features among all tasks.Finally,LS-MTL incorporates the private and shared sensitive features to identify the detected text as steganographic or non-steganographic.Experimental results demonstrate that the proposed framework LS-MTL outperforms the baseline in the multi-category linguistic steganalysis task,while average Acc,Pre,and Rec are increased by 0.5%,1.4%,and 0.4%,respectively.More ablation experimental results show that LS-MTL with the shared module has robust generalization capability and achieves good detection performance even in the case of spare data.展开更多
Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively u...Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively utilize multi-modal remote sensing data to break through the performance bottleneck of single-modal interpretation.In addition,semantic segmentation and height estimation in remote sensing data are two tasks with strong correlation,but existing methods usually study individual tasks separately,which leads to high computational resource overhead.To this end,we propose a Multi-Task learning framework for Multi-Modal remote sensing images(MM_MT).Specifically,we design a Cross-Modal Feature Fusion(CMFF)method,which aggregates complementary information of different modalities to improve the accuracy of semantic segmentation and height estimation.Besides,a dual-stream multi-task learning method is introduced for Joint Semantic Segmentation and Height Estimation(JSSHE),extracting common features in a shared network to save time and resources,and then learning task-specific features in two task branches.Experimental results on the public multi-modal remote sensing image dataset Potsdam show that compared to training two tasks independently,multi-task learning saves 20%of training time and achieves competitive performance with mIoU of 83.02%for semantic segmentation and accuracy of 95.26%for height estimation.展开更多
文摘Deep neural networks(DNNs)are effective in solving both forward and inverse problems for nonlinear partial differential equations(PDEs).However,conventional DNNs are not effective in handling problems such as delay differential equations(DDEs)and delay integrodifferential equations(DIDEs)with constant delays,primarily due to their low regularity at delayinduced breaking points.In this paper,a DNN method that combines multi-task learning(MTL)which is proposed to solve both the forward and inverse problems of DIDEs.The core idea of this approach is to divide the original equation into multiple tasks based on the delay,using auxiliary outputs to represent the integral terms,followed by the use of MTL to seamlessly incorporate the properties at the breaking points into the loss function.Furthermore,given the increased training dificulty associated with multiple tasks and outputs,we employ a sequential training scheme to reduce training complexity and provide reference solutions for subsequent tasks.This approach significantly enhances the approximation accuracy of solving DIDEs with DNNs,as demonstrated by comparisons with traditional DNN methods.We validate the effectiveness of this method through several numerical experiments,test various parameter sharing structures in MTL and compare the testing results of these structures.Finally,this method is implemented to solve the inverse problem of nonlinear DIDE and the results show that the unknown parameters of DIDE can be discovered with sparse or noisy data.
基金supported by the National Natural Science Foundation of China(Grant Nos.42030708,42375138,42030608,42105128,42075079)the Opening Foundation of Key Laboratory of Atmospheric Sounding,China Meteorological Administration(CMA),and the CMA Research Center on Meteorological Observation Engineering Technology(Grant No.U2021Z03),and the Opening Foundation of the Key Laboratory of Atmospheric Chemistry,CMA(Grant No.2022B02)。
文摘Aerosol optical depth(AOD)and fine particulate matter with a diameter of less than or equal to 2.5μm(PM_(2.5))play crucial roles in air quality,human health,and climate change.However,the complex correlation of AOD–PM_(2.5)and the limitations of existing algorithms pose a significant challenge in realizing the accurate joint retrieval of these two parameters at the same location.On this point,a multi-task learning(MTL)model,which enables the joint retrieval of PM_(2.5)concentration and AOD,is proposed and applied on the top-of-the-atmosphere reflectance data gathered by the Fengyun-4A Advanced Geosynchronous Radiation Imager(FY-4A AGRI),and compared to that of two single-task learning models—namely,Random Forest(RF)and Deep Neural Network(DNN).Specifically,MTL achieves a coefficient of determination(R^(2))of 0.88 and a root-mean-square error(RMSE)of 0.10 in AOD retrieval.In comparison to RF,the R^(2)increases by 0.04,the RMSE decreases by 0.02,and the percentage of retrieval results falling within the expected error range(Within-EE)rises by 5.55%.The R^(2)and RMSE of PM_(2.5)retrieval by MTL are 0.84 and 13.76μg m~(-3)respectively.Compared with RF,the R^(2)increases by 0.06,the RMSE decreases by 4.55μg m~(-3),and the Within-EE increases by 7.28%.Additionally,compared to DNN,MTL shows an increase of 0.01 in R^(2)and a decrease of 0.02 in RMSE in AOD retrieval,with a corresponding increase of 2.89%in Within-EE.For PM_(2.5)retrieval,MTL exhibits an increase of 0.05 in R^(2),a decrease of 1.76μg m~(-3)in RMSE,and an increase of 6.83%in Within-EE.The evaluation suggests that MTL is able to provide simultaneously improved AOD and PM_(2.5)retrievals,demonstrating a significant advantage in efficiently capturing the spatial distribution of PM_(2.5)concentration and AOD.
基金supported by the National Natural Science Foundation of China(No.52474435)China Baowu Low Carbon Metallurgy Innovation Foundation(BWLCF202307).
文摘Accurate forecasting of blast furnace gas(BFG)production is an essential prerequisite for reasonable energy scheduling and management to reduce carbon emissions.Coupling forecasting between BFG generation and consumption dynamics was taken as the research object.A multi-task learning(MTL)method for BFG forecasting was proposed,which integrated a coupling correlation coefficient(CCC)and an inverted transformer structure.The CCC method could enhance key information extraction by establishing relationships between multiple prediction targets and relevant factors,while MTL effectively captured the inherent correlations between BFG generation and consumption.Finally,a real-world case study was conducted to compare the proposed model with four benchmark models.Results indicated significant reductions in average mean absolute percentage error by 33.37%,achieving 1.92%,with a computational time of 76 s.The sensitivity analysis of hyperparameters such as learning rate,batch size,and units of the long short-term memory layer highlights the importance of hyperparameter tuning.
基金funded by the Excellent Talent Training Funding Project in Dongcheng District,Beijing,with project number 2024-dchrcpyzz-9.
文摘Predicting the health status of stroke patients at different stages of the disease is a critical clinical task.The onset and development of stroke are affected by an array of factors,encompassing genetic predisposition,environmental exposure,unhealthy lifestyle habits,and existing medical conditions.Although existing machine learning-based methods for predicting stroke patients’health status have made significant progress,limitations remain in terms of prediction accuracy,model explainability,and system optimization.This paper proposes a multi-task learning approach based on Explainable Artificial Intelligence(XAI)for predicting the health status of stroke patients.First,we design a comprehensive multi-task learning framework that utilizes the task correlation of predicting various health status indicators in patients,enabling the parallel prediction of multiple health indicators.Second,we develop a multi-task Area Under Curve(AUC)optimization algorithm based on adaptive low-rank representation,which removes irrelevant information from the model structure to enhance the performance of multi-task AUC optimization.Additionally,the model’s explainability is analyzed through the stability analysis of SHAP values.Experimental results demonstrate that our approach outperforms comparison algorithms in key prognostic metrics F1 score and Efficiency.
基金supported by the Key Research and Development Program of Heilongjiang Province(No.2022ZX01A35).
文摘As the group-buying model shows significant progress in attracting new users,enhancing user engagement,and increasing platform profitability,providing personalized recommendations for group-buying users has emerged as a new challenge in the field of recommendation systems.This paper introduces a group-buying recommendation model based on multi-head attention mechanisms and multi-task learning,termed the Multi-head Attention Mechanisms and Multi-task Learning Group-Buying Recommendation(MAMGBR)model,specifically designed to optimize group-buying recommendations on e-commerce platforms.The core dataset of this study comes from the Chinese maternal and infant e-commerce platform“Beibei,”encompassing approximately 430,000 successful groupbuying actions and over 120,000 users.Themodel focuses on twomain tasks:recommending items for group organizers(Task Ⅰ)and recommending participants for a given group-buying event(Task Ⅱ).In model evaluation,MAMGBR achieves an MRR@10 of 0.7696 for Task I,marking a 20.23%improvement over baseline models.Furthermore,in Task II,where complex interaction patterns prevail,MAMGBR utilizes auxiliary loss functions to effectively model the multifaceted roles of users,items,and participants,leading to a 24.08%increase in MRR@100 under a 1:99 sample ratio.Experimental results show that compared to benchmark models,such as NGCF and EATNN,MAMGBR’s integration ofmulti-head attentionmechanisms,expert networks,and gating mechanisms enables more accurate modeling of user preferences and social associations within group-buying scenarios,significantly enhancing recommendation accuracy and platform group-buying success rates.
基金supported by the research on key technologies for monitoring and identifying drug abuse of anesthetic drugs and psychotropic drugs,and intervention for addiction(No.2023YFC3304200)the program of a study on the diagnosis of addiction to synthetic cannabinoids and methods of assessing the risk of abuse(No.2022YFC3300905)+1 种基金the program of Ab initio design and generation of AI models for small molecule ligands based on target structures(No.2022PE0AC03)ZHIJIANG LAB.
文摘The accurate prediction of drug absorption,distribution,metabolism,excretion,and toxicity(ADMET)properties represents a crucial step in early drug development for reducing failure risk.Current deep learning approaches face challenges with data sparsity and information loss due to single-molecule representation limitations and isolated predictive tasks.This research proposes molecular properties prediction with parallel-view and collaborative learning(MolP-PC),a multi-view fusion and multi-task deep learning framework that integrates 1D molecular fingerprints(MFs),2D molecular graphs,and 3D geometric representations,incorporating an attention-gated fusion mechanism and multi-task adaptive learning strategy for precise ADMET property predictions.Experimental results demonstrate that MolP-PC achieves optimal performance in 27 of 54 tasks,with its multi-task learning(MTL)mechanism significantly enhancing predictive performance on small-scale datasets and surpassing single-task models in 41 of 54 tasks.Additional ablation studies and interpretability analyses confirm the significance of multi-view fusion in capturing multi-dimensional molecular information and enhancing model generalization.A case study examining the anticancer compound Oroxylin A demonstrates MolP-PC’s effective generalization in predicting key pharmacokinetic parameters such as half-life(T0.5)and clearance(CL),indicating its practical utility in drug modeling.However,the model exhibits a tendency to underestimate volume of distribution(VD),indicating potential for improvement in analyzing compounds with high tissue distribution.This study presents an efficient and interpretable approach for ADMET property prediction,establishing a novel framework for molecular optimization and risk assessment in drug development.
基金jointly supported by the National Natural Science Foundation of China(Grant Nos.42075138 and 42375147)the Program on Key Basic Research Project of Jiangsu(Grant No.BE2023829)。
文摘Tropical cyclones(TCs)are one of the most serious types of natural disasters,and accurate TC activity predictions are key to disaster prevention and mitigation.Recently,TC track predictions have made significant progress,but the ability to predict their intensity is obviously lagging behind.At present,research on TC intensity prediction takes atmospheric reanalysis data as the research object and mines the relationship between TC-related environmental factors and intensity through deep learning.However,reanalysis data are non-real-time in nature,which does not meet the requirements for operational forecasting applications.Therefore,a TC intensity prediction model named TC-Rolling is proposed,which can simultaneously extract the degree of symmetry for strong TC convective cloud and convection intensity,and fuse the deviation-angle variance with satellite images to construct the correlation between TC convection structure and intensity.For TCs'complex dynamic processes,a convolutional neural network(CNN)is used to learn their temporal and spatial features.For real-time intensity estimation,multi-task learning acts as an implicit time-series enhancement.The model is designed with a rolling strategy that aims to moderate the long-term dependent decay problem and improve accuracy for short-term intensity predictions.Since multiple tasks are correlated,the loss function of 12 h and 24 h are corrected.After testing on a sample of TCs in the Northwest Pacific,with a 4.48 kt root-mean-square error(RMSE)of 6 h intensity prediction,5.78 kt for 12 h,and 13.94 kt for 24 h,TC records from official agencies are used to assess the validity of TC-Rolling.
文摘The utilization of millimeter-wave frequencies and cognitive radio(CR)are promising ways to increase the spectral efficiency of wireless communication systems.However,conventional CR spectrum sensing techniques entail sampling the received signal at a Nyquist rate,and they are not viable for wideband signals due to their high cost.This paper expounds on how sub-Nyquist sampling in conjunction with deep learning can be leveraged to remove this limitation.To this end,we propose a multi-task learning(MTL)framework using convolutional neural networks for the joint inference of the underlying narrowband signal number,their modulation scheme,and their location in a wideband spectrum.We demonstrate the effectiveness of the proposed framework for real-world millimeter-wave wideband signals collected by physical devices,exhibiting a 91.7% accuracy in the joint inference task when considering up to two narrowband signals over a wideband spectrum.Ultimately,the proposed data-driven approach enables on-the-fly wideband spectrum sensing,combining accuracy,and computational efficiency,which are indispensable for CR and opportunistic networking.
基金Supported by the Scientific and Technological Innovation 2030-Major Project of New Generation Artificial Intelligence(2020AAA0109300)Science and Technology Commission of Shanghai Municipality(21DZ2203100)2023 Anhui Province Key Research and Development Plan Project-Special Project of Science and Technology Cooperation(2023i11020002)。
文摘In recent years,self-supervised learning has achieved great success in areas such as computer vision and natural language processing because it can mine supervised signals from unlabeled data and reduce the reliance on manual labels.However,the currently generated self-supervised signals are either neighbor discrimination or self-discrimination,and there is no model to integrate neighbor discrimination and self-discrimination.Based on this,this paper proposes Fu-Rec that integrates neighbor-discrimination contrastive learning and self-discrimination contrastive learning,which consists of three modules:(1)neighbor-discrimination contrastive learning,(2)selfdiscrimination contrastive learning,and(3)recommendation module.The neighbor-discrimination contrastive learning and selfdiscrimination contrastive learning tasks are used as auxiliary tasks to assist the recommendation task.The Fu-Rec model effectively utilizes the respective advantages of neighbor-discrimination and self-discrimination to consider the information of the user’s neighbors as well as the user and the item itself for the recommendation,which results in better performance of the recommendation module.Experimental results on several public datasets demonstrate the effectiveness of the Fu-Rec proposed in this paper.
基金financially supported by the National Natural Science Foundation of China(Grant Nos.42130810,42204135,42174170,and 42074165)the Natural Science Foundation of Hunan Province(Grant No.2023JJ40716)。
文摘Pre-stack seismic inversion is an effective way to investigate the characteristics of hydrocarbon-bearing reservoirs.Multi-parameter application is the key to identifying reservoir lithology and fluid in pre-stack inversion.However,multi-parameter inversion may bring coupling effects on the parameters and destabilize the inversion.In addition,the lateral recognition accuracy of geological structures receives great attention.To address these challenges,a multi-task learning network considering the angle-gather difference is proposed in this work.The deep learning network is usually assumed as a black box and it is unclear what it can learn.However,the introduction of angle-gather difference can force the deep learning network to focus on the lateral differences,thus improving the lateral accuracy of the prediction profile.The proposed deep learning network includes input and output blocks.First,angle gathers and the angle-gather difference are fed into two separate input blocks with Res Net architecture and Unet architecture,respectively.Then,three elastic parameters,including P-and S-wave velocities and density,are simultaneously predicted based on the idea of multi-task learning by using three separate output blocks with the same convolutional network layers.Experimental and field data tests demonstrate the effectiveness of the proposed method in improving the prediction accuracy of seismic elastic parameters.
基金supported by the National Natural Science Foundation of China(Nos.1187050492,12005303,and 12175170).
文摘The global nuclear mass based on the macroscopic-microscopic model was studied by applying a newly designed multi-task learning artificial neural network(MTL-ANN). First, the reported nuclear binding energies of 2095 nuclei(Z ≥ 8, N ≥ 8) released in the latest Atomic Mass Evaluation AME2020 and the deviations between the fitting result of the liquid drop model(LDM)and data from AME2020 for each nucleus were obtained.To compensate for the deviations and investigate the possible ignored physics in the LDM, the MTL-ANN method was introduced in the model. Compared to the single-task learning(STL) method, this new network has a powerful ability to simultaneously learn multi-nuclear properties,such as the binding energies and single neutron and proton separation energies. Moreover, it is highly effective in reducing the risk of overfitting and achieving better predictions. Consequently, good predictions can be obtained using this nuclear mass model for both the training and validation datasets and for the testing dataset. In detail, the global root mean square(RMS) of the binding energy is effectively reduced from approximately 2.4 MeV of LDM to the current 0.2 MeV, and the RMS of Sn, Spcan also reach approximately 0.2 MeV. Moreover, compared to STL, for the training and validation sets, 3-9% improvement can be achieved with the binding energy, and 20-30% improvement for S_(n), S_(p);for the testing sets, the reduction in deviations can even reach 30-40%, which significantly illustrates the advantage of the current MTL.
基金National Key R&D Program of China under Grant No.2019YFC1511005the National Natural Science Foundation of China under Grant Nos.51921006,52192661 and 52008138+2 种基金the China Postdoctoral Science Foundation under Grant Nos.BX20190102 and 2019M661286the Heilongjiang Natural Science Foundation under Grant No.LH2022E070the Heilongjiang Province Postdoctoral Science Foundation under Grant Nos.LBH-TZ2016 and LBH-Z19064。
文摘Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new tasks and datasets,which is inevitably time-consuming.This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components,three-type seismic damage,and four-type deterioration states.The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head,task-specific recognition subnetwork.The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures.The multi-head,task-specific recognition subnetwork consists of three individual self-attention pipelines,each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task.A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one.Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity.The results show that the proposed method can simultaneously recognize different structural components,seismic damage,and deterioration states,and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.
文摘There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them.However,the volume of internet data is beyond the processing capabilities of the current internet infrastructure.Therefore,engineering works using technology to organize and analyze information and extract useful information are interesting in both industry and academia.The goal of this paper is to explore the entity relationship based on deep learning,introduce semantic knowledge by using the prepared language model,develop an advanced entity relationship information extraction method by combining Robustly Optimized BERT Approach(RoBERTa)and multi-task learning,and combine the intelligent characters in the field of linguistic,called Robustly Optimized BERT Approach+Multi-Task Learning(RoBERTa+MTL).To improve the effectiveness of model interaction,multi-task teaching is used to implement the observation information of auxiliary tasks.Experimental results show that our method has achieved an accuracy of 88.95 entity relationship extraction,and a further it has achieved 86.35%of accuracy after being combined with multi-task learning.
基金supported by the Science and Technology Grant No.520120210003,Jibei Electric Power Company of the State Grid Corporation of China。
文摘Convective storms and lightning are among the most important weather phenomena that are challenging to forecast.In this study,a novel multi-task learning(MTL)encoder-decoder U-net neural network was developed to forecast convective storms and lightning with lead times for up to 90 min,using GOES-16 geostationary satellite infrared brightness temperatures(IRBTs),lightning flashes from Geostationary Lightning Mapper(GLM),and vertically integrated liquid(VIL)from Next Generation Weather Radar(NEXRAD).To cope with the heavily skewed distribution of lightning data,a spatiotemporal exponent-weighted loss function and log-transformed lightning normalization approach were developed.The effects of MTL,single-task learning(STL),and IRBTs as auxiliary input features on convection and lightning nowcasting were investigated.The results showed that normalizing the heavily skew-distributed lightning data along with a log-transformation dramatically outperforms the min-max normalization method for nowcasting an intense lightning event.The MTL model significantly outperformed the STL model for both lightning nowcasting and VIL nowcasting,particularly for intense lightning events.The MTL also helped delay the lightning forecast performance decay with the lead times.Furthermore,incorporating satellite IRBTs as auxiliary input features substantially improved lightning nowcasting,but produced little difference in VIL forecasting.Finally,the MTL model performed better for forecasting both lightning and the VIL of organized convective storms than for isolated cells.
基金This work was supported by the Major Project for New Generation of AI(No.2018AAA0100400)the National Natural Science Foundation of China(No.41706010)+1 种基金the Joint Fund of the Equipments Pre-Research and Ministry of Education of China(No.6141A020337)and the Fundamental Research Funds for the Central Universities of China.
文摘Sea state bias(SSB)is an important component of errors for the radar altimeter measurements of sea surface height(SSH).However,existing SSB estimation methods are almost all based on single-task learning(STL),where one model is built on the data from only one radar altimeter.In this paper,taking account of the data from multiple radar altimeters available,we introduced a multi-task learning method,called trace-norm regularized multi-task learning(TNR-MTL),for SSB estimation.Corresponding to each individual task,TNR-MLT involves only three parameters.Hence,it is easy to implement.More importantly,the convergence of TNR-MLT is theoretically guaranteed.Compared with the commonly used STL models,TNR-MTL can effectively utilize the shared information between data from multiple altimeters.During the training of TNR-MTL,we used the JASON-2 and JASON-3 cycle data to solve two correlated SSB estimation tasks.Then the optimal model was selected to estimate SSB on the JASON-2 and the HY-270-71 cycle intersection data.For the JSAON-2 cycle intersection data,the corrected variance(M)has been reduced by 0.60 cm^2 compared to the geophysical data records(GDR);while for the HY-2 cycle intersection data,M has been reduced by 1.30 cm^2 compared to GDR.Therefore,TNR-MTL is proved to be effective for the SSB estimation tasks.
基金This work is supported by State Grid Science and Technology Project under Grant No.520613180002,62061318C002the Fundamental Research Funds for the Central Universities(Grant No.HIT.NSRIF.201714)+4 种基金Weihai Science and Technology Development Program(2016DXGJMS15)Key Research and Development Program in Shandong Provincial(2017GGX90103)Sanming Science and Technology Project,Grant No.2015-G-6,Shandong province vocational education educational reform research project.Grant No.2017209Study and Development of Smart Agriculture Control System Based on Spark Big Data Decision(2017N0029)Jiangsu Province industrial Communication Technology Application Technology Innovation Team Project.
文摘With the rapid development of the mobile Internet,users generate massive data in different forms in social network every day,and different characteristics of users are reflected by these social media data.How to integrate multiple heterogeneous information and establish user profiles from multiple perspectives plays an important role in providing personalized services,marketing,and recommendation systems.In this paper,we propose Multi-source&Multi-task Learning for User Profiles in Social Network which integrates multiple social data sources and contains a multi-task learning framework to simultaneously predict various attributes of a user.Firstly,we design their own feature extraction models for multiple heterogeneous data sources.Secondly,we design a shared layer to fuse multiple heterogeneous data sources as general shared representation for multi-task learning.Thirdly,we design each task’s own unique presentation layer for discriminant output of specific-task.Finally,we design a weighted loss function to improve the learning efficiency and prediction accuracy of each task.Our experimental results on more than 5000 Sina Weibo users demonstrate that our approach outperforms state-of-the-art baselines for inferring gender,age and region of social media users.
文摘Buiding data-driven models using machine learning methods has gradually become a common approach for studying reservoir parameters.Among these methods,deep learning methods are highly effective.From the perspective of multi-task learning,this paper uses six types of logging data—acoustic logging(AC),gamma ray(GR),compensated neutron porosity(CNL),density(DEN),deep and shallow lateral resistivity(LLD)and shallow lateral resistivity(LLS)—that are inputs and three reservoir parameters that are outputs to build a porosity saturation permeability network(PSP-Net)that can predict porosity,saturation,and permeability values simultaneously.These logging data are obtained from 108 training wells in a medium₋low permeability oilfield block in the western district of China.PSP-Net method adopts a serial structure to realize transfer learning of reservoir-parameter characteristics.Compared with other existing methods at the stage of academic exploration to simulating industrial applications,the proposed method overcomes the disadvantages inherent in single-task learning reservoir-parameter prediction models,including easily overfitting and heavy model-training workload.Additionally,the proposed method demonstrates good anti-overfitting and generalization capabilities,integrating professional knowledge and experience.In 37 test wells,compared with the existing method,the proposed method exhibited an average error reduction of 10.44%,27.79%,and 28.83%from porosity,saturation,permeability calculation.The prediction and actual permeabilities are within one order of magnitude.The training on PSP-Net are simpler and more convenient than other single-task learning methods discussed in this paper.Furthermore,the findings of this paper can help in the re-examination of old oilfield wells and the completion of logging data.
文摘Aspect-based sentiment analysis(ABSA)is a fine-grained process.Its fundamental subtasks are aspect termextraction(ATE)and aspect polarity classification(APC),and these subtasks are dependent and closely related.However,most existing works on Arabic ABSA content separately address them,assume that aspect terms are preidentified,or use a pipeline model.Pipeline solutions design different models for each task,and the output from the ATE model is used as the input to the APC model,which may result in error propagation among different steps because APC is affected by ATE error.These methods are impractical for real-world scenarios where the ATE task is the base task for APC,and its result impacts the accuracy of APC.Thus,in this study,we focused on a multi-task learning model for Arabic ATE and APC in which the model is jointly trained on two subtasks simultaneously in a singlemodel.This paper integrates themulti-task model,namely Local Cotext Foucse-Aspect Term Extraction and Polarity classification(LCF-ATEPC)and Arabic Bidirectional Encoder Representation from Transformers(AraBERT)as a shred layer for Arabic contextual text representation.The LCF-ATEPC model is based on a multi-head selfattention and local context focus mechanism(LCF)to capture the interactive information between an aspect and its context.Moreover,data augmentation techniques are proposed based on state-of-the-art augmentation techniques(word embedding substitution with constraints and contextual embedding(AraBERT))to increase the diversity of the training dataset.This paper examined the effect of data augmentation on the multi-task model for Arabic ABSA.Extensive experiments were conducted on the original and combined datasets(merging the original and augmented datasets).Experimental results demonstrate that the proposed Multi-task model outperformed existing APC techniques.Superior results were obtained by AraBERT and LCF-ATEPC with fusion layer(AR-LCF-ATEPC-Fusion)and the proposed data augmentation word embedding-based method(FastText)on the combined dataset.
基金This paper is partly supported by the National Natural Science Foundation of China unde rGrants 61972057 and 62172059Hunan ProvincialNatural Science Foundation of China underGrant 2022JJ30623 and 2019JJ50287Scientific Research Fund of Hunan Provincial Education Department of China under Grant 21A0211 and 19A265。
文摘Prevailing linguistic steganalysis approaches focus on learning sensitive features to distinguish a particular category of steganographic texts from non-steganographic texts,by performing binary classification.While it remains an unsolved problem and poses a significant threat to the security of cyberspace when various categories of non-steganographic or steganographic texts coexist.In this paper,we propose a general linguistic steganalysis framework named LS-MTL,which introduces the idea of multi-task learning to deal with the classification of various categories of steganographic and non-steganographic texts.LS-MTL captures sensitive linguistic features from multiple related linguistic steganalysis tasks and can concurrently handle diverse tasks with a constructed model.In the proposed framework,convolutional neural networks(CNNs)are utilized as private base models to extract sensitive features for each steganalysis task.Besides,a shared CNN is built to capture potential interaction information and share linguistic features among all tasks.Finally,LS-MTL incorporates the private and shared sensitive features to identify the detected text as steganographic or non-steganographic.Experimental results demonstrate that the proposed framework LS-MTL outperforms the baseline in the multi-category linguistic steganalysis task,while average Acc,Pre,and Rec are increased by 0.5%,1.4%,and 0.4%,respectively.More ablation experimental results show that LS-MTL with the shared module has robust generalization capability and achieves good detection performance even in the case of spare data.
基金National Key R&D Program of China(No.2022ZD0118401).
文摘Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively utilize multi-modal remote sensing data to break through the performance bottleneck of single-modal interpretation.In addition,semantic segmentation and height estimation in remote sensing data are two tasks with strong correlation,but existing methods usually study individual tasks separately,which leads to high computational resource overhead.To this end,we propose a Multi-Task learning framework for Multi-Modal remote sensing images(MM_MT).Specifically,we design a Cross-Modal Feature Fusion(CMFF)method,which aggregates complementary information of different modalities to improve the accuracy of semantic segmentation and height estimation.Besides,a dual-stream multi-task learning method is introduced for Joint Semantic Segmentation and Height Estimation(JSSHE),extracting common features in a shared network to save time and resources,and then learning task-specific features in two task branches.Experimental results on the public multi-modal remote sensing image dataset Potsdam show that compared to training two tasks independently,multi-task learning saves 20%of training time and achieves competitive performance with mIoU of 83.02%for semantic segmentation and accuracy of 95.26%for height estimation.
