[Objective]Accurate prediction of tomato growth height is crucial for optimizing production environments in smart farming.However,current prediction methods predominantly rely on empirical,mechanistic,or learning-base...[Objective]Accurate prediction of tomato growth height is crucial for optimizing production environments in smart farming.However,current prediction methods predominantly rely on empirical,mechanistic,or learning-based models that utilize either images data or environmental data.These methods fail to fully leverage multi-modal data to capture the diverse aspects of plant growth comprehensively.[Methods]To address this limitation,a two-stage phenotypic feature extraction(PFE)model based on deep learning algorithm of recurrent neural network(RNN)and long short-term memory(LSTM)was developed.The model integrated environment and plant information to provide a holistic understanding of the growth process,emploied phenotypic and temporal feature extractors to comprehensively capture both types of features,enabled a deeper understanding of the interaction between tomato plants and their environment,ultimately leading to highly accurate predictions of growth height.[Results and Discussions]The experimental results showed the model's ef‐fectiveness:When predicting the next two days based on the past five days,the PFE-based RNN and LSTM models achieved mean absolute percentage error(MAPE)of 0.81%and 0.40%,respectively,which were significantly lower than the 8.00%MAPE of the large language model(LLM)and 6.72%MAPE of the Transformer-based model.In longer-term predictions,the 10-day prediction for 4 days ahead and the 30-day prediction for 12 days ahead,the PFE-RNN model continued to outperform the other two baseline models,with MAPE of 2.66%and 14.05%,respectively.[Conclusions]The proposed method,which leverages phenotypic-temporal collaboration,shows great potential for intelligent,data-driven management of tomato cultivation,making it a promising approach for enhancing the efficiency and precision of smart tomato planting management.展开更多
The sixth generation(6G)of mobile communication system is witnessing a new paradigm shift,i.e.,integrated sensing-communication system.A comprehensive dataset is a prerequisite for 6G integrated sensing-communication ...The sixth generation(6G)of mobile communication system is witnessing a new paradigm shift,i.e.,integrated sensing-communication system.A comprehensive dataset is a prerequisite for 6G integrated sensing-communication research.This paper develops a novel simulation dataset,named M3SC,for mixed multi-modal(MMM)sensing-communication integration,and the generation framework of the M3SC dataset is further given.To obtain multimodal sensory data in physical space and communication data in electromagnetic space,we utilize Air-Sim and WaveFarer to collect multi-modal sensory data and exploit Wireless InSite to collect communication data.Furthermore,the in-depth integration and precise alignment of AirSim,WaveFarer,andWireless InSite are achieved.The M3SC dataset covers various weather conditions,multiplex frequency bands,and different times of the day.Currently,the M3SC dataset contains 1500 snapshots,including 80 RGB images,160 depth maps,80 LiDAR point clouds,256 sets of mmWave waveforms with 8 radar point clouds,and 72 channel impulse response(CIR)matrices per snapshot,thus totaling 120,000 RGB images,240,000 depth maps,120,000 LiDAR point clouds,384,000 sets of mmWave waveforms with 12,000 radar point clouds,and 108,000 CIR matrices.The data processing result presents the multi-modal sensory information and communication channel statistical properties.Finally,the MMM sensing-communication application,which can be supported by the M3SC dataset,is discussed.展开更多
A novel face recognition method, which is a fusion of muhi-modal face parts based on Gabor feature （MMP-GF）, is proposed in this paper. Firstly, the bare face image detached from the normalized image was convolved w...A novel face recognition method, which is a fusion of muhi-modal face parts based on Gabor feature （MMP-GF）, is proposed in this paper. Firstly, the bare face image detached from the normalized image was convolved with a family of Gabor kernels, and then according to the face structure and the key-points locations, the calculated Gabor images were divided into five parts： Gabor face, Gabor eyebrow, Gabor eye, Gabor nose and Gabor mouth. After that multi-modal Gabor features were spatially partitioned into non-overlapping regions and the averages of regions were concatenated to be a low dimension feature vector, whose dimension was further reduced by principal component analysis （PCA）. In the decision level fusion, match results respectively calculated based on the five parts were combined according to linear discriminant analysis （LDA） and a normalized matching algorithm was used to improve the performance. Experiments on FERET database show that the proposed MMP-GF method achieves good robustness to the expression and age variations.展开更多
The unsupervised multi-modal image translation is an emerging domain of computer vision whose goal is to transform an image from the source domain into many diverse styles in the target domain.However,the multi-genera...The unsupervised multi-modal image translation is an emerging domain of computer vision whose goal is to transform an image from the source domain into many diverse styles in the target domain.However,the multi-generator mechanism is employed among the advanced approaches available to model different domain mappings,which results in inefficient training of neural networks and pattern collapse,leading to inefficient generation of image diversity.To address this issue,this paper introduces a multi-modal unsupervised image translation framework that uses a generator to perform multi-modal image translation.Specifically,firstly,the domain code is introduced in this paper to explicitly control the different generation tasks.Secondly,this paper brings in the squeeze-and-excitation(SE)mechanism and feature attention(FA)module.Finally,the model integrates multiple optimization objectives to ensure efficient multi-modal translation.This paper performs qualitative and quantitative experiments on multiple non-paired benchmark image translation datasets while demonstrating the benefits of the proposed method over existing technologies.Overall,experimental results have shown that the proposed method is versatile and scalable.展开更多
Machining process data is the core of computer aided process planning application systems.It is also provides essen- tial content for product life cycle engineering.The character of CAPP that supports product LCE and ...Machining process data is the core of computer aided process planning application systems.It is also provides essen- tial content for product life cycle engineering.The character of CAPP that supports product LCE and virtual manufacturing is an- alyzed.The structure and content of machining process data concerning green manufacturing is also examined.A logic model of Machining Process Data has been built based on an object oriented approach,using UML technology and a physical model of machin- ing process data that utilizes XML technology.To realize the integration of design and process,an approach based on graph-based volume decomposition was apposed.Instead,to solve the problem of generation in the machining process,case-based reasoning and rule-based reasoning have been applied synthetically.Finally,the integration framework and interface that deal with the CAPP integration with CAD,CAM,PDM,and ERP are discussed.展开更多
In light of degradation of particle filtering and robust weakness in the utilization of single feature tracking,this paper presents a kernel particle filtering tracking method based on multi-feature integration.In thi...In light of degradation of particle filtering and robust weakness in the utilization of single feature tracking,this paper presents a kernel particle filtering tracking method based on multi-feature integration.In this paper,a new weight upgrading method is given out during kernel particle filtering at first,and then robust tracking is realized by integrating color and texture features under the framework of kernel particle filtering.Space histogram and integral histogram is adopted to calculate color and texture features respectively.These two calculation methods effectively overcome their own defectiveness,and meanwhile,improve the real timing for particle filtering.This algorithm has also improved sampling effectiveness,resolved redundant calculation for particle filtering and degradation of particles.Finally,the experiment for target tracking is realized by using the method under complicated background and shelter.Experiment results show that the method can reliably and accurately track target and deal with target sheltering situation properly.展开更多
In geometry processing,symmetry research benefits from global geo-metric features of complete shapes,but the shape of an object captured in real-world applications is often incomplete due to the limited sensor resoluti...In geometry processing,symmetry research benefits from global geo-metric features of complete shapes,but the shape of an object captured in real-world applications is often incomplete due to the limited sensor resolution,single viewpoint,and occlusion.Different from the existing works predicting symmetry from the complete shape,we propose a learning approach for symmetry predic-tion based on a single RGB-D image.Instead of directly predicting the symmetry from incomplete shapes,our method consists of two modules,i.e.,the multi-mod-al feature fusion module and the detection-by-reconstruction module.Firstly,we build a channel-transformer network(CTN)to extract cross-fusion features from the RGB-D as the multi-modal feature fusion module,which helps us aggregate features from the color and the depth separately.Then,our self-reconstruction net-work based on a 3D variational auto-encoder(3D-VAE)takes the global geo-metric features as input,followed by a prediction symmetry network to detect the symmetry.Our experiments are conducted on three public datasets:ShapeNet,YCB,and ScanNet,we demonstrate that our method can produce reliable and accurate results.展开更多
In order to solve difficult detection of far and hard objects due to the sparseness and insufficient semantic information of LiDAR point cloud,a 3D object detection network with multi-modal data adaptive fusion is pro...In order to solve difficult detection of far and hard objects due to the sparseness and insufficient semantic information of LiDAR point cloud,a 3D object detection network with multi-modal data adaptive fusion is proposed,which makes use of multi-neighborhood information of voxel and image information.Firstly,design an improved ResNet that maintains the structure information of far and hard objects in low-resolution feature maps,which is more suitable for detection task.Meanwhile,semantema of each image feature map is enhanced by semantic information from all subsequent feature maps.Secondly,extract multi-neighborhood context information with different receptive field sizes to make up for the defect of sparseness of point cloud which improves the ability of voxel features to represent the spatial structure and semantic information of objects.Finally,propose a multi-modal feature adaptive fusion strategy which uses learnable weights to express the contribution of different modal features to the detection task,and voxel attention further enhances the fused feature expression of effective target objects.The experimental results on the KITTI benchmark show that this method outperforms VoxelNet with remarkable margins,i.e.increasing the AP by 8.78%and 5.49%on medium and hard difficulty levels.Meanwhile,our method achieves greater detection performance compared with many mainstream multi-modal methods,i.e.outperforming the AP by 1%compared with that of MVX-Net on medium and hard difficulty levels.展开更多
In allusion to the difficulty of integrating data with different models in integrating spatial information, the characteristics of raster structure, vector structure and mixed model were analyzed, and a hierarchical v...In allusion to the difficulty of integrating data with different models in integrating spatial information, the characteristics of raster structure, vector structure and mixed model were analyzed, and a hierarchical vector-raster integrative full feature model was put forward by integrating the advantage of vector and raster model and using the object-oriented method. The data structures of the four basic features, i.e. point, line, surface and solid, were described. An application was analyzed and described, and the characteristics of this model were described. In this model, all objects in the real world are divided into and described as features with hierarchy, and all the data are organized in vector. This model can describe data based on feature, field, network and other models, and avoid the disadvantage of inability to integrate data based on different models and perform spatial analysis on them in spatial information integration.展开更多
Visual Attention Prediction(VAP)is widely applied in GIS research,such as navigation task identification and driver assistance systems.Previous studies commonly took color information to detect the visual saliency of ...Visual Attention Prediction(VAP)is widely applied in GIS research,such as navigation task identification and driver assistance systems.Previous studies commonly took color information to detect the visual saliency of natural scene images.However,these studies rarely considered adaptively feature integration to different geospatial scenes in specific tasks.To better predict visual attention while driving tasks,in this paper,we firstly propose an Adaptive Feature Integration Fully Convolutional Network(AdaFI-FCN)using Scene-Adaptive Weights(SAW)to integrate RGB-D,motion and semantic features.The quantitative comparison results on the DR(eye)VE dataset show that the proposed framework achieved the best accuracy and robustness performance compared with state-of-the-art models(AUC-Judd=0.971,CC=0.767,KL=1.046,SIM=0.579).In addition,the experimental results of the ablation study demonstrated the positive effect of the SAW method on the prediction robustness in response to scene changes.The proposed model has the potential to benefit adaptive VAP research in universal geospatial scenes,such as AR-aided navigation,indoor navigation,and street-view image reading.展开更多
Gesture recognition is used in many practical applications such as human-robot interaction, medical rehabilitation and sign language. With increasing motion sensor development, multiple data sources have become availa...Gesture recognition is used in many practical applications such as human-robot interaction, medical rehabilitation and sign language. With increasing motion sensor development, multiple data sources have become available, which leads to the rise of multi-modal gesture recognition. Since our previous approach to gesture recognition depends on a unimodal system, it is difficult to classify similar motion patterns. In order to solve this problem, a novel approach which integrates motion, audio and video models is proposed by using dataset captured by Kinect. The proposed system can recognize observed gestures by using three models. Recognition results of three models are integrated by using the proposed framework and the output becomes the final result. The motion and audio models are learned by using Hidden Markov Model. Random Forest which is the video classifier is used to learn the video model. In the experiments to test the performances of the proposed system, the motion and audio models most suitable for gesture recognition are chosen by varying feature vectors and learning methods. Additionally, the unimodal and multi-modal models are compared with respect to recognition accuracy. All the experiments are conducted on dataset provided by the competition organizer of MMGRC, which is a workshop for Multi-Modal Gesture Recognition Challenge. The comparison results show that the multi-modal model composed of three models scores the highest recognition rate. This improvement of recognition accuracy means that the complementary relationship among three models improves the accuracy of gesture recognition. The proposed system provides the application technology to understand human actions of daily life more precisely.展开更多
Feature recognition aims at extracting manufacturing features with geometrical information from solid model and is considered to be an efficient way of changing the interactive NC machining programming mode.Existing r...Feature recognition aims at extracting manufacturing features with geometrical information from solid model and is considered to be an efficient way of changing the interactive NC machining programming mode.Existing recognition methods have some disadvantages in practical applications.They can essentially handle prismatic components with regular shapes and are difficult to recognize the intersecting features and curved surfaces.Besides,the robustness of them is not strong enough.A new feature recognition approach is proposed based on the analysis of aircraft integral panels＇ geometry and machining characteristics.In this approach,the aircraft integral panel is divided into a number of local machining domains.The machining domains are extracted and recognized first by finding the principal face of machining domain and extracting the sides around the principal face.Then the machining domains are divided into various features in terms of the face type.The main sections of the proposed method are presented including the definition,classification and structure of machining domain,the relationship between machining domain and principal face loop,the rules of machining domains recognition,and the algorithm of machining feature recognition.In addition,a robotic feature recognition module is developed for aircraft integral panels and tested with several panels.Test results show that the strategy presented is robust and valid.Features extracted can be post processed and linked to various downstream applications.The approach is able to solve the difficulties in recognizing the aircraft integral panel＇s features and automatic obtaining the machining zone in NC programming,and can be used to further develop the automatic programming of NC machining.展开更多
In this paper, we present a novel and efficient scheme for extracting, indexing and retrieving color images. Our motivation was to reduce the space overhead of partition-based approaches taking advantage of the fact t...In this paper, we present a novel and efficient scheme for extracting, indexing and retrieving color images. Our motivation was to reduce the space overhead of partition-based approaches taking advantage of the fact that only a relatively low number of distinct values of a particular visual feature is present in most images. To extract color feature and build indices into our image database we take into consideration factors such as human color perception and perceptual range, and the image is partitioned into a set of regions by using a simple classifying scheme. The compact color feature vector and the spatial color histogram, which are extracted from the seqmented image region, are used for representing the color and spatial information in the image. We have also developed the region-based distance measures to compare the similarity of two images. Extensive tests on a large image collection were conducted to demonstrate the effectiveness of the proposed approach.展开更多
Recent advances in spatially resolved transcriptomic technologies have enabled unprecedented opportunities to elucidate tissue architecture and function in situ.Spatial transcriptomics can provide multimodal and compl...Recent advances in spatially resolved transcriptomic technologies have enabled unprecedented opportunities to elucidate tissue architecture and function in situ.Spatial transcriptomics can provide multimodal and complementary information simultaneously,including gene expression profiles,spatial locations,and histology images.However,most existing methods have limitations in efficiently utilizing spatial information and matched high-resolution histology images.To fully leverage the multi-modal information,we propose a SPAtially embedded Deep Attentional graph Clustering(SpaDAC)method to identify spatial domains while reconstructing denoised gene expression profiles.This method can efficiently learn the low-dimensional embeddings for spatial transcriptomics data by constructing multi-view graph modules to capture both spatial location connectives and morphological connectives.Benchmark results demonstrate that SpaDAC outperforms other algorithms on several recent spatial transcriptomics datasets.SpaDAC is a valuable tool for spatial domain detection,facilitating the comprehension of tissue architecture and cellular microenvironment.The source code of SpaDAC is freely available at Github(https://github.com/huoyuying/SpaDAC.git).展开更多
Painleve integrability has been tested for （2＋1）D Boussinesq equation with disturbance term using the standard WTC approach after introducing the Kruskai＇s simplification. New breather solitary solutions depending...Painleve integrability has been tested for （2＋1）D Boussinesq equation with disturbance term using the standard WTC approach after introducing the Kruskai＇s simplification. New breather solitary solutions depending on constant equilibrium solution are obtained by using Extended Homoclinic Test Method. Moreover, the spatiotemporal feature of breather solitary wave is exhibited.展开更多
A blank panel design algorithm based on feature mapping methods for integral wing skin panels with supercritical airfoil surface is presented.The model of a wing panel is decomposed into features,and features of the p...A blank panel design algorithm based on feature mapping methods for integral wing skin panels with supercritical airfoil surface is presented.The model of a wing panel is decomposed into features,and features of the panel are decomposed into information of location,direction,dimension and Boolean types.Features are mapped into the plane through optimal surface development algorithm.The plane panel is modeled by rebuilding the mapped features.Blanks of shot-peen forming panels are designed to identify the effectiveness of the methods.展开更多
Due to the small size,variety,and high degree of mixing of herbaceous vegetation,remote sensing-based identification of grassland types primarily focuses on extracting major grassland categories,lacking detailed depic...Due to the small size,variety,and high degree of mixing of herbaceous vegetation,remote sensing-based identification of grassland types primarily focuses on extracting major grassland categories,lacking detailed depiction.This limitation significantly hampers the development of effective evaluation and fine supervision for the rational utilization of grassland resources.To address this issue,this study concentrates on the representative grassland of Zhenglan Banner in Inner Mongolia as the study area.It integrates the strengths of Sentinel-1 and Sentinel-2 active-passive synergistic observations and introduces innovative object-oriented techniques for grassland type classification,thereby enhancing the accuracy and refinement of grassland classification.The results demonstrate the following:(1)To meet the supervision requirements of grassland resources,we propose a grassland type classification system based on remote sensing and the vegetation-habitat classification method,specifically applicable to natural grasslands in northern China.(2)By utilizing the high-spatial-resolution Normalized Difference Vegetation Index(NDVI)synthesized through the Spatial and Temporal Non-Local Filter-based Fusion Model(STNLFFM),we are able to capture the NDVI time profiles of grassland types,accurately extract vegetation phenological information within the year,and further enhance the temporal resolution.(3)The integration of multi-seasonal spectral,polarization,and phenological characteristics significantly improves the classification accuracy of grassland types.The overall accuracy reaches 82.61%,with a kappa coefficient of 0.79.Compared to using only multi-seasonal spectral features,the accuracy and kappa coefficient have improved by 15.94%and 0.19,respectively.Notably,the accuracy improvement of the gently sloping steppe is the highest,exceeding 38%.(4)Sandy grassland is the most widespread in the study area,and the growth season of grassland vegetation mainly occurs from May to September.The sandy meadow exhibits a longer growing season compared with typical grassland and meadow,and the distinct differences in phenological characteristics contribute to the accurate identification of various grassland types.展开更多
The multi-modal characteristics of mineral particles play a pivotal role in enhancing the classification accuracy,which is critical for obtaining a profound understanding of the Earth's composition and ensuring ef...The multi-modal characteristics of mineral particles play a pivotal role in enhancing the classification accuracy,which is critical for obtaining a profound understanding of the Earth's composition and ensuring effective exploitation utilization of its resources.However,the existing methods for classifying mineral particles do not fully utilize these multi-modal features,thereby limiting the classification accuracy.Furthermore,when conventional multi-modal image classification methods are applied to planepolarized and cross-polarized sequence images of mineral particles,they encounter issues such as information loss,misaligned features,and challenges in spatiotemporal feature extraction.To address these challenges,we propose a multi-modal mineral particle polarization image classification network(MMGC-Net)for precise mineral particle classification.Initially,MMGC-Net employs a two-dimensional(2D)backbone network with shared parameters to extract features from two types of polarized images to ensure feature alignment.Subsequently,a cross-polarized intra-modal feature fusion module is designed to refine the spatiotemporal features from the extracted features of the cross-polarized sequence images.Ultimately,the inter-modal feature fusion module integrates the two types of modal features to enhance the classification precision.Quantitative and qualitative experimental results indicate that when compared with the current state-of-the-art multi-modal image classification methods,MMGC-Net demonstrates marked superiority in terms of mineral particle multi-modal feature learning and four classification evaluation metrics.It also demonstrates better stability than the existing models.展开更多
To address the challenge of missing modal information in entity alignment and to mitigate information loss or bias arising frommodal heterogeneity during fusion,while also capturing shared information acrossmodalities...To address the challenge of missing modal information in entity alignment and to mitigate information loss or bias arising frommodal heterogeneity during fusion,while also capturing shared information acrossmodalities,this paper proposes a Multi-modal Pre-synergistic Entity Alignmentmodel based on Cross-modalMutual Information Strategy Optimization(MPSEA).The model first employs independent encoders to process multi-modal features,including text,images,and numerical values.Next,a multi-modal pre-synergistic fusion mechanism integrates graph structural and visual modal features into the textual modality as preparatory information.This pre-fusion strategy enables unified perception of heterogeneous modalities at the model’s initial stage,reducing discrepancies during the fusion process.Finally,using cross-modal deep perception reinforcement learning,the model achieves adaptive multilevel feature fusion between modalities,supporting learningmore effective alignment strategies.Extensive experiments on multiple public datasets show that the MPSEA method achieves gains of up to 7% in Hits@1 and 8.2% in MRR on the FBDB15K dataset,and up to 9.1% in Hits@1 and 7.7% in MRR on the FBYG15K dataset,compared to existing state-of-the-art methods.These results confirm the effectiveness of the proposed model.展开更多
文摘[Objective]Accurate prediction of tomato growth height is crucial for optimizing production environments in smart farming.However,current prediction methods predominantly rely on empirical,mechanistic,or learning-based models that utilize either images data or environmental data.These methods fail to fully leverage multi-modal data to capture the diverse aspects of plant growth comprehensively.[Methods]To address this limitation,a two-stage phenotypic feature extraction(PFE)model based on deep learning algorithm of recurrent neural network(RNN)and long short-term memory(LSTM)was developed.The model integrated environment and plant information to provide a holistic understanding of the growth process,emploied phenotypic and temporal feature extractors to comprehensively capture both types of features,enabled a deeper understanding of the interaction between tomato plants and their environment,ultimately leading to highly accurate predictions of growth height.[Results and Discussions]The experimental results showed the model's ef‐fectiveness:When predicting the next two days based on the past five days,the PFE-based RNN and LSTM models achieved mean absolute percentage error(MAPE)of 0.81%and 0.40%,respectively,which were significantly lower than the 8.00%MAPE of the large language model(LLM)and 6.72%MAPE of the Transformer-based model.In longer-term predictions,the 10-day prediction for 4 days ahead and the 30-day prediction for 12 days ahead,the PFE-RNN model continued to outperform the other two baseline models,with MAPE of 2.66%and 14.05%,respectively.[Conclusions]The proposed method,which leverages phenotypic-temporal collaboration,shows great potential for intelligent,data-driven management of tomato cultivation,making it a promising approach for enhancing the efficiency and precision of smart tomato planting management.
基金This work was supported in part by the Ministry National Key Research and Development Project(Grant No.2020AAA0108101)the National Natural Science Foundation of China(Grants No.62125101,62341101,62001018,and 62301011)+1 种基金Shandong Natural Science Foundation(Grant No.ZR2023YQ058)the New Cornerstone Science Foundation through the XPLORER PRIZE.The authors would like to thank Mengyuan Lu and Zengrui Han for their help in the construction of electromagnetic space in Wireless InSite simulation platform and Weibo Wen,Qi Duan,and Yong Yu for their help in the construction of phys ical space in AirSim simulation platform.
文摘The sixth generation(6G)of mobile communication system is witnessing a new paradigm shift,i.e.,integrated sensing-communication system.A comprehensive dataset is a prerequisite for 6G integrated sensing-communication research.This paper develops a novel simulation dataset,named M3SC,for mixed multi-modal(MMM)sensing-communication integration,and the generation framework of the M3SC dataset is further given.To obtain multimodal sensory data in physical space and communication data in electromagnetic space,we utilize Air-Sim and WaveFarer to collect multi-modal sensory data and exploit Wireless InSite to collect communication data.Furthermore,the in-depth integration and precise alignment of AirSim,WaveFarer,andWireless InSite are achieved.The M3SC dataset covers various weather conditions,multiplex frequency bands,and different times of the day.Currently,the M3SC dataset contains 1500 snapshots,including 80 RGB images,160 depth maps,80 LiDAR point clouds,256 sets of mmWave waveforms with 8 radar point clouds,and 72 channel impulse response(CIR)matrices per snapshot,thus totaling 120,000 RGB images,240,000 depth maps,120,000 LiDAR point clouds,384,000 sets of mmWave waveforms with 12,000 radar point clouds,and 108,000 CIR matrices.The data processing result presents the multi-modal sensory information and communication channel statistical properties.Finally,the MMM sensing-communication application,which can be supported by the M3SC dataset,is discussed.
基金Supported by the National Key Technology R&D Program (No. 2006BAK08B07)
文摘A novel face recognition method, which is a fusion of muhi-modal face parts based on Gabor feature （MMP-GF）, is proposed in this paper. Firstly, the bare face image detached from the normalized image was convolved with a family of Gabor kernels, and then according to the face structure and the key-points locations, the calculated Gabor images were divided into five parts： Gabor face, Gabor eyebrow, Gabor eye, Gabor nose and Gabor mouth. After that multi-modal Gabor features were spatially partitioned into non-overlapping regions and the averages of regions were concatenated to be a low dimension feature vector, whose dimension was further reduced by principal component analysis （PCA）. In the decision level fusion, match results respectively calculated based on the five parts were combined according to linear discriminant analysis （LDA） and a normalized matching algorithm was used to improve the performance. Experiments on FERET database show that the proposed MMP-GF method achieves good robustness to the expression and age variations.
基金the National Natural Science Foundation of China(No.61976080)the Academic Degrees&Graduate Education Reform Project of Henan Province(No.2021SJGLX195Y)+1 种基金the Teaching Reform Research and Practice Project of Henan Undergraduate Universities(No.2022SYJXLX008)the Key Project on Research and Practice of Henan University Graduate Education and Teaching Reform(No.YJSJG2023XJ006)。
文摘The unsupervised multi-modal image translation is an emerging domain of computer vision whose goal is to transform an image from the source domain into many diverse styles in the target domain.However,the multi-generator mechanism is employed among the advanced approaches available to model different domain mappings,which results in inefficient training of neural networks and pattern collapse,leading to inefficient generation of image diversity.To address this issue,this paper introduces a multi-modal unsupervised image translation framework that uses a generator to perform multi-modal image translation.Specifically,firstly,the domain code is introduced in this paper to explicitly control the different generation tasks.Secondly,this paper brings in the squeeze-and-excitation(SE)mechanism and feature attention(FA)module.Finally,the model integrates multiple optimization objectives to ensure efficient multi-modal translation.This paper performs qualitative and quantitative experiments on multiple non-paired benchmark image translation datasets while demonstrating the benefits of the proposed method over existing technologies.Overall,experimental results have shown that the proposed method is versatile and scalable.
文摘Machining process data is the core of computer aided process planning application systems.It is also provides essen- tial content for product life cycle engineering.The character of CAPP that supports product LCE and virtual manufacturing is an- alyzed.The structure and content of machining process data concerning green manufacturing is also examined.A logic model of Machining Process Data has been built based on an object oriented approach,using UML technology and a physical model of machin- ing process data that utilizes XML technology.To realize the integration of design and process,an approach based on graph-based volume decomposition was apposed.Instead,to solve the problem of generation in the machining process,case-based reasoning and rule-based reasoning have been applied synthetically.Finally,the integration framework and interface that deal with the CAPP integration with CAD,CAM,PDM,and ERP are discussed.
基金Sponsored by Natural Science Foundation of Heilongjiang Province of China(Grant No.QC2001C060)the Science and Technology Research Projectsin Office of Education of Heilongjiang province(Grant No.11531307)
文摘In light of degradation of particle filtering and robust weakness in the utilization of single feature tracking,this paper presents a kernel particle filtering tracking method based on multi-feature integration.In this paper,a new weight upgrading method is given out during kernel particle filtering at first,and then robust tracking is realized by integrating color and texture features under the framework of kernel particle filtering.Space histogram and integral histogram is adopted to calculate color and texture features respectively.These two calculation methods effectively overcome their own defectiveness,and meanwhile,improve the real timing for particle filtering.This algorithm has also improved sampling effectiveness,resolved redundant calculation for particle filtering and degradation of particles.Finally,the experiment for target tracking is realized by using the method under complicated background and shelter.Experiment results show that the method can reliably and accurately track target and deal with target sheltering situation properly.
文摘In geometry processing,symmetry research benefits from global geo-metric features of complete shapes,but the shape of an object captured in real-world applications is often incomplete due to the limited sensor resolution,single viewpoint,and occlusion.Different from the existing works predicting symmetry from the complete shape,we propose a learning approach for symmetry predic-tion based on a single RGB-D image.Instead of directly predicting the symmetry from incomplete shapes,our method consists of two modules,i.e.,the multi-mod-al feature fusion module and the detection-by-reconstruction module.Firstly,we build a channel-transformer network(CTN)to extract cross-fusion features from the RGB-D as the multi-modal feature fusion module,which helps us aggregate features from the color and the depth separately.Then,our self-reconstruction net-work based on a 3D variational auto-encoder(3D-VAE)takes the global geo-metric features as input,followed by a prediction symmetry network to detect the symmetry.Our experiments are conducted on three public datasets:ShapeNet,YCB,and ScanNet,we demonstrate that our method can produce reliable and accurate results.
基金National Youth Natural Science Foundation of China(No.61806006)Innovation Program for Graduate of Jiangsu Province(No.KYLX160-781)Jiangsu University Superior Discipline Construction Project。
文摘In order to solve difficult detection of far and hard objects due to the sparseness and insufficient semantic information of LiDAR point cloud,a 3D object detection network with multi-modal data adaptive fusion is proposed,which makes use of multi-neighborhood information of voxel and image information.Firstly,design an improved ResNet that maintains the structure information of far and hard objects in low-resolution feature maps,which is more suitable for detection task.Meanwhile,semantema of each image feature map is enhanced by semantic information from all subsequent feature maps.Secondly,extract multi-neighborhood context information with different receptive field sizes to make up for the defect of sparseness of point cloud which improves the ability of voxel features to represent the spatial structure and semantic information of objects.Finally,propose a multi-modal feature adaptive fusion strategy which uses learnable weights to express the contribution of different modal features to the detection task,and voxel attention further enhances the fused feature expression of effective target objects.The experimental results on the KITTI benchmark show that this method outperforms VoxelNet with remarkable margins,i.e.increasing the AP by 8.78%and 5.49%on medium and hard difficulty levels.Meanwhile,our method achieves greater detection performance compared with many mainstream multi-modal methods,i.e.outperforming the AP by 1%compared with that of MVX-Net on medium and hard difficulty levels.
基金Project (40473029) supported bythe National Natural Science Foundation of China project (04JJ3046) supported bytheNatural Science Foundation of Hunan Province , China
文摘In allusion to the difficulty of integrating data with different models in integrating spatial information, the characteristics of raster structure, vector structure and mixed model were analyzed, and a hierarchical vector-raster integrative full feature model was put forward by integrating the advantage of vector and raster model and using the object-oriented method. The data structures of the four basic features, i.e. point, line, surface and solid, were described. An application was analyzed and described, and the characteristics of this model were described. In this model, all objects in the real world are divided into and described as features with hierarchy, and all the data are organized in vector. This model can describe data based on feature, field, network and other models, and avoid the disadvantage of inability to integrate data based on different models and perform spatial analysis on them in spatial information integration.
基金supported by the National Natural Science Foundation of China(NSFC)under Grant No.42230103the State Key Laboratory of Geographic Information Engineering and the Key Laboratory of Surveying and Mapping Science and Geospatial Information Technology of the Ministry of Natural Resources Jointly Funded Project under Grant No.2021-04-03.
文摘Visual Attention Prediction(VAP)is widely applied in GIS research,such as navigation task identification and driver assistance systems.Previous studies commonly took color information to detect the visual saliency of natural scene images.However,these studies rarely considered adaptively feature integration to different geospatial scenes in specific tasks.To better predict visual attention while driving tasks,in this paper,we firstly propose an Adaptive Feature Integration Fully Convolutional Network(AdaFI-FCN)using Scene-Adaptive Weights(SAW)to integrate RGB-D,motion and semantic features.The quantitative comparison results on the DR(eye)VE dataset show that the proposed framework achieved the best accuracy and robustness performance compared with state-of-the-art models(AUC-Judd=0.971,CC=0.767,KL=1.046,SIM=0.579).In addition,the experimental results of the ablation study demonstrated the positive effect of the SAW method on the prediction robustness in response to scene changes.The proposed model has the potential to benefit adaptive VAP research in universal geospatial scenes,such as AR-aided navigation,indoor navigation,and street-view image reading.
基金Supported by Grant-in-Aid for Young Scientists(A)(Grant No.26700021)Japan Society for the Promotion of Science and Strategic Information and Communications R&D Promotion Programme(Grant No.142103011)Ministry of Internal Affairs and Communications
文摘Gesture recognition is used in many practical applications such as human-robot interaction, medical rehabilitation and sign language. With increasing motion sensor development, multiple data sources have become available, which leads to the rise of multi-modal gesture recognition. Since our previous approach to gesture recognition depends on a unimodal system, it is difficult to classify similar motion patterns. In order to solve this problem, a novel approach which integrates motion, audio and video models is proposed by using dataset captured by Kinect. The proposed system can recognize observed gestures by using three models. Recognition results of three models are integrated by using the proposed framework and the output becomes the final result. The motion and audio models are learned by using Hidden Markov Model. Random Forest which is the video classifier is used to learn the video model. In the experiments to test the performances of the proposed system, the motion and audio models most suitable for gesture recognition are chosen by varying feature vectors and learning methods. Additionally, the unimodal and multi-modal models are compared with respect to recognition accuracy. All the experiments are conducted on dataset provided by the competition organizer of MMGRC, which is a workshop for Multi-Modal Gesture Recognition Challenge. The comparison results show that the multi-modal model composed of three models scores the highest recognition rate. This improvement of recognition accuracy means that the complementary relationship among three models improves the accuracy of gesture recognition. The proposed system provides the application technology to understand human actions of daily life more precisely.
文摘Feature recognition aims at extracting manufacturing features with geometrical information from solid model and is considered to be an efficient way of changing the interactive NC machining programming mode.Existing recognition methods have some disadvantages in practical applications.They can essentially handle prismatic components with regular shapes and are difficult to recognize the intersecting features and curved surfaces.Besides,the robustness of them is not strong enough.A new feature recognition approach is proposed based on the analysis of aircraft integral panels＇ geometry and machining characteristics.In this approach,the aircraft integral panel is divided into a number of local machining domains.The machining domains are extracted and recognized first by finding the principal face of machining domain and extracting the sides around the principal face.Then the machining domains are divided into various features in terms of the face type.The main sections of the proposed method are presented including the definition,classification and structure of machining domain,the relationship between machining domain and principal face loop,the rules of machining domains recognition,and the algorithm of machining feature recognition.In addition,a robotic feature recognition module is developed for aircraft integral panels and tested with several panels.Test results show that the strategy presented is robust and valid.Features extracted can be post processed and linked to various downstream applications.The approach is able to solve the difficulties in recognizing the aircraft integral panel＇s features and automatic obtaining the machining zone in NC programming,and can be used to further develop the automatic programming of NC machining.
文摘In this paper, we present a novel and efficient scheme for extracting, indexing and retrieving color images. Our motivation was to reduce the space overhead of partition-based approaches taking advantage of the fact that only a relatively low number of distinct values of a particular visual feature is present in most images. To extract color feature and build indices into our image database we take into consideration factors such as human color perception and perceptual range, and the image is partitioned into a set of regions by using a simple classifying scheme. The compact color feature vector and the spatial color histogram, which are extracted from the seqmented image region, are used for representing the color and spatial information in the image. We have also developed the region-based distance measures to compare the similarity of two images. Extensive tests on a large image collection were conducted to demonstrate the effectiveness of the proposed approach.
基金supported by National Natural Science Foundation of China(62003028).X.L.was supported by a Scholarship from the China Scholarship Council.
文摘Recent advances in spatially resolved transcriptomic technologies have enabled unprecedented opportunities to elucidate tissue architecture and function in situ.Spatial transcriptomics can provide multimodal and complementary information simultaneously,including gene expression profiles,spatial locations,and histology images.However,most existing methods have limitations in efficiently utilizing spatial information and matched high-resolution histology images.To fully leverage the multi-modal information,we propose a SPAtially embedded Deep Attentional graph Clustering(SpaDAC)method to identify spatial domains while reconstructing denoised gene expression profiles.This method can efficiently learn the low-dimensional embeddings for spatial transcriptomics data by constructing multi-view graph modules to capture both spatial location connectives and morphological connectives.Benchmark results demonstrate that SpaDAC outperforms other algorithms on several recent spatial transcriptomics datasets.SpaDAC is a valuable tool for spatial domain detection,facilitating the comprehension of tissue architecture and cellular microenvironment.The source code of SpaDAC is freely available at Github(https://github.com/huoyuying/SpaDAC.git).
基金Supported by National Natural Science Foundation of China under Grant Nos. 11061028, 11261049, Yunnan Natural Science Foundation under Grant Nos. 2010CD086 and 2011Y012 and Qujing Normal University Natural Science Foundation under Grant Nos. 2009ZD002 and 2012QN016
文摘Painleve integrability has been tested for （2＋1）D Boussinesq equation with disturbance term using the standard WTC approach after introducing the Kruskai＇s simplification. New breather solitary solutions depending on constant equilibrium solution are obtained by using Extended Homoclinic Test Method. Moreover, the spatiotemporal feature of breather solitary wave is exhibited.
文摘A blank panel design algorithm based on feature mapping methods for integral wing skin panels with supercritical airfoil surface is presented.The model of a wing panel is decomposed into features,and features of the panel are decomposed into information of location,direction,dimension and Boolean types.Features are mapped into the plane through optimal surface development algorithm.The plane panel is modeled by rebuilding the mapped features.Blanks of shot-peen forming panels are designed to identify the effectiveness of the methods.
基金supported by the National Natural Science Foundation of China[grant number 42001386,42271407]within the ESA-MOST China Dragon 5 Cooperation(ID:59313).
文摘Due to the small size,variety,and high degree of mixing of herbaceous vegetation,remote sensing-based identification of grassland types primarily focuses on extracting major grassland categories,lacking detailed depiction.This limitation significantly hampers the development of effective evaluation and fine supervision for the rational utilization of grassland resources.To address this issue,this study concentrates on the representative grassland of Zhenglan Banner in Inner Mongolia as the study area.It integrates the strengths of Sentinel-1 and Sentinel-2 active-passive synergistic observations and introduces innovative object-oriented techniques for grassland type classification,thereby enhancing the accuracy and refinement of grassland classification.The results demonstrate the following:(1)To meet the supervision requirements of grassland resources,we propose a grassland type classification system based on remote sensing and the vegetation-habitat classification method,specifically applicable to natural grasslands in northern China.(2)By utilizing the high-spatial-resolution Normalized Difference Vegetation Index(NDVI)synthesized through the Spatial and Temporal Non-Local Filter-based Fusion Model(STNLFFM),we are able to capture the NDVI time profiles of grassland types,accurately extract vegetation phenological information within the year,and further enhance the temporal resolution.(3)The integration of multi-seasonal spectral,polarization,and phenological characteristics significantly improves the classification accuracy of grassland types.The overall accuracy reaches 82.61%,with a kappa coefficient of 0.79.Compared to using only multi-seasonal spectral features,the accuracy and kappa coefficient have improved by 15.94%and 0.19,respectively.Notably,the accuracy improvement of the gently sloping steppe is the highest,exceeding 38%.(4)Sandy grassland is the most widespread in the study area,and the growth season of grassland vegetation mainly occurs from May to September.The sandy meadow exhibits a longer growing season compared with typical grassland and meadow,and the distinct differences in phenological characteristics contribute to the accurate identification of various grassland types.
基金supported by the National Natural Science Foundation of China(Grant Nos.62071315 and 62271336).
文摘The multi-modal characteristics of mineral particles play a pivotal role in enhancing the classification accuracy,which is critical for obtaining a profound understanding of the Earth's composition and ensuring effective exploitation utilization of its resources.However,the existing methods for classifying mineral particles do not fully utilize these multi-modal features,thereby limiting the classification accuracy.Furthermore,when conventional multi-modal image classification methods are applied to planepolarized and cross-polarized sequence images of mineral particles,they encounter issues such as information loss,misaligned features,and challenges in spatiotemporal feature extraction.To address these challenges,we propose a multi-modal mineral particle polarization image classification network(MMGC-Net)for precise mineral particle classification.Initially,MMGC-Net employs a two-dimensional(2D)backbone network with shared parameters to extract features from two types of polarized images to ensure feature alignment.Subsequently,a cross-polarized intra-modal feature fusion module is designed to refine the spatiotemporal features from the extracted features of the cross-polarized sequence images.Ultimately,the inter-modal feature fusion module integrates the two types of modal features to enhance the classification precision.Quantitative and qualitative experimental results indicate that when compared with the current state-of-the-art multi-modal image classification methods,MMGC-Net demonstrates marked superiority in terms of mineral particle multi-modal feature learning and four classification evaluation metrics.It also demonstrates better stability than the existing models.
基金partially supported by the National Natural Science Foundation of China under Grants 62471493 and 62402257(for conceptualization and investigation)partially supported by the Natural Science Foundation of Shandong Province,China under Grants ZR2023LZH017,ZR2024MF066,and 2023QF025(for formal analysis and validation)+1 种基金partially supported by the Open Foundation of Key Laboratory of Computing Power Network and Information Security,Ministry of Education,Qilu University of Technology(Shandong Academy of Sciences)under Grant 2023ZD010(for methodology and model design)partially supported by the Russian Science Foundation(RSF)Project under Grant 22-71-10095-P(for validation and results verification).
文摘To address the challenge of missing modal information in entity alignment and to mitigate information loss or bias arising frommodal heterogeneity during fusion,while also capturing shared information acrossmodalities,this paper proposes a Multi-modal Pre-synergistic Entity Alignmentmodel based on Cross-modalMutual Information Strategy Optimization(MPSEA).The model first employs independent encoders to process multi-modal features,including text,images,and numerical values.Next,a multi-modal pre-synergistic fusion mechanism integrates graph structural and visual modal features into the textual modality as preparatory information.This pre-fusion strategy enables unified perception of heterogeneous modalities at the model’s initial stage,reducing discrepancies during the fusion process.Finally,using cross-modal deep perception reinforcement learning,the model achieves adaptive multilevel feature fusion between modalities,supporting learningmore effective alignment strategies.Extensive experiments on multiple public datasets show that the MPSEA method achieves gains of up to 7% in Hits@1 and 8.2% in MRR on the FBDB15K dataset,and up to 9.1% in Hits@1 and 7.7% in MRR on the FBYG15K dataset,compared to existing state-of-the-art methods.These results confirm the effectiveness of the proposed model.
