As a supplementary of [Xu L. Front. Electr. Electron. Eng. China, 2010, 5（3）： 281-328], this paper outlines current status of efforts made on Bayesian Ying- Yang （BYY） harmony learning, plus gene analysis appli- ...As a supplementary of [Xu L. Front. Electr. Electron. Eng. China, 2010, 5（3）： 281-328], this paper outlines current status of efforts made on Bayesian Ying- Yang （BYY） harmony learning, plus gene analysis appli- cations. At the beginning, a bird＇s-eye view is provided via Gaussian mixture in comparison with typical learn- ing algorithms and model selection criteria. Particularly, semi-supervised learning is covered simply via choosing a scalar parameter. Then, essential topics and demand- ing issues about BYY system design and BYY harmony learning are systematically outlined, with a modern per- spective on Yin-Yang viewpoint discussed, another Yang factorization addressed, and coordinations across and within Ying-Yang summarized. The BYY system acts as a unified framework to accommodate unsupervised, su- pervised, and semi-supervised learning all in one formu- lation, while the best harmony learning provides novelty and strength to automatic model selection. Also, mathe- matical formulation of harmony functional has been ad- dressed as a unified scheme for measuring the proximity to be considered in a BYY system, and used as the best choice among others. Moreover, efforts are made on a number of learning tasks, including a mode-switching factor analysis proposed as a semi-blind learning frame- work for several types of independent factor analysis, a hidden Markov model （HMM） gated temporal fac- tor analysis suggested for modeling piecewise stationary temporal dependence, and a two-level hierarchical Gaus- sian mixture extended to cover semi-supervised learning, as well as a manifold learning modified to facilitate au- tomatic model selection. Finally, studies are applied to the problems of gene analysis, such as genome-wide asso- ciation, exome sequencing analysis, and gene transcrip- tional regulation.展开更多
One paper in a preceding issue of this journal has introduced the Bayesian Ying-Yang(BYY)harmony learning from a perspective of problem solving,parameter learning,and model selection.In a complementary role,the paper ...One paper in a preceding issue of this journal has introduced the Bayesian Ying-Yang(BYY)harmony learning from a perspective of problem solving,parameter learning,and model selection.In a complementary role,the paper provides further insights from another perspective that a co-dimensional matrix pair(shortly co-dim matrix pair)forms a building unit and a hierarchy of such building units sets up the BYY system.The BYY harmony learning is re-examined via exploring the nature of a co-dim matrix pair,which leads to improved learning performance with refined model selection criteria and a modified mechanism that coordinates automatic model selection and sparse learning.Besides updating typical algorithms of factor analysis(FA),binary FA(BFA),binary matrix factorization(BMF),and nonnegative matrix factorization(NMF)to share such a mechanism,we are also led to(a)a new parametrization that embeds a de-noise nature to Gaussian mixture and local FA(LFA);(b)an alternative formulation of graph Laplacian based linear manifold learning;(c)a codecomposition of data and covariance for learning regularization and data integration;and(d)a co-dim matrix pair based generalization of temporal FA and state space model.Moreover,with help of a co-dim matrix pair in Hadamard product,we are led to a semi-supervised formation for regression analysis and a semi-blind learning formation for temporal FA and state space model.Furthermore,we address that these advances provide with new tools for network biology studies,including learning transcriptional regulatory,Protein-Protein Interaction network alignment,and network integration.展开更多
Advances on bidirectional intelligence are overviewed along three threads,with extensions and new perspectives.The first thread is about bidirectional learning architecture,exploring five dualities that enable Lmser s...Advances on bidirectional intelligence are overviewed along three threads,with extensions and new perspectives.The first thread is about bidirectional learning architecture,exploring five dualities that enable Lmser six cognitive functions and provide new perspectives on which a lot of extensions and particularlly flexible Lmser are proposed.Interestingly,either or two of these dualities actually takes an important role in recent models such as U-net,ResNet,and Dense Net.The second thread is about bidirectional learning principles unified by best yIng-yAng(IA)harmony in BYY system.After getting insights on deep bidirectional learning from a bird-viewing on existing typical learning principles from one or both of the inward and outward directions,maximum likelihood,variational principle,and several other learning principles are summarised as exemplars of the BYY learning,with new perspectives on advanced topics.The third thread further proceeds to deep bidirectional intelligence,driven by long term dynamics(LTD)for parameter learning and short term dynamics(STD)for image thinking and rational thinking in harmony.Image thinking deals with information flow of continuously valued arrays and especially image sequence,as if thinking was displayed in the real world,exemplified by the flow from inward encoding/cognition to outward reconstruction/transformation performed in Lmser learning and BYY learning.In contrast,rational thinking handles symbolic strings or discretely valued vectors,performing uncertainty reasoning and problem solving.In particular,a general thesis is proposed for bidirectional intelligence,featured by BYY intelligence potential theory(BYY-IPT)and nine essential dualities in architecture,fundamentals,and implementation,respectively.Then,problems of combinatorial solving and uncertainty reasoning are investigated from this BYY IPT perspective.First,variants and extensions are suggested for AlphaGoZero like searching tasks,such as traveling salesman problem(TSP)and attributed graph matching(AGM)that are turned into Go like problems with help of a feature enrichment technique.Second,reasoning activities are summarized under guidance of BYY IPT from the aspects of constraint satisfaction,uncertainty propagation,and path or tree searching.Particularly,causal potential theory is proposed for discovering causal direction,with two roads developed for its implementation.展开更多
Radar high-resolution range profiles(HRRPs)are typical high-dimensional and interdimension dependently distributed data,the statistical modeling of which is a challenging task for HRRP-based target recognition.Supposi...Radar high-resolution range profiles(HRRPs)are typical high-dimensional and interdimension dependently distributed data,the statistical modeling of which is a challenging task for HRRP-based target recognition.Supposing that HRRP samples are independent and jointly Gaussian distributed,a recent work[Du L,Liu H W,Bao Z.IEEE Transactions on Signal Processing,2008,56(5):1931–1944]applied factor analysis(FA)to model HRRP data with a two-phase approach for model selection,which achieved satisfactory recognition performance.The theoretical analysis and experimental results reveal that there exists high temporal correlation among adjacent HRRPs.This paper is thus motivated to model the spatial and temporal structure of HRRP data simultaneously by employing temporal factor analysis(TFA)model.For a limited size of high-dimensional HRRP data,the two-phase approach for parameter learning and model selection suffers from intensive computation burden and deteriorated evaluation.To tackle these problems,this work adopts the Bayesian Ying-Yang(BYY)harmony learning that has automatic model selection ability during parameter learning.Experimental results show stepwise improved recognition and rejection performances from the twophase learning based FA,to the two-phase learning based TFA and to the BYY harmony learning based TFA with automatic model selection.In addition,adding many extra free parameters to the classic FA model and thus becoming even worse in identifiability,the model of a general linear dynamical system is even inferior to the classic FA model.展开更多
Three Bayesian related approaches,namely,variational Bayesian(VB),minimum message length(MML)and Bayesian Ying-Yang(BYY)harmony learning,have been applied to automatically determining an appropriate number of componen...Three Bayesian related approaches,namely,variational Bayesian(VB),minimum message length(MML)and Bayesian Ying-Yang(BYY)harmony learning,have been applied to automatically determining an appropriate number of components during learning Gaussian mixture model(GMM).This paper aims to provide a comparative investigation on these approaches with not only a Jeffreys prior but also a conjugate Dirichlet-Normal-Wishart(DNW)prior on GMM.In addition to adopting the existing algorithms either directly or with some modifications,the algorithm for VB with Jeffreys prior and the algorithm for BYY with DNW prior are developed in this paper to fill the missing gap.The performances of automatic model selection are evaluated through extensive experiments,with several empirical findings:1)Considering priors merely on the mixing weights,each of three approaches makes biased mistakes,while considering priors on all the parameters of GMM makes each approach reduce its bias and also improve its performance.2)As Jeffreys prior is replaced by the DNW prior,all the three approaches improve their performances.Moreover,Jeffreys prior makes MML slightly better than VB,while the DNW prior makes VB better than MML.3)As the hyperparameters of DNW prior are further optimized by each of its own learning principle,BYY improves its performances while VB and MML deteriorate their performances when there are too many free hyper-parameters.Actually,VB and MML lack a good guide for optimizing the hyper-parameters of DNW prior.4)BYY considerably outperforms both VB and MML for any type of priors and whether hyper-parameters are optimized.Being different from VB and MML that rely on appropriate priors to perform model selection,BYY does not highly depend on the type of priors.It has model selection ability even without priors and performs already very well with Jeffreys prior,and incrementally improves as Jeffreys prior is replaced by the DNW prior.Finally,all algorithms are applied on the Berkeley segmentation database of real world images.Again,BYY considerably outperforms both VB and MML,especially in detecting the objects of interest from a confusing background.展开更多
Cerebellar model articulation controller(CMAC)is a popular associative memory neural network that imitates human’s cerebellum,which allows it to learn fast and carry out local generalization efficiently.This research...Cerebellar model articulation controller(CMAC)is a popular associative memory neural network that imitates human’s cerebellum,which allows it to learn fast and carry out local generalization efficiently.This research aims to integrate evolutionary computation into fuzzy CMAC Bayesian Ying-Yang(FCMACBYY)learning,which is referred to as FCMAC-EBYY,to achieve a synergetic development in the search for optimal fuzzy sets and connection weights.Traditional evolutionary approaches are limited to small populations of short binary string length and as such are not suitable for neural network training,which involves a large searching space due to complex connections as well as real values.The methodology employed by FCMACEBYY is coevolution,in which a complex solution is decomposed into some pieces to be optimized in different populations/species and then assembled.The developed FCMAC-EBYY is compared with various neuro-fuzzy systems using a real application of traffic flow prediction.展开更多
文摘As a supplementary of [Xu L. Front. Electr. Electron. Eng. China, 2010, 5（3）： 281-328], this paper outlines current status of efforts made on Bayesian Ying- Yang （BYY） harmony learning, plus gene analysis appli- cations. At the beginning, a bird＇s-eye view is provided via Gaussian mixture in comparison with typical learn- ing algorithms and model selection criteria. Particularly, semi-supervised learning is covered simply via choosing a scalar parameter. Then, essential topics and demand- ing issues about BYY system design and BYY harmony learning are systematically outlined, with a modern per- spective on Yin-Yang viewpoint discussed, another Yang factorization addressed, and coordinations across and within Ying-Yang summarized. The BYY system acts as a unified framework to accommodate unsupervised, su- pervised, and semi-supervised learning all in one formu- lation, while the best harmony learning provides novelty and strength to automatic model selection. Also, mathe- matical formulation of harmony functional has been ad- dressed as a unified scheme for measuring the proximity to be considered in a BYY system, and used as the best choice among others. Moreover, efforts are made on a number of learning tasks, including a mode-switching factor analysis proposed as a semi-blind learning frame- work for several types of independent factor analysis, a hidden Markov model （HMM） gated temporal fac- tor analysis suggested for modeling piecewise stationary temporal dependence, and a two-level hierarchical Gaus- sian mixture extended to cover semi-supervised learning, as well as a manifold learning modified to facilitate au- tomatic model selection. Finally, studies are applied to the problems of gene analysis, such as genome-wide asso- ciation, exome sequencing analysis, and gene transcrip- tional regulation.
基金supported by the General Research Fund from Research Grant Council of Hong Kong(Project No.CUHK4180/10E)the National Basic Research Program of China(973 Program)(No.2009CB825404).
文摘One paper in a preceding issue of this journal has introduced the Bayesian Ying-Yang(BYY)harmony learning from a perspective of problem solving,parameter learning,and model selection.In a complementary role,the paper provides further insights from another perspective that a co-dimensional matrix pair(shortly co-dim matrix pair)forms a building unit and a hierarchy of such building units sets up the BYY system.The BYY harmony learning is re-examined via exploring the nature of a co-dim matrix pair,which leads to improved learning performance with refined model selection criteria and a modified mechanism that coordinates automatic model selection and sparse learning.Besides updating typical algorithms of factor analysis(FA),binary FA(BFA),binary matrix factorization(BMF),and nonnegative matrix factorization(NMF)to share such a mechanism,we are also led to(a)a new parametrization that embeds a de-noise nature to Gaussian mixture and local FA(LFA);(b)an alternative formulation of graph Laplacian based linear manifold learning;(c)a codecomposition of data and covariance for learning regularization and data integration;and(d)a co-dim matrix pair based generalization of temporal FA and state space model.Moreover,with help of a co-dim matrix pair in Hadamard product,we are led to a semi-supervised formation for regression analysis and a semi-blind learning formation for temporal FA and state space model.Furthermore,we address that these advances provide with new tools for network biology studies,including learning transcriptional regulatory,Protein-Protein Interaction network alignment,and network integration.
基金supported by the Zhi-Yuan Chair Professorship Start-up Grant (WF220103010) from Shanghai Jiao Tong University
文摘Advances on bidirectional intelligence are overviewed along three threads,with extensions and new perspectives.The first thread is about bidirectional learning architecture,exploring five dualities that enable Lmser six cognitive functions and provide new perspectives on which a lot of extensions and particularlly flexible Lmser are proposed.Interestingly,either or two of these dualities actually takes an important role in recent models such as U-net,ResNet,and Dense Net.The second thread is about bidirectional learning principles unified by best yIng-yAng(IA)harmony in BYY system.After getting insights on deep bidirectional learning from a bird-viewing on existing typical learning principles from one or both of the inward and outward directions,maximum likelihood,variational principle,and several other learning principles are summarised as exemplars of the BYY learning,with new perspectives on advanced topics.The third thread further proceeds to deep bidirectional intelligence,driven by long term dynamics(LTD)for parameter learning and short term dynamics(STD)for image thinking and rational thinking in harmony.Image thinking deals with information flow of continuously valued arrays and especially image sequence,as if thinking was displayed in the real world,exemplified by the flow from inward encoding/cognition to outward reconstruction/transformation performed in Lmser learning and BYY learning.In contrast,rational thinking handles symbolic strings or discretely valued vectors,performing uncertainty reasoning and problem solving.In particular,a general thesis is proposed for bidirectional intelligence,featured by BYY intelligence potential theory(BYY-IPT)and nine essential dualities in architecture,fundamentals,and implementation,respectively.Then,problems of combinatorial solving and uncertainty reasoning are investigated from this BYY IPT perspective.First,variants and extensions are suggested for AlphaGoZero like searching tasks,such as traveling salesman problem(TSP)and attributed graph matching(AGM)that are turned into Go like problems with help of a feature enrichment technique.Second,reasoning activities are summarized under guidance of BYY IPT from the aspects of constraint satisfaction,uncertainty propagation,and path or tree searching.Particularly,causal potential theory is proposed for discovering causal direction,with two roads developed for its implementation.
基金The work described in this paper was supported by a grant of the General Research Fund(GRF)from the Research Grant Council of the Hong Kong SAR(No.CUHK4180/10E)the National Natural Science Foundation of China(Grant Nos.60901067 and 61001212)+1 种基金Program for New Century Excellent Talents in University(No.NCET-09-0630)Program for Changjiang Scholars and Innovative Research Team in University(No.IRT0954),and the Fundamental Research Funds for the Central Universities.
文摘Radar high-resolution range profiles(HRRPs)are typical high-dimensional and interdimension dependently distributed data,the statistical modeling of which is a challenging task for HRRP-based target recognition.Supposing that HRRP samples are independent and jointly Gaussian distributed,a recent work[Du L,Liu H W,Bao Z.IEEE Transactions on Signal Processing,2008,56(5):1931–1944]applied factor analysis(FA)to model HRRP data with a two-phase approach for model selection,which achieved satisfactory recognition performance.The theoretical analysis and experimental results reveal that there exists high temporal correlation among adjacent HRRPs.This paper is thus motivated to model the spatial and temporal structure of HRRP data simultaneously by employing temporal factor analysis(TFA)model.For a limited size of high-dimensional HRRP data,the two-phase approach for parameter learning and model selection suffers from intensive computation burden and deteriorated evaluation.To tackle these problems,this work adopts the Bayesian Ying-Yang(BYY)harmony learning that has automatic model selection ability during parameter learning.Experimental results show stepwise improved recognition and rejection performances from the twophase learning based FA,to the two-phase learning based TFA and to the BYY harmony learning based TFA with automatic model selection.In addition,adding many extra free parameters to the classic FA model and thus becoming even worse in identifiability,the model of a general linear dynamical system is even inferior to the classic FA model.
基金The work described in this paper was supported by a grant of the General Research Fund(GRF)from the Research Grant Council of Hong Kong SAR(Project No.CUHK418011E).
文摘Three Bayesian related approaches,namely,variational Bayesian(VB),minimum message length(MML)and Bayesian Ying-Yang(BYY)harmony learning,have been applied to automatically determining an appropriate number of components during learning Gaussian mixture model(GMM).This paper aims to provide a comparative investigation on these approaches with not only a Jeffreys prior but also a conjugate Dirichlet-Normal-Wishart(DNW)prior on GMM.In addition to adopting the existing algorithms either directly or with some modifications,the algorithm for VB with Jeffreys prior and the algorithm for BYY with DNW prior are developed in this paper to fill the missing gap.The performances of automatic model selection are evaluated through extensive experiments,with several empirical findings:1)Considering priors merely on the mixing weights,each of three approaches makes biased mistakes,while considering priors on all the parameters of GMM makes each approach reduce its bias and also improve its performance.2)As Jeffreys prior is replaced by the DNW prior,all the three approaches improve their performances.Moreover,Jeffreys prior makes MML slightly better than VB,while the DNW prior makes VB better than MML.3)As the hyperparameters of DNW prior are further optimized by each of its own learning principle,BYY improves its performances while VB and MML deteriorate their performances when there are too many free hyper-parameters.Actually,VB and MML lack a good guide for optimizing the hyper-parameters of DNW prior.4)BYY considerably outperforms both VB and MML for any type of priors and whether hyper-parameters are optimized.Being different from VB and MML that rely on appropriate priors to perform model selection,BYY does not highly depend on the type of priors.It has model selection ability even without priors and performs already very well with Jeffreys prior,and incrementally improves as Jeffreys prior is replaced by the DNW prior.Finally,all algorithms are applied on the Berkeley segmentation database of real world images.Again,BYY considerably outperforms both VB and MML,especially in detecting the objects of interest from a confusing background.
基金This research was supported by the Ministry of Knowledge Economy(MKE),Korea,under the Information Technology Research Center(ITRC)supervised by the National IT Industry Promotion Agency(NIPA)(NIPA-2010-(C1090-1021-0002))It was sponsored by Daegu Gyungpook Development Institute 2010.
文摘Cerebellar model articulation controller(CMAC)is a popular associative memory neural network that imitates human’s cerebellum,which allows it to learn fast and carry out local generalization efficiently.This research aims to integrate evolutionary computation into fuzzy CMAC Bayesian Ying-Yang(FCMACBYY)learning,which is referred to as FCMAC-EBYY,to achieve a synergetic development in the search for optimal fuzzy sets and connection weights.Traditional evolutionary approaches are limited to small populations of short binary string length and as such are not suitable for neural network training,which involves a large searching space due to complex connections as well as real values.The methodology employed by FCMACEBYY is coevolution,in which a complex solution is decomposed into some pieces to be optimized in different populations/species and then assembled.The developed FCMAC-EBYY is compared with various neuro-fuzzy systems using a real application of traffic flow prediction.
