論文 on 村田昇

Object embedding using an information geometrical perspective

Fri, 21 Jul 2023 00:00:00 +0900

Sugiura, T., Murata, N. Information Geometry Volume 6, November 2023, pages 435-462 https://doi.org/10.1007/s41884-023-00114-z

Acquiring vector representations of objects is essential for applying machine learning, statistical inference, and visualization. Although various vector acquisition methods have been proposed considering the relationship between objects in target data, most of them are supposed to use only a specific relevance level. In real-world data, however, there are cases where multiple relationships are contained between objects, such as time-varying similarity in time-series data or various weighted edges on graph-structured data. In this paper, a vector acquisition method which assigns vectors in a single coordinate system to objects preserving the information given by multiple relations between objects is proposed. In the proposed method, a logarithmic bilinear model parameterized by representation vectors is utilized for approximating relations between objects based on a stochastic embedding idea. The inference algorithm proposed in this study is interpreted in terms of information geometry: the m-projection from the probability distribution constructed from observed relations on the model manifold and the e-mixture in the model manifold are alternately repeated to estimate the parameters. Finally, the performance of the proposed method is evaluated using artificial data, and a case study is conducted using real data.

Detecting cell assemblies by NMF-based clustering from calcium imaging data

Sun, 01 May 2022 00:00:00 +0900

#+begin_quote Nagayama, M., Aritake, T., Hino, H., Kanda, T., Miyazaki, T., Yanagisawa, M., Akaho, S., Murata, N. Neural Networks Volume 149, May 2022, pages 29-39 https://doi.org/10.1016/j.neunet.2022.01.023

A large number of neurons form cell assemblies that process information in the brain. Recent developments in measurement technology, one of which is calcium imaging, have made it possible to study cell assemblies. In this study, we aim to extract cell assemblies from calcium imaging data. We propose a clustering approach based on non-negative matrix factorization (NMF). The proposed approach first obtains a similarity matrix between neurons by NMF and then performs spectral clustering on it. The application of NMF entails the problem of model selection. The number of bases in NMF affects the result considerably, and a suitable selection method is yet to be established. We attempt to resolve this problem by model averaging with a newly defined estimator based on NMF. Experiments on simulated data suggest that the proposed approach is superior to conventional correlation-based clustering methods over a wide range of sampling rates. We also analyzed calcium imaging data of sleeping/waking mice and the results suggest that the size of the cell assembly depends on the degree and spatial extent of slow wave generation in the cerebral cortex.

Single-molecule localization by voxel-wise regression using convolutional neural network

Tue, 03 Nov 2020 00:00:00 +0900

Aritake, T., Hino, H., Namiki, S., Asanuma, D., Hirose, K., Murata, N. Results in Optics Volume 1, November 2020, 100019 https://doi.org/10.1016/j.rio.2020.100019

Single‐molecule localization microscopy is widely used in biological research for measuring the nanostructures of samples smaller than the diffraction limit. In this paper, a novel method for regression of the coordinates of molecules for multifocal plane microscopy is presented. A regression problem for the target space is decomposed into regression problems for small subsets of the target space. Then, a deep neural network is used to solve these problems. By decomposing the regression problem, a fully convolutional neural network can be used to solve the regression problems. The computation of the network is efficient, and a simple and parameter‐free loss function can be used to train the network. The proposed algorithm is validated by both simulated and real data obtained by quad‐plane microscopy.

Estimation of neural connections from partially observed neural spikes

Sat, 01 Dec 2018 00:00:00 +0900

Iwasaki, T., Hino, H., Tatsuno, M., Akaho, S., Murata, N. Neural Networks Volume 108, December 2018, Pages 172-191 https://doi.org/10.1016/j.neunet.2018.07.019

Plasticity is one of the most important properties of the nervous system, which enables animals to adjust their behavior to the ever-changing external environment. Changes in synaptic efficacy between neurons constitute one of the major mechanisms of plasticity. Therefore, estimation of neural connections is crucial for investigating information processing in the brain. Although many analysis methods have been proposed for this purpose, most of them suffer from one or all the following mathematical difficulties: (1) only partially observed neural activity is available; (2) correlations can include both direct and indirect pseudo-interactions; and (3) biological evidence that a neuron typically has only one type of connection (excitatory or inhibitory) should be considered. To overcome these difficulties, a novel probabilistic framework for estimating neural connections from partially observed spikes is proposed in this paper. First, based on the property of a sum of random variables, the proposed method estimates the influence of unobserved neurons on observed neurons and extracts only the correlations among observed neurons. Second, the relationship between pseudo-correlations and target connections is modeled by neural propagation in a multiplicative manner. Third, a novel information-theoretic framework is proposed for estimating neuron types. The proposed method was validated using spike data generated by artificial neural networks. In addition, it was applied to multi-unit data recorded from the CA1 area of a rat’s hippocampus. The results confirmed that our estimates are consistent with previous reports. These findings indicate that the proposed method is useful for extracting crucial interactions in neural signals as well as in other multi-probed point process data.

EEG dipole source localization with information criteria for multiple particle filters

Sat, 01 Dec 2018 00:00:00 +0900

Sonoda, S., Nakamura, K., Kaneda, Y., Hino, H., Akaho, S., Murata, N., Miyauchi, E., Kawasaki, M. Neural Networks Volume 108, December 2018, Pages 68–82 https://doi.org/10.1016/j.neunet.2018.08.008

Electroencephalography (EEG) is a non-invasive brain imaging technique that describes neural electrical activation with good temporal resolution. Source localization is required for clinical and functional interpretations of EEG signals, and most commonly is achieved via the dipole model; however, the number of dipoles in the brain should be determined for a reasonably accurate interpretation. In this paper, we propose a dipole source localization (DSL) method that adaptively estimates the dipole number by using a novel information criterion. Since the particle filtering process is nonparametric, it is not clear whether conventional information criteria such as Akaike’s information criterion (AIC) and Bayesian information criterion (BIC) can be applied. In the proposed method, multiple particle filters run in parallel, each of which respectively estimates the dipole locations and moments, with the assumption that the dipole number is known and fixed; at every time step, the most predictive particle filter is selected by using an information criterion tailored for particle filters. We tested the proposed information criterion first through experiments on artificial datasets; these experiments supported the hypothesis that the proposed information criterion would outperform both AIC and BIC. We then analyzed real human EEG datasets collected during an auditory short-term memory task using the proposed method. We found that the alpha-band dipoles were localized to the right and left auditory areas during the auditory short-term memory task, which is consistent with previous physiological findings. These analyses suggest the proposed information criterion can work well in both model and real-world situations.

Neural network with unbounded activation functions is universal approximator

Fri, 01 Sep 2017 00:00:00 +0900

Sonoda S., Murata, N. Applied and Computational Harmonic Analysis Volume 43, Issue 2, September 2017, Pages 233-268 https://doi.org/10.1016/j.acha.2015.12.005

This paper presents an investigation of the approximation property of neural networks with unbounded activation functions, such as the rectified linear unit (ReLU), which is the new de-facto standard of deep learning. The ReLU network can be analyzed by the ridgelet transform with respect to Lizorkin distributions. By showing three reconstruction formulas by using the Fourier slice theorem, the Radon transform, and Parseval’s relation, it is shown that a neural network with unbounded activation functions still satisfies the universal approximation property. As an additional consequence, the ridgelet transform, or the backprojection filter in the Radon domain, is what the network learns after backpropagation. Subject to a constructive admissibility condition, the trained network can be obtained by simply discretizing the ridgelet transform, without backpropagation. Numerical examples not only support the consistency of the admissibility condition but also imply that some non-admissible cases result in low-pass filtering.

An estimation of generalized Bradley-Terry models based on the em algorithm

Wed, 01 Jun 2011 00:00:00 +0900

Fujimoto, Y., Hino, H., Murata, N. Neural Computation Volume 23, Issue 6, June 2011, Pages 1623-1659 https://doi.org/10.1162/NECO_a_00129

The Bradley-Terry model is a statistical representation for one’s preference or ranking data by using pairwise comparison results of items. For estimation of the model, several methods based on the sum of weighted Kullback-Leibler divergences have been proposed from various contexts. The purpose of this letter is to interpret an estimation mechanism of the Bradley-Terry model from the viewpoint of flatness, a fundamental notion used in information geometry. Based on this point of view, a new estimation method is proposed on a framework of the em algorithm. The proposed method is different in its objective function from that of conventional methods, especially in treating unobserved comparisons, and it is consistently interpreted in a probability simplex. An estimation method with weight adaptation is also proposed from a viewpoint of the sensitivity. Experimental results show that the proposed method works appropriately, and weight adaptation improves accuracy of the estimate. © 2011 Massachusetts Institute of Technology.

Information geometry of U-Boost and Bregman divergence

Thu, 01 Jul 2004 00:00:00 +0900

Murata, N., Takenouchi, T., Kanamori, T., Eguchi, S. Neural Computation Volume 16, Issue 7, July 2004, Pages 1437-1481 https://doi.org/10.1162/089976604323057452

We aim at an extension of AdaBoost to U-Boost, in the paradigm to build a stronger classification machine from a set of weak learning machines. A geometric understanding of the Bregman divergence defined by a generic convex function U leads to the U-Boost method in the framework of information geometry extended to the space of the finite measures over a label set. We propose two versions of U-Boost learning algorithms by taking account of whether the domain is restricted to the space of probability functions. In the sequential step, we observe that the two adjacent and the initial classifiers are associated with a right triangle in the scale via the Bregman divergence, called the Pythagorean relation. This leads to a mild convergence property of the U-Boost algorithm as seen in the expectation-maximization algorithm. Statistical discussions for consistency and robustness elucidate the properties of the U-Boost methods based on a stochastic assumption for training data.

An approach to blind source separation based on temporal structure of speech signals

Mon, 01 Jan 2001 00:00:00 +0900

Murata, N., Ikeda, S., Ziehe, A. Neurocomputing Volume 41, Issue 1-4, 2001, Pages 1-24 https://doi.org/10.1016/S0925-2312(00)00345-3

In this paper, we introduce a new technique for blind source separation of speech signals. We focus on the temporal structure of the signals. The idea is to apply the decorrelation method proposed by Molgedey and Schuster in the time-frequency domain. Since we are applying separation algorithm on each frequency separately, we have to solve the amplitude and permutation ambiguity properly to reconstruct the separated signals. For solving the amplitude ambiguity, we use the matrix inversion and for the permutation ambiguity, we introduce a method based on the temporal structure of speech signals. We show some results of experiments with both artificially controlled data and speech data recorded in the real environment.

An integral representation of functions using three-layered networks and their approximation bounds

Thu, 01 Aug 1996 00:00:00 +0900

Murata, N. Neural Networks Volume 9, Issue 6, August 1996, Pages 947-956 https://doi.org/10.1016/0893-6080(96)00000-7

Neural networks are widely known to provide a method of approximating nonlinear functions. In order to clarify its approximation ability, a new theorem on an integral transform of ridge functions is presented. By using this theorem, an approximation bound, which evaluates the quantitative relationship between the approximation accuracy and the number of elements in the hidden layer, can be obtained. This result shows that the approximation accuracy depends on the smoothness of target functions. It also shows that the approximation methods which use ridge functions are free from the ‘curse of dimensionality’.

Network Information Criterion—Determining the number of hidden units for an artificial neural network model

Tue, 01 Nov 1994 00:00:00 +0900

Murata, N., Yoshizawa, S., Amari, S. IEEE Transactions on Neural Networks Volume 5, Issue 6, November 1994, Pages 865-872 https://doi.org/10.1109/72.329683

The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike’s information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.

論文 on 村田 昇