Sumio Watanabe, Ph.D.
Algebraic Geometry and Statistical Learning Theory
Sumio Watanabe,
Algebraic Geometry and Statistical Learning Theory, Cambridge University Press, 2009.
A learning machine or a statistical model is called singular if
its Fisher information matrix is singular. (A matrix A is singular if det(A)=0).
Almost all learning machines
which have hidden variables or hierarchical structure are singular.
In singular learning machines statistical theory of regular models does not hold
because a probability distribution is not defined by the maximal ideal.
We are now establishing a new learning theory based on algebraic geometry and
algebraic analysis.
Singular Learning Theory .
New Information
- The universal learning curve is proved under the renormalizable condition, even if
a true distribution is unrealizable and singular for a learning machine.
Sumio Watanabe,
``Asymptotic Learning Curve and Renormalizable Condition
in Statistical Learning Theory," arXiv:1001.2957, 2010. (2010/Jan/18).
-
International Workshop on Statistical-Mechanical Informatics
(Kyoto, Japan, 7-10/March/2010).
-
International Conference on Mathematical Quantum Field Theory and Renormalization Theory,
Chairs: Professor Takashi Hara, Professor Taku Matsui, and Professor Fumio Hiroshima.
, A Singular Limit Theorem in Statistical Learning Theory
(Kyushu, Japan, 26-29/Nov/2009).
-
The fifth Symposium on Singularities (IRMA Institute, France, 24-28/Aug/2009).
The PDF file .
- Professor Sturmfels visited Japan by Professor Takemura's invitation. (2009/July/6-10).
-
Equations of states hold even if the true distribution is not contained in a parametric model.
Sumio Watanabe,
Equations of States in Statistical Learning for a Nonparametrizable and Regular Case
, arXiv:0906.0211 (1/June/2009). This paper will appear in IEICE Transactions,
whose title is ``Equations of states in statistical learning for an unrealizable and regular case."
-
AIC in a regression problem is generalized so that it can be used in singular
statistical models.
Sumio Watanabe, A limit theorem in singular regression problem,
arXiv:0901.2376v1 (16/Jan/2009). This paper was accepted for publication in
Advanced Studies of Pure Mathematics ,
Also see here (9/Aug/2009).
-
Algebraic Statistics (15-18/Dec/2008)
-
2008-09 Program on Algebraic Methods in Systems Biology and Statistics
(14-17/September/2008)
-
Probabilistic Approach to Geometry
(July 28 - August 8 /2008).
-
In World Congress on Computational Intelligence, the following paper
was published, S.Watanabe,
"A formula of equations of states in
statistical estimation," Proc. of WCCI, Honkong, 2008, June, 1-6. (Jun/2008).
-
We found the
Equations of states in singular statistical estimation ,
by which we can predict both Bayes and Gibbs generalization errors
from Bayes and Gibbs training errors without any knowledge of
the true distribution (5/Dec/2007). This paper was submitted (5/Dec/2007),
and published in International Journal of Neural Networks, Vol.23, No.1, pp.20-34, 2010.
The review process was one year and eight months.
-
Niigata Workshop on Complex Geometry and Singularities (26/July/2007).
-
Professor Morihiko Saito in Research Institute for Mathematical Sciences
in Kyoto University proved the relation between
the real log canonical threshold and roots of b-function.
On real log canonical thresholds .
(17/July/2007).
-
S. Watanabe, ``Almost all learning machines are singular."
invited paper in IEEE FOCI 2007 (2007/4/5).
-
In The Clay Mathematics Institute,
a workshop of
Algebraic Statistics and Computational Biology (2005/Nov/11-15).
-
The following paper has been published in Neurocomputing.
geometrical methods in neural network and learning .
Sumio Watanabe,``Algebraic geometry of singular learning machines and symmetry of
generalization and training errors," Neurocomputing, Vol.67, pp.198-213,2005.
PDF file . (2005)
-
The following paper, M.Aoyagi, S. Watanabe, "Stochastic complexities of reduced rank regression
in Bayesian estimation," will appear in Neural Networks.
PDF file.
-
AMS Meeting : Biological Computation and Learning in Intelligent Systems
AMS 2002 Fall Central Section Meeting was held in
Madison, Wisconsin, October 12-13, 2002
University of Wisconsin. (2002).
-
Frontier of Non-Commutative Analysis and Mathematical Quantum Theory (2002).
-
Sumio Watanabe,"Algebraic analysis for non-identifiable learning machines",
Neural Computation, Vol.13, No.4, pp.899-933, 2001.
Article, Postscript file, gzipped .
You can get the PDF file from
Neural Compuation, 2001, Vol.13, No.4.
-
Advances in Neural Information Processing Systems.
NIPS*2002 in Vancouver, Canada.
Vancouver is a very beautiful city. (2002).
-
You can obtain articles in NIPS papers from
Advances in Neural Information Processing Systems .
Links
The Annals of Mathematics
Inventiones Mathematicae
Communications in
Mathematical Physics
The Annals of Statistics
Journal of the Royal Statistical Society
Mathematical Journals
AMS
MathSciNet
Applied Mathematics Research Express
Neural Networks
Journal of Machine Learning Research
Geometry and Statistics in Neural Network Learning Theory
Mathematical Calendar
ACM Calendar
Singularity Theory and Its Applications
Scirus
AMS Meeting
Math. and Phys.
Open Problems
The Clay Mathematics Institute
Institute for Advanced Study, Princeton ,
Math Link
Isaac Newton Institute for Mathematical Sciences
Institute for Mathematics and Its Applications
Mathematical Sciences Research Institute
Recent Topics on Real and Complex Singularities
Minimum Description Length
Neural Nets
ISAAC
People
Professor Huzihiro Araki (Mathematical Physics)
Hal Tasaki (Theoretical Physics)
Takashi Hara (Mathematical Physics)
David Mumford (Algebraic geometry, Pattern Theory)
Stephan E. Feinberg (Mathematical Statistics)
Bernd Sturmfels (Algebraic Statistics)
Akimichi Takemura (Mathematical Statistics)
Giovanni Pistone
(Algebraic Statistics)
Lior Pachter
(Mathematical Biology)
Seth Sullivant (Algebraic Statistics)
Mathias Drton (Algebraic Statistics)
Ruriko Yoshida (Algebraic Statistics)
Luis David Garcia Puente (Algebraic Statistics)
Jason Morton (Algebraic Statistics)
Shaowei Lin (Algebraic Statistics)
Piotr Zwiernik (Algebraic Statistics)
Dan Geiger (Computer Science)
Dmitry Rusakov (Computer Science)
Russell Steele (Mathematical Statistics)
**********
Sumio Watanabe Homepage
**********
Research Field:
Probability Theory, Mathematical Statistics, and Learning Theory.
Research Purpose:
(1) To establish mathematical foundation of statistical learning.
(2) To construct a new research field between mathematics and neuroscience.
Recently Published Books:
(1) S. Watanabe, "Neural Networks for Robotic Control - Theory and Applications,"
Prentice Hall, 1996.
(2) S.Watanabe and K.Fukumizu, "Algorithms and Architectures," Academic Press, 1998.
(3) S.Watanabe,
Algebraic Geometry and Statistical Learning Theory
UK,
US (2009/August).
Watanabe's Main Formulas
If you are a mathematician or a statistician, you can understand the importance
of the following formulas. From the mathematical point of view, these formula
clarified the relation between algebraic geometry and statistics. From the
statistical point of view, these are generalized BIC and AIC for singular statistical
models. These two main formulas are mathematically very beautiful and statistically
very useful.
Main Formula 1
Let X1, X2, ..., Xn are random variables which are independently subject
to the probability distribution q(x)dx. Even if the Fisher information matrix
of a statistical model p(x|w) is degenerate, the following formula holds.
The stochastic complexity or the Bayes marginal likelihood
F = -log \int p(X1|w) p(X2|w) ... p(Xn|w) \phi(w) dw
has asymptotic expansion
F= nS + A log n - (B-1) log log n + R
where nS is equal to n times empirical entropy of the true distribution,
A is a positive rational number, B is a natural number, and R is a
random variable of constant order.
Here (-A) and B are determined as the largest pole and its order of the
Zeta function of the statistical model,
J(z)= \int
H(w)^{z} \phi(w)dw
which can be analytically continued to the entire complex plane.
Here H(w) is the Kullback distance from the true distribution q(x)dx to the
parametric model p(x|w)dx.
The Zeta function J(z) is the mathematical bridge between statistics and
algebraic geometry. The expectation value of the Bayes generalization error
is asymptotically equal to
E[Bg] = A/n - (B-1)/(n log n) + o(1/(nlog n)),
where E[ ] is the expectation value over all sets of random samles.
Also we can algorithmically calculate A and B by applying
Hironaka's
resolution of singularities to the Kullback information, and obtain
that A is not larger than D/2, and that B is not larger than D,
where D is the number of parameters. The constants A and B are
determined by the algebraic geometrical structure of the learning machine.
If a model p(x|w) is regular, then A=D/2 and B=1, hence this formula is
the generalized version of BIC and MDL.
Main Formula 2
Let Bg, Bt, Gg, Gt be Bayes generalization error, Bayes training error,
Gibbs generalization error, and Gibbs training error, respectively.
Then the following formulas hold for arbitrary true distribution, arbitrary
parametric model, arbitrary a priori distribution, and arbitrary singularities.
E[Bg]=E[Bt]+2bE[Gt-Bt],
E[Gg]=E[Gt]+2bE[Gt-Bt],
where b is the inverse temperature of the a posteriori distribution.
By using these formulas, we can estimate Bayes and Gibbs generalization errors
from Bayes and Gibbs training errors. If a model is regular then
E[Gt-Bt]=D/2, where d is the dimension of the parameter space.
Hence these formulas contain AIC as a very special case.
For references,
S. Watanabe, ``Algebraic analysis for nonidentifiable learning machines,"
Neural Computation, Vol.13, No.4, pp.899-933, 2001.
S. Watanabe, ``A formula of equations of states in singular learning machines,"
Proceedings of WCCI, Hongkong, 2008.
We would like to claim that these results were firstly discovered,
which were unknown even in statistics, information theory, and learning
theory. Also we expect that these results mathematically clarify the
essential difference between neural networks and regular statistical models.
In other words, the reason why neural networks in Bayesian estimation
are more useful than regular statistical models is firstly proven
mathematically. Algebraic geometry and algebraic analysis play
an important role in the theory of complicated learning machines.
For more detail, see
Singular learning theory .
Mathematical Structure
Related Topic: Zeta functions
(1) As is well known, the Riemann zeta function is defined by
f(z)=\sum_{n=1}^{\infty} 1/n^{z}
which can be analytically continued to the entire complex plane.
You must know the Riemann's Hypothesis that will clarify the distribution
of prime numbers.
(2) The zeta function of Kullback information H(w) and the prior p(w) is
defined by
J(z)=\int H(w)^{z}\phi(w)dw
which can be analytically continued to the entire complex plane.
Professor Gel'fand conjectured in 1954 that this function is meromorphic,
and both Professor Bernstein and Professor Atiyah clarified this fact.
In Atiyah's method, Hironaka's resolution of singularities plays a
central role.
This function clarifies the Bayesian Statistics as I have shown in
the foregoing sections.
(3) The zeta function of the Replica method will be defined by
L(z)=E[ Z(X1,X2,...,Xn)^{z}]
where X1,X2,...,Xn are training samples taken from the
true distribution and E shows the expectation value overall sets of
training samples. Z(X1,X2,...,Xn) is the partition function or the
evidence of the learning. It is strongly expected that this function
plays an important role in clarifying the mathematical structure
of the Replica method in mathematical physics.
Published Papers with Comments:
(1) S.Watanabe, M.Yoneyama, "Ultrasonic Robot Eyes Using Neural Networks,"
IEEE Trans. on Ultrasonics, Ferroelectrics, and Frequency Control, Vol.37,
No.3, pp.141-127, 1990.
(2) S.Watanabe, M.Yoneyama, "A three-dimensional object recognition method
using acoustical imaging and neural networks," The Journal of the Acoustical Society of Japan,
Vol.47, No.11, pp.825-833, 1991.
(3) S.Watanabe, M.Yoneyama, "An Ultrasonic 3-D visual Sensor Using Neural
Networks", IEEE Trans. on Robotics and Automation," Vol.6, No.2, pp.240-249,
1992.
(4) S.Watanabe, M.Yoneyama, "A restoration method of acoustic images
using a neural network," The Journal of the Acoustical Society of Japan,
Vol.48, No.10, pp.711-719, 1992.
(5) S.Watanabe, M.Yoneyama,"A classification method of 3-D objects
by a neuro-ultrasonic visual sensor using position and rotation
invariant feature values,"
The Journal of the Acoustical Society of Japan, Vol.48, No.10, pp.720-726,
1992.
(6) K.Takatsu, H.Sawai, S.Watanabe, M.Yoneyama, "Genetic algorithms
applied to Bayesian image restoration," IEICE Trans.,
Vol.J77-D-2, No.9, pp.1768-1777, 1994.
(7) S.Watanabe, K.Fukumizu, "Probabilistic design of Layered Neural networks
based on their unified framework," IEEE Transactions on Neural Networks,
Vol.6, No.3, pp.691-702, 1995.
(8) S.Watanabe, "A modified information criterion for automatic
model and parameter selection in neural network learning,"
IEICE Transactions, Vol.E78-D, No.4, pp.490-499, 1995.
(9) S.Ishii, K.Fukumizu, S.Watanabe, "A Network of Chaotic Elements for
Information Processing," Neural Networks, Vol.9, No.1, pp.25-40, 1996.
(10) S.Watanabe, "Solvable models of layered neural networks based
on their differential structure,"
Advances in Computational Mathematics, Vol.5, No.1, pp.205-231, 1996.
(11) K.Fukumizu, S.Watanabe, "Optimal training data and predictive error
of polynomial approximation," IEICE Trans.,
Vol.J79-A, No.5, pp.1100-1108, 1996.
(12) S.Watanabe, "A finite wavelet decomposition method," IEICE Trans.,
Vol.J79-A, No.12, pp.1948-1956, 1996.
(13) N. Ishimasa, Y.Yokota, S.Watanabe, "Route Optimization of Mobil Service
Station by Genetic Algorithms with the Variable Number of Gene," IEICE Trans.
Vol.J81-A, No.9, pp.1221-1229,1998.
(14) S.Watanabe, "On the generalization error by a layered statistical model
with Bayesian estimation," IEICE Trans., Vol.J81-A, No.10, pp.1442-1452, 1998.
English Version : Electronics and Communications in Japan,(2000) pp.95-104.
(15) M.Yoneyama, K.Yuasa, S.Watanabe, "Identification of system characteristics
of the ultrasonic imaging system using genetic algorithm," The Journal of
Acoustic Society of Japan, Vol.55, No.1,pp.3-11, 1999.
(16) S.Watanabe,"Algebraic Analysis for Non-identifiable Learning Machines,"
Neural Computation, Vol.13, No.4, pp.899-933, 2001.
Article, Postscript file, gzipped .
This paper clarified the complete asymptotic form of the stochastic
complexity or the freee energy. It is different from that of the
regular statistical model. Based on algebraic analysis, algebraic geometry,
and theory of functions of several complex variabales, it is
clarified how the algebraic structure of the Fisher metric determines
the learning efficiency of a complex learning machine.
I would like to say, if you are interested in mathematical information
theory, then this is a paper worth reading. Even if you consider that
almost all papers in neural computing have no essential advances,
I promise you that this paper truly discovers a new structure in
complex learning machines. The relation between algebraic geometry and
complex learning machines was firstly discovered. As you know, the communicator of
this paper is Professor David Mumford who is the Fields Medalist, 1974 by the
researches in algebraic geometry.
(17) S. Watanabe, "Training and generalization error of learning machines
with algebraic singularities." IEICE Transactions, Vol.J84-A, No.1,
pp.99-108, Jan, 2001.
(18) S. Watanabe, "Algebraic geometry of learning machines with singularities
and their prior distributions," Japanese Journal of Artificial
Intelligence, Vol.16, No.2, pp.308-315, March, 2001.
(19) S. Watanabe, "Algebraic geometrical methods for hierarchical learning
machines," International Journal of Neural Networks, Vol.14, No.8,pp.1049-1060, 2001.
(20) S. Watanabe, "Learning efficiency of redundant neural networks
in Bayesian estimation," IEEE Transactions on Neural Networks, Vol.12, No.6,
1475-1486,2001.
Errata: IEEE Transactions on Neural Networks, Vol.13, No.1,pp.254, 2002.
(21) K.Yamazaki, S.Watanabe,"A probabilistic algorithm to calculate the learning curves of
hierarchical learning machines with singularities," Trans. on IEICE,
Vol.J85-D-II,No.3,pp.363-372,Mar. 2002.
(22) K.Nishiue, S.Watanabe,"Effects of priors in model selection of learning
machines with singularities," To appear in IEICE Trans., Vol.J86-D-II,No.1,Jan.2003.
(23) K. Watanabe, S.Watanabe,"On the Bayes generalization error of the reduced
rank regression," To appear in IEICE Trans., Vol.J86-A,No.3,2003.
(24) S.Watanabe, S.-I.Amari,"Learning coefficients of layered models when the true distribution mismatches the singularities", Neural Computation, Vol.15,No.5,1013-1033, 2003.
(25) K.Yamazaki, S.Watanabe,``Singularities in mixture models and upper bounds of stochastic complexity." International Journal of Neural Networks, Vol.16, No.7, pp.1029-1038,2003.
(26) K.Yamazaki, S.Watanabe,`` Singularities in Complete bipartite graph-type Boltzmann machines and upper bounds of stochastic complexities", IEEE Trans. on Neural Networks, Vol. 16 (2), pp 312-324, 2005.
(28) S.Watanabe, K.Fukumizu,K.Hagiwara, S.Amari,``Learning Theory of Singular Statistical Models," "Vol.J88-D2 No.2 pp.159-169,2005.(Survay Paper).
(28) K. Yamazaki and S. Watanabe, "Algebraic geometry and stochastic complexity of hidden Markov models", Neurocomputing,Vol.69,pp.62-84,2005.
(29) S.Watanabe,``Algebraic geometry of singular learning machines and symmetry of generalization and training errors," Neurocomputing, Vol.67,pp.198-213,2005.
(30) M.Aoyagi, S.Watanabe,``Stochastic complexities of reduced rank regression in Bayesian estimation," Neural Networks, Vol.18,No.7,pp.924-933,2005.
(31) K.Nagata, S.Watanabe,``A method to estimate the generalization error of singular learning machines by decomposition of Kullback information," Vol.J88-II, No.6, pp.994-1002,2005.
(32) N. Nakano, K.Takahashi, S.Watanabe,``A method to estimate the efficiency of Markov chain Monte Carlo in singular learning machines," Vol.J88-D-II,No.10,pp.2011-2020,2005.
(33) M.Aoyagi,S.Watanabe,``Resolution of singularities and generalization error with Bayesian estimation for layered neural network," Vol.J88-D-II,No.10,pp.2112-2124,2005.
(34) S.Nakajima,S.Watanabe,``Generalization performance of subspace Bayes approach in linear neural networks, " to appear in IEICE Transactions (A).
(35) T.Hosino, K.Watanabe,S.Watanabe,``Stochastic complexity of Hidden Markov Models on the Variational Bayesian Learning," to appear in IEICE Transactions (D-II).
(36) Kazuho Watanabe, Sumio Watanabe,
"Stochastic complexities of gaussian mixtures in variational bayesian approximation,"
Journal of Machine Learning Research, Vol.7, pp.625-644, 2006.
(37) Shinichi Nakajima, Sumio Watanabe,
"Generalization Performance of Subspace Bayes Approach in Linear Neural Networks,"
IEICE Transactions, Vol.E89-D, no.3, pp.1128-1138, 2006.
(38) K.Watanabe, S.Watanabe, ``Stochastic complexities of general mixture models in variational Bayesian learning,"
Neural Networks, Vol.20, No.2, March, pp.210-217, 2007.
(The best paper award of 2008 Japanese Neural Network Society).
(39) S. Nakajima, S.Watanabe, ''Variational Bayes Solution of Linear Neural Networks and
its Generalization Performance.''Neural Computation, vol.19, no.4, pp.1112-1153, 2007.
(40) K. Watanabe, S. Watanabe,
Estimating the Data Region Using Minimum and Maximum Values,
Interdisciplinary Information Sciences,
Vol. 13 , No. 2, pp. 151-161, 2007.
(41) K. Watanabe, S. Watanabe,
Stochastic complexity for mixture of exponential families
in generalized variational Bayes,
Theoretical Computer Science, Vol.387, pp.4-17, 2007.
(42) Kenji Nagata, Sumio Watanabe,
``Asymptotic Behavior of Exchange Ratio in Exchange Monte Carlo Method,''
International Journal of Neural Networks, Vol. 21, No. 7, pp. 980-988, 2008.
(43) Kenji Nagata, Sumio Watanabe,
``Exchange Monte Carlo Sampling From Bayesian Posterior for Singular Learning Machines,"
IEEE Transactions on Neural Networks, Vol.19, No.7, pp.1253-1266, 2008.
(44) Kenji Nagata, Sumio Watanabe,
``Theory and Experiments of Exchange Ratio for Exchange Monte Carlo
Method'', Neural Information Processing - Letters and Reviews, Vol.12, No. 1-3, pp.21-30, 2008.
(45) K.Watanabe, M.Shiga, S.Watanabe,"Upper bound for variational free energy of Bayesian networks,"
Machine Learning, vol.75, no.2, pp.199-215, 2009.
(46) Y. Nishiyama, S. Watanabe, ``Accuracy of Loopy Belief Propagation in Gaussian Models,"
Neural Networks, Vol.22, No.4, pp.385-394, May 2009.
(47) K. Yamazaki, M. Aoyagi, S. Watanabe, ``Asymptotic Analysis of Bayesian Generalization Error with Newton Diagram
Neural Networks", Neural Networks, Vol.23, No.1, pp.35-43, 2010.
(48) Sumio Watanabe, "Equations of states in singular statistical estimation",
Neural Networks, Vol.23, No.1, pp.20-34, 2010.
(49) Sumio Watanabe, "A limit theorem in singular regression problem,"
Advanced Studies of Pure Mathematics, to appear.
Proceedings of International Conference:
(1) M.Yoneyama, S.Watanabe, H.Kitagawa, T.Okamoto, T.Morita, ``Neural Network
Recognizing 3-Diminsional Object Through Ultrasonic Scattering Waves",
Proc. of IEEE Ultrasonics Symp., (Chicago), pp.595-598, 1988.
(2) S.Watanabe, M.Yoneyama, ``The Ultrasonic Robot Eye System Using Neural
Network", Proc. of 13th Intern. Cong. on Acoustics, (Belgrade), pp.91-95, 1989.
(3) S.Watanabe, M.Yoneyama, ``An Ultrasonic Robot Eye for Object Recognition
Using Neural Network", Proc. IEEE Ultrason. Symp., (Montreal),
pp.1083-1086, 1989.
(4) S.Watanabe, M.Yoneyama, ``An Ultrasonic Robot Eye for Three-Dimensional
Object Recognition Uisng Neural Network", Proc. of EUSIPCO-90, (Barcelona),
pp.1687-1690, 1990.
(5) S.Watanabe, M.Yoneyama, ``Three-Dimensional Object Recognition Sysytem
Combining Acoustical Imaging with Neural Network", Proc. of ISITA-90,
(Honolulu), pp.655-658, 1990.
(6) S.Watanabe, M.Yoeneyama, ``An Ultrasonic Robot Eye System for Three-
dimensional Object Recognition Using Neural Network", Proc. of
IEEE Ultrasonics Symp., (Honolulu), pp.351-354, 1990.
(7) S.Watanabe, H.Watanabe, A.Saitou. M.Yoneyama, ``An Application of Neural
Networks to an Ultrasonic 3-D Visual Sensor", Proc. of IJCNN,
pp.1397 -1402, (Singapole) 1991.
(8) S.Watanabe, M.Yoneyama, ``A 3-D Visual Sensor Using Neural Networks and
Its Application for Factory Automation", Proc. of FENDT91, (Seoul), pp.379-386,
1991.
(9) S.Watanabe, M.Yoneyama, ``An Ultrasonic Visual Sensor Using a Neural
Network and Its Application for Automatic Object Recognition,"
IEEE Ultrasonics Symp. (Florida) pp.781-784, 1991.
(10) S.Watanabe, K.Fukumizu, ``The Unified Neural Network Theory and
Proposal of New models," 2nd Int. conf. on Fuzzy logic
and Neural Networks, (Iizuka) pp.725-728, 1992.
(11) S.Watanabe, M.Yoneyama, ``An Ultrasonic Robot Eye Using Neural Networks",
Acoustical Imaging, Plrenum Press, New York, Vol.18, pp.83-95, 1992.
(12) K.Takatsu, S.Watanabe, H.Sawai, M.Yoneyama, ``A Proposal of image
restoration using Genetic Algorithms," Proc. of IJCNN
(Beijing), Vol.1, pp.642-647, 1992.
(13) S.Watanabe, K.Fukumizu, ``The Unified Neural Network Theory and
Its Application to New Models," Proc. of IJCNN (Beijing),
Vol.2, pp.381-386, 1992.
(14) S.Watanabe, M.Yoneyama, ``An Ultrasonic 3-D Object Recognition
Method Based on the Unified Neural Network Theory,"
Proc. of IEEE US Symp. (Tucson, Arizona), pp.1191-1194, 1992.
(15) S.Watanabe, M.Yoneyama, S. Ueha, ``An ultrasonic 3-D object
identification system combining ultrasonic imaging with a probability
competition neural network," Proc. of Ultrasonics International 93
conference, (Vienna), pp.767-770, 1993.
(16) S.Watanabe, ``Differential equations accompanying neural networks
and solvable nonlinear learning machines," Proc. of IJCNN (Nagoya),
pp.2968-2971, 1993.
(17) K.Fukumizu, S.Watanabe, ``Probability density estimation
by regularization method," Proc. of IJCNN (Nagoya), pp.1727-1730, 1993.
(18) S.Ishii, K.Fukumizu, S.Watanabe, ``Associative memory
using spatiotemporal chaos," Proc. of IJCNN (Nagoya), pp.2638-2641, 1993.
(19) S.Ishii, K. Fukumizu, S.Watanabe, ``Globally coupled map model for
information processing," Proc. of International Symp., on Nonlinear
Theory and Its Applications, (Honolulu),pp.1157-1160, 1993.
(20) K.Fukumizu, S.Watanabe, ``Error estimation and learning data
arrangement for neural networks," proc. of IEEE world congress
on computational intelligence, (Florida), Vol.2 pp.777-780, 1994.
(21) S.Watanabe, ``Solvable moldes of artificial neural netwroks,"
Advances in Neural Information Processing Systems,
Morgan Kauffmann, New York, Vol.6, pp.423-430, 1994.
(22) S.Watanabe, M.Yoneyama, ``A 3-D Object Classification Method
Combining Acoustical Imaging with Probability Competition Neural Networks,"
Acoustical Imaging, Plenum Press, New York, Vol.20, pp.65-72, 1994.
(23) S.Watanabe, ``An optimization method of artificial neural networks
based on the modified infromation criterion,"
Advances in Neural Information Processing Systems, Morgan Kaufmann, New York,
Vol.6, pp.293-300, 1994.
(24) S.Watanabe, ``A generalized Bayesian framework for neural networks
with singular Fisher information matrices,"
Proc. of International Symposium on Nonlinear Theory and
Its applications, (Las Vegas), pp.207-210, 1995.
(25) S.Watanabe, M.Yoneyama, ``A nonlinear ultrasonic imaging method
based on the modified information criterion,"
Acoustical Imaging, Vol.22, Plenum Press, New York, pp.549-554, 1996.
(26) S.Watanabe,"On the essential difference between neural networks and
regular statistical models," Proc. of Int. Conf. on Computational
Intelligence and Neuroscience, Vol.2, pp.149-152, 1997.
(27) S.Watanabe, "Realizable approximation bounds for a solvable
neural network," Approximation Theory, Vol.1, Vaderbilt University Press,
pp.347-354, 1998.
(28) S.Watanabe,"Inequalities of Generalization Errors for Layered
Neural Networks in Bayesian Learning," Proc. of Int. Conf. on
Neural Information Processing, pp.59-62, 1998.
(29) S.Watanabe, "Approximation bounds for layered learning machines
and environmental probability measures," Proc. of Int. Conf. on
Comupational Intelligence and Neuraoscience, Vol.2, pp.135-138, 1998.
(30) S.Watanabe, "Algebraic analysis for neural network learning",
Proc. of IEEE SMC symp., 1999.
(31) S.Watanabe,"Algebraic analysis for singular statistical estimation,"
Lecture Notes in Computer Sciences, Vol.1720, pp.39-50.
(32) S.Watanabe,"Algebraic analysis for non-regular learning machines,"
Advances in Neural Information Processing Systems, Vol.12, 2000, 356-362.
Article, postscript, gzipped
(33) S.Watanabe,"Algebraic information geometry for learning machines
with singularities", Advances in Neural Information
Processing Systems,(Denver, USA),
pp.329-336. 2001.
Article, postscript, gzipped
(34) Sumio Watanabe, "Algebraic geometry of neural network learning,"
Special session in AMS 2002 Fall Central Section Meeting will be held in
Madison, Wisconsin, October 12-13, 2002
University of Wisconsin.
(35)
Keisuke Yamazaki and Sumio Watanabe, ``Resolution of singularities in mixture models and
the upper bounds of the stochastic complexity." Proc. of International conference on
Neural Information Processing, CD-ROM, 2002.
(36) Sumio Watanabe and Shun-ich Amari, ``Singularities in Neural Networks Make Bayes
Generalization Errors Smaller Even if They Do Not Contain the True,"
Proc. of International conference on
Neural Information Processing, CD-ROM, 2002.
(37) S. Watanabe, S-I. Amari,"The effect of singularities when the true
parameter do not lie on such singularities," NIPS*2002, Vacouver, canada, 2002.
To be continued.
Sumio Watanabe, Ph. D.
Curriculum Vitae
Sumio Watanabe was born in Japan, March 31, 1959.
He received the B.S. degree in Physics from University of Tokyo, Japan,
the M.S. degree in Mathematics from Research Institute for
Mathematical Sciences (RIMS), Kyoto University, Japan,
and the Ph. D. degree in applied electronics in 1993 from Tokyo
Institute of Technology, Japan.
Dr. Watanabe is currently
a professor at Precision and Intelligence Laboratory in Tokyo Institute of Technology.
PI Lab is a research institute for
precise control and artificial intelligence.
His research interest includes probability theory, mathematical statistics,
and learning theory. He is now studying algebraic geometry,
algebraic analysis, and singular learning theory. He firstly
discovered algebraic geometrical structure in statistical learning theory, and
proposed that the standard form of the likelihood function can be derived from
resolution of singularities, on which we can establish a new mathematical statistics.
|
