Vladimir N. Vapnik

Vladimir N. Vapnik


Sunday, December 6, 1936

Vladimir N. (Naumovich) Vapnik and Alexey Ya. (Yakovlevich) Chervonenkis added several glorious pages to the history of the Institute of Control Sciences. In the early 1960s, they joined Lerner’s Laboratory and became leading experts at the Institute. At that time, the theory of pattern recognition theory gained popularity but was investigated by a few researchers (M.A. Aizerman, M.M. Braverman, L.I. Rozonoer, and M.M. Bongard). From 1962 to 1971, Vapnik and Chervonenkis developed the generalized portrait method for pattern recognition. In 1968, they presented proof for a fundamental result, i.e., the conditions of uniform convergence of relative frequencies to probabilities over an infinite number of events, a generalization of the classical law of large numbers. They established the necessary and sufficient conditions for the uniform convergence of empirical means to expectations over a family of random variables. Currently, these results are widely known, and the concept of the Vapnik—Chervonenkis (VC) dimension has entered the international scientific lexicon. Workshops on the Vapnik—Chervonenkis Dimension were held in Rehovot (Israel, 1995), Edinburgh (Great Britain, 1996), Ma’ale HaHamisha (Israel, 1998), Paris (France, 2003), etc.

From 1971, Vapnik and Chervonenkis continued their research in Petrovskii’s Laboratory. The uniform convergence conditions were adopted to show the convergence of learning methods based on minimizing empirical risk and to obtain estimates of the convergence rate. In particular, the learning methods of this class are applied to design piecewise linear decisive rules minimizing the number of errors on the learning material. Note that neural networks are formal means of implementing such rules. Therefore, the Vapnik—Chervonenkis theory has become popular to analyze the performance of neural networks.

Also, Vapnik and Chervonenkis developed structural risk minimization methods. Nowadays, they are widely used in pattern recognition, regression reconstruction, and inverse problems arising in physics, statistics, and other sciences.

The most significant applications of the generalized portrait and structural risk minimization methods include medical diagnosis and risk group selection. Such a problem was solved jointly with the All-Union Cancer Center, the USSR Academy of Medical Sciences. These methods were also applied in geology: an automatic optimal ore mapping system based on operational exploration data was developed jointly with the Institute of Geology of Ore Deposits, the USSR Academy of Sciences.

In 1990, Vapnik moved to the USA and joined AT&T Bell Laboratories. Based on the generalized portrait method, he developed the theory of Support Vector Machines (SVMs).

Vapnik’s main books are as follows:

  1. The Nature of Statistical Learning Theory, Springer, 1999. — 334 p.;
  2. Statistical Learning Theory, New York: John Wiley, 1998. — 768 p.;
  3. Algoritmy i programmy vosstanovleniya zavisimostei (Algorithms and Programs to Reconstruct Dependences), Moscow: Nauka, 1984. — 816 p. (coauthors T.G. Glazkova, V.A. Koshcheev, A.I. Mikhalsky, and A.Ya. Chervonenkis);
  4. Estimation of Dependences Based on Empirical Data, New York: Springer-Verlag, 1982. — 400 p.;
  5. Theorie der Zeichenerkennung, Berlin: Academie-Verlag, 1979. — 343 p. (coauthor A. J. Tscherwonenkis);
  6. Vosstanovlenie zavisimostei po empiricheskim dannym (Reconstruction of Dependences by Empirical Data), Moscow: Nauka, 1979. — 448 p.;
  7. Teoriya raspoznavaniya obrazov (Theory of Pattern Recognition), Moscow: Nauka, 1974. — 416 p. (coauthor A.Ya. Chervonenkis);
  8. Zadacha obucheniya raspoznavaniyu obrazov (Learning to Recognize Patterns), Moscow: Znanie, 1971. — 64 p.

Many of them are presented in the Institute’s database:

The list of journal papers by Vapnik is available at Math-Net.Ru:

1. V. N. Vapnik, Complete Statistical Theory of Learning, Avtomat. i Telemekh., 2019, 11,  24—58  Autom. Remote Control80:11 (2019), 1949—1975.    
2. V. N. Vapnik, N. M. Markovich, A. R. Stefanyuk, On the Rate of Convergence in L2 of a Projection Estimator of a Probability Density, Avtomat. i Telemekh., 1992, 5,  64—74      Autom. Remote Control53:5 (1992), 677—686.
3. F. A. Aidu, V. N. Vapnik, Estimating the Probability Density by the Stochastic Regularization Method, Avtomat. i Telemekh., 1989, 4,  84—97      Autom. Remote Control50:4 (1989), 499—509.
4. V. N. Vapnik, Yu. K. Danileiko, T. P. Lebedeva, Yu. P. Minaev, A. I. Mikhal’skii, Numerical Simulation of Laser Damage to an Optical Material with Defects, Kvantovaya Elektronika14:2 (1987),  295—299  ; Sov J Quantum Electron17:2 (1987), 177—180.  
5. V. N. Vapnik, A. Ja. Červonenkis, Necessary and Sufficient Conditions for the Uniform Convergence of Empirical Means to Their True Values, Teor. Veroyatnost. i Primenen.26:3 (1981),  543—563      Theory Probab. Appl.26:3 (1982), 532—553.  
6. V. N. Vapnik, A. R. Stefanyuk, Nonparametric Methods for Restoring the Probability Densities, Avtomat. i Telemekh., 1978, 8,  38—52      Autom. Remote Control39:8 (1979), 1127—1140.
7. V. N. Vapnik, A. M. Sterin, Controlled Minimization of the Total Risk in Pattern Recognition, Avtomat. i Telemekh., 1977, 10,  83—92    Autom. Remote Control38:10 (1978), 1495—1503.
8. V. N. Vapnik, A. Ya. Chervonenkis, Asymptotic Properties of the Method of Ordered Minimization, Avtomat. i Telemekh., 1975, 12,  65—77    Autom. Remote Control36:12 (1975), 1986—1999.
9. V. N. Vapnik, A. Ya. Červonenkis, On Uniform Convergence of the Frequencies of Events to Their Probabilities, Teor. Veroyatnost. i Primenen., 16:2 (1971),  264—279      Theory Probab. Appl.16:2 (1971), 264—280.
10. V. N. Vapnik, A. Ya. Chervonenkis, The Uniform Convergence of Frequencies of the Appearance of Events to Their Probabilities, Dokl. Akad. Nauk SSSR181:4 (1968),  781—783.      


The list of his papers in Avtomatika i Telemekhanika can be found at:

For their English versions, see the microfilm collection of Automation and Remote Control (1956—1994):
and the journal page at SpringerLink (2001—2022):

Also, see the Wikipedia page devoted to Vapnik:

Scopus Author ID: 6604096045