In 1955 John Von Neumann the eminent Mathematician of the early part of the 20th century published his first concept of the Singularity – the Technological singularity. A time when the “ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.” Here von Neumann was looking mainly at the speed that technological advancement could overwhelm the traditional values held by humanity. However, the singularity could now be seen as the time that Technological Artificial Intelligence overtakes Human Intelligence to the point where the human mind can no longer comprehend, imagine nor control what A.I. thinks or what it is capable of thinking of. This is indeed a technological crisis. This is where the technological child exceeds the parent and becomes like a God : – Deus est Machina. In the next part, we will look at how this singularity can be achieved and how near we are to the Singularity itself.
The article continues in the post above.
Photo courtesy of : http://www.lanl.gov/history/atomicbomb/images/NeumannL.GIF
In this paper, we analyze a deep neural network model from the viewpoint of singularities. First, we show that there exist a large number of critical points introduced by a hierarchical structure in the deep neural network as straight lines. Next, we derive sufficient conditions for the deep neural network having no critical points introduced by a hierarchical structure. The existing overlap singularities in the parameter space significantly affect the learning dynamics of the multilayer perceptrons. From the obtained theoretical learning trajectories near overlap singularity, when the learning process has been affected by the overlap singularity, the influence area of the overlap singularity is just the line space where the two hidden units equal to each other. However, in the practical applications, different case has been observed and the influence area of such singularity may be larger. By analysing the generalization error of multilayer perceptrons, we find that the error surface is much flatter near overlap singularity and the singularity would have much larger influence area. Finally, the validity of the obtained results are verified by taking an artificial experiment and two real data experiments.
Hi Матвей Many thanks for your input – Here you are referring to a paper by Tohru Nitta given to the Conference: International Conference on Neural Information Processing in 2016 a very interesting paper which, as usual, starts off as you have outlined above but rather quickly delves into rather abstract mathematical constructs. The paper is obtainable from research gate and if anyone wishes to see it you can Click Here Again Many thanks for your input..