NEU Meta-Learning and its Universal Approximation Properties. (arXiv:1809.00082v2 [stat.ML] UPDATED)

We introduce a new meta-learning procedure, called non-Euclidean upgrading (NEU), which learns algorithm-specific geometries by deforming the ambient space until the algorithm can achieve optimal performance. We prove that these deformations have several novel and semi-classical universal approximation properties. These deformations can be used to approximate any continuous, Borel, or modular-Lebesgue integrable functions to arbitrary precision. Further, these deformations can transport any data-set into any other data-set in a finite number of iterations while leaving most of the space fixed. The NEU meta-algorithm embeds these deformations into a wide range of learning algorithms. We prove that the NEU version of the original algorithm must perform better than the original learning algorithm. Moreover, by quantifying model-free learning algorithms as specific unconstrained optimization problems, we find that the NEU version of a learning algorithm must perform better than the model-free extension of the original algorithm. The properties and performance of the NEU meta-algorithm are examined in various simulation studies and applications to financial data.