PP: UMAP: uniform manifold approximation and projection for dimension reduction

From Tutte institute for mathematics and computing

Problem: dimension reduction

Theoretical foundations:

At a high level, UMAP uses local manifold approximations and patches together their local fuzzy simplicial set representations to construct a topological representation of the high dimensional data. Given some low dimensional representation of the data, a similar process can be used to construct an equivalent topological representation. UMAP then optimizes the layout of the data representation in the low dimensional space, to minimize the cross-entropy between the two topological representations.

解释：使用local manifold approximations and local fuzzy simplicial set presentations, 在高维空间上构建了一个拓扑表征topological representation，在低维空间上，同样构建一个等价的拓扑表征，之后运用交叉熵作为优化函数，来计算两个空间拓扑表征的差异性，从而使差异性最小化。

Construction of fuzzy topological representations:

1. approximating a manifold on which the data is assumed to lie;

2. constructing a fuzzy simplicial set representation of the approximated manifold.

解释：

疑问：一组高维数据究竟落在哪？高维数据应该用哪个空间进行衡量？Euclidean space, topological space, Riemannian space还是啥空间测量？还是应用不同的空间策略都能得到相似的结果？

1. approximating a manifold on which the data is assumed to lie,

Suppose the manifold is not known in advance and we wish to approximate geodesic distance on it. Let the input data be X = {X1 , . . . , XN }.

A Computational view of UMAP:

Two phases.

In the first phase, a particular weighted k-neighbour graph is constructed. In the second phase, a low dimensional layout of this graph is computed

1. weighted k-neighbour graph construction

Use the nearest neighbor descent algorithm of [1]

2. low dimensional layout

Use force-directed graph layout in low dimensional space.

Implementation and hyper-parameters:

Supplementary knowledge:

1. simplicial sets. 单纯集

In mathematics, a simplicial set is an object made up of "simplices单纯形" in a specific way. Simplicial sets are higher-dimensional generalizations of directed graphs, partially ordered sets and categories.

simplex: 单纯形,

In geometry, a simplex (plural: simplexes or simplices) is a generalization of the notion of a triangle or tetrahedron四边形 to arbitrary dimensions.

For example,

a 0-simplex is a point,
a 1-simplex is a line segment,
a 2-simplex is a triangle,
a 3-simplex is a tetrahedron,
a 4-simplex is a 5-cell.

2. Hadamard product/ pointwise product

3. What is n-skeleton?

4. mathematical conception

Convergent Sequence, 收敛序列

Space, space_conception_2,

The concept of a space is an extremely general and important mathematical construct. Members of the space obey certain addition properties. Spaces which have been investigated and found to be of interest are usually named after one or more of their investigators.

The everyday type of space familiar to most people is called Euclidean space. In Einstein's theory of Special Relativity, Euclidean three-space plus time (the "fourth dimension") are unified into the so-called Minkowski space. One of the most general type of mathematical spaces is the topological space.

Metric Space

A metric space is a set with a global distance function (the metric ) that, for every two points in , gives the distance between them as a nonnegative real number . A metric space must also satisfy

1. iff ,

2. ,

3. The triangle inequality .

Euclidean space:

Euclidean -space, sometimes called Cartesian space or simply -space, is the space of all n-tuples of real numbers, (, , ..., ). Such -tuples are sometimes called points, although other nomenclature may be used (see below). The totality of -space is commonly denoted ,.

Topological space:

A topological space, also called an abstract topological space, is a set together with a collection of open subsets that satisfies the four conditions: