In this project I try to extend the data clustering algorithm presented in a paper by Casagrande et al. titled "Hamiltonian-Based Clustering: Algorithms for Static and Dynamic Clustering in Data Mining and Image Processing" to a more general setting. Hamiltonian dynamics live in an even dimensional space, and my aim is to instead work on a more general space that allows the Hamiltonian dynamics to live on an odd dimensional space. The primary assumption is that the data live in the phase space and at each data point a Gaussian function is assigned to it. Hamiltonian systems tend to evolve along trajectories in the phase space where the Hamiltonian remains constant and when the motion is period that corresponds to closed loops in the phase space. The idea to employ Hamiltonian systems to find the boundaries of the sets.

As it is with Hamiltonian dynamics, it requires the points to be defined on a cotangent bundle, and cotangent bundles are symplectic manifolds. The limitations of that are that symplectic manifolds are even dimensional manifolds and thus for the data clustering algorithm to work, it requires the data to be even dimensional. The main example used in aforementioned paper obtain data in the form of images, which is a two-dimensional plane. The complications arise when the data are gathered from the real world, say data that lives on the surface of a sphere. In this case the data lives in a three-dimensional space and it cannot be a symplectic manifold. One possible solution, suggested in the paper [1], is to embed odd-dimensional data into an even-dimensional space. At this stage, we try to circumvent such solution by not adding an extra dimension.

We consider an example in three dimensions where we have \(N\) data points, \(\mathbf{x_k} \in \mathbb{R}^3\) for\(k=1, \dots, N \), for a spiral on the surface of a sphere. The aim is to find a set, \(\mathcal{C}\), whose elements \(\mathbf{x_i}\) satisfy certain condition, and in this case the condition is that the elements of the sets needs to be adjacent to each other.

This figure shows the result of Poisson data clustering algorithm. The data point are represented by solid blue dots and the boundary of the set is represented by the red curve.

References:

[1] - D. Casagrande, M. Sassano and A. Astolfi, "Hamiltonian-Based Clustering: Algorithms for Static and Dynamic Clustering in Data Mining and Image Processing," in IEEE Control Systems Magazine, vol. 32, no. 4, pp. 74-91, Aug. 2012.

In this exploratory project, the link between deep learning and fluid dynamics is investigated. A detailed exposition for this project can be found on my personal project page. This enables us to find the limit of deep learning system when both the number of neurons and the number of layers approaches infinity.Further, results of this project are being written in two seperate academic papers: the first focuses on the application Hamilton-Jacobi theory to deep learning and the second regards multisymplectic geometry formulation of deep learning.

Currently, there has been an interest in representing deep learning problems in terms of dynamical systems as was discussed in "A Proposal on Machine Learning via Dynamical Systems" by E. In fact, dynamical systems appear in many neural network architectures, such as recurrent neural network, echo state machines, and spiking neurons to name a few. For the case of spiking neurons, the continuum limit yields a hyperbolic partial differential equation and the field of study is known as neural field theory. This point of view allows for training neural networks to be cast as an optimal control problem. This is of great interest as it provides links between deep learning, hydrodynamics, symplectic and multisymplectic geometry as it is being explored in the papers in preparation. The hydrodynamics connection can be viewed as an extrapolation of the work of Bloch et al in "An Optimal Control Formulation For Inviscid Incompressible Ideal Fluid Flow" .

References:

[1] - W. E. "A proposal on machine learning via dynamical systems." in Communications in Mathematics and Statistics 5.1 (2017): 1-11.

[2] - A. M. Bloch, D. D. Holm, P. E. Crouch and J. E. Marsden, "An optimal control formulation for inviscid incompressible ideal fluid flow," Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187), Sydney, NSW, 2000, pp. 1273-1278 vol.2.

During my undergraduate studies I took courses on signals and systems and Bayesian inference. I became interested in studying systems and using mathematical models to not only control them, but also predicting their behaviour. There are a number of ways, depending on the type of system and available data; one can forecast the states of the system in the future. I mainly worked on three methods, and they are: reservoir computers, ensemble Kalman filter and symbolic regression.

One of the types of neural network that intrigued me the most is reservoir computers, which are a variant of recurrent neural networks. What makes them interesting is that their efficiency in training, where they don't suffer most of the pathologies and complexities of recurrent nerual networks. Employing them to forecast chaotic systems were presented in many papers such as Pathak et al's "Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data." This captured my attention as I wanted to use such networks to implement data-driven control scheme of a black-box system. In the process of learning and implementing reservoir computers, I published a blog post on Medium.com, which can be accessed here.

The second method used for forecasting is the Ensemble Kalman filter. Unlike the previous method, filtering does not substitute the model it enhances the current model. Such algorithms are used in many applications, from implementing feedback controllers to data assimilation and forecasting. This project is my implementation of Ensemble Kalman filter. The filter is an algorithm that, given a set of measures, generates the optimal estimate of the desired quantities through a recursive processing. The Ensemble Kalman filter and it uses Monte Carlo methods to approximate the statistics of the system. The algorithm relies on having the knowledge of the system and its measurement dynamics. In addition to that, the statistical description of the noise associated with the system, the measurement error and the uncertainty of the used dynamic models.

To showcase my implementation of ensemble Kalman filter to estimate the state of the van der Pol oscillator, a nonlinear dynamical system, described by

$$ \begin{bmatrix} x_0^{k+1} \\ x_1^{k+1} \end{bmatrix} = \begin{bmatrix} x_0^{k} + \Delta t \, x_1^{k} + w_0\\ x_1^{k} + \Delta t \, \left( \mu \left(1 - (x_0^{k})^2 \right)x_1^{k} - x_0^{k} \right) + w_1 \end{bmatrix} $$ where \(\Delta t\) is the time step, and here it was chosen to be \(0.01\), \(\mu = 1\), and \(\bar{w} \sim \mathcal{N}(0, Q)\) is a normally distributed random number with covariance \(Q = \mathrm{diag}(0.0262, 0.008)\). 

Another example used to test the implementation of the ensemble Kalman filter is the Lorenz 63 model, whose solution produces the famous Lorenz attractor. The dynamics of the system are described by

$$ \begin{bmatrix} x_0^{k+1} \\ x_1^{k+1} \\ x_2^{k+1} \end{bmatrix} = \begin{bmatrix} x_0^{k} + \Delta t \, a \, \left(x_1^{k} - x_0^{k} \right) + w_0\\ x_1^{k} + \Delta t \, \left( x_0^{k} \left( b - x_2^{k}\right) - x_1^{k} \right) + w_1 \\ x_2^{k} + \Delta t \, \left( x_0^{k} \, x_1^{k} - c x_2^{k} \right) + w_2 \end{bmatrix} $$ and the measurement is done according to $$ \bar{y}^{k+1} = C \bar{x}^{k+1} + \bar{v} $$

With \(\bar{w} \sim \mathcal{N}(0, Q)\), where \(Q = \mathrm{diag}(0.01, 0.03,0.1)\) and \(\bar{v} \sim \mathcal{N}(0, R)\), where \(R = 0.003\). The figures show the approximated state using the measured data \(y\).

The third method for forecasting is symbolic regression. The aim is instead of finding a network's weights, or changing the model's parameters, symbolic regression finds a mathematical expression that closely fits the observed data. Evolutionary programming is often used to find the mathematical expression. I have used symbolic regression, however not for forecasting but for the autonomous navigation project.

References:

[1] - J. Pathak, Z. Lu, B. R. Hunt, M. Girvan, & E. Ott, "Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data" in Chaos: An Interdisciplinary Journal of Nonlinear Science, 27(12), 121102. (2017).

While learning about stochastic optimal control, I came across reinforcement learning which is a machine learning paradigm that is frequently used when the mathematical model is unavailable. However, that was before the introduction of deep Q-networks and things have changed drastically thereafter. Here are few projects that I worked on to help me learn about reinforcement learning:

Stabilising an inverted double:

One of the entry problems in reinforcement learning is stabilising an inverted pendulum on a moving cart. The controls are either move the cart left or right depending on where the pendulum is starting to swing. Here a similar problem is considered and instead of a single pendulum a double pendulum remains stationary on a moving cart. This renders the problem more challenging as the double pendulum is a chaotic system, but despite that it possesses two equilibrium point. At this point, a deep Q-network is used to stabilise the inverted pendulum by moving the cart either left or right.

Playing a side scrolling shoot 'em up:

Deep Q-network came to prominence when it was used to learn and play Atari games. Here instead of using the standard Atari games, a slightly different game is considered, which is coded from scratch. The game is based on Thunder Force IV. In order to use reinforcement learning to teach a computer to learn video games, either use a convolutional neural network to "read" the screen, or access an emulator's memory directly. The third option is to code a game from scratch and use it as a control system and feed the states directly to DQN. For this reason the game was coded using Gym AI.

One of the types of neural networks that caught my attention was Restricted Boltzmann Machine. I came across RBMs after reading David MacKay's book "Information Theory, Inference, and Learning Algorithms" [1]. A Restricted Boltzmann Machine is a class of Markov random field, which is an undirected graph used to visualise probability distributions. I worked on this project mainly to learn about Restricted Boltzmann Machines and Deep Belief Networks by understanding how they work by implementing them. RBMs is used in a number of real-world applications such as classification and collaborative filtering. The main source used to learn about RBMs is a paper by Fischer and Igel titled "An Introduction to Restricted Boltzmann Machines" [2]. The code is intended to be used for future research project related to the field of information geometry.

An RBM consists of \(N_v \) visible units \( \vec{v} \), which represents observed data and \( N_h \) hidden units \(\vec{h} \). The units are interconnected, however, units from the same type are not connected with each other. The most common used type of RBMs, the hidden and visible units are binary random variables, that obey a Binomial distribution. The probability distributions over the model is given by

\( P(\vec{v}, \vec{h}) = \frac{1}{Z}e^{-E(\vec{v}, \vec{h})}, \)

where \(Z \) is the normalising factor and \( E(\vec{v}, \vec{h})\) is the energy given by

\( E(\vec{v}, \vec{h}) = -\sum_{i=1}^{N_h}\sum_{j=1}^{N_v} w_{ij}v_jh_i - \sum_{i=1}^{N_h}c_ih_i - \sum_{j=1}^{N_v}b_j v_j, \)

where \(w_ij \) is the weight assigned to the connection between the \(j^{th} \) visible unit and the \(i^{th} \) hidden one and \(b_j\) and \(c_i \) are biases. The repository for my implementation of RBM can be found here.

References:

[1] - D. J.C. MacKay. "Information theory, inference and learning algorithms". Cambridge university press, 2003.

[2] - A. Fischer, and C. Igel, "An introduction to restricted Boltzmann machines. In Iberoamerican Congress on Pattern Recognition." (2012, September), (pp. 14-36). Springer, Berlin, Heidelberg.