Self-organizing maps in sequence processing

Abstract

Models are abstractions of observed real world phenomena or processes. A good model captures the essential properties of the modeled phenomena. In the statistical learning paradigm the processes that generate observations are assumed unknown and too complex for analytical modeling, thus the models are trained from more general templates with measured observations. A substantial part of the processes we seek to model have temporal dependencies between observations thus defining templates that can account for these dependencies improves their ability to capture the properties of such processes.

In this work we discuss using the self organizing map with sequentially dependent data. Self-Organizing map (SOM) is perhaps the most popular non supervised neural network model that has found varied applications in the field of data mining for example. The original SOM paradigm, however, considers independent data, where context of a sample does not influence its interpretation. However, throwing away the temporal context of an observation when we know we are dealing with sequential data seems wasteful. Consequently methods for incorporating time into the SOM paradigm have been rather extensively studied. Such models if powerful enough would be very usable when tracking dynamic processes.

In this work a Self-Organizing map for temporal sequence processing dubbed Recurrent Self-Organizing Map (RSOM) was proposed and analyzed. The model has been used in time series prediction combined with local linear models. Deeper analysis provides insight into how much and what kind of contextual information the model is able to capture. The other topic covered by the publications in a sense considers an inverse problem. In this topic SOM was used to create sequential dependence and order into initially unordered data by modeling a surface and creating a path over the surface for a surface manipulating robot.