Reducing the dimensionality of real-time sensor data

Abstract

In many scientific field such as oil & gas, physics, or weather analysis, as well as in domains such as security or risk assessment, massive amounts of sensor generated measurement data is produced that has to be analyzed and mined to gain insights. The actual data is collected over time and the vast amount of observations lead to a collection of ordered data on a time line. Traditionally, time series data reflects high-dimensional data that requires large amounts of memory and storage space. In oil & gas, data collected on rigs may exceed the capacity potential of the network link (for uploading the data), as well as the actual local storage that is available. Hence, the traditional approach of defining machine learning algorithms that operate on the stored datasets is not feasible. This talk focuses on bringing machine learning to the source (sensors) and so reduce the dimensionality of the data in flight. The major challenge is to represent the meaningful information of the time series' data via a low-dimensional representation while capturing the essence of the data pattern in flight.