Incremental-Decremental Methods for Time Series Analysis

An incremental-decremental algorithm adds and removes points from a sample/training set in an efficient way - typically with finite memory overhead, and finite computation for each add/remove operation. We will explain some incremental-decremental methods we developed for the purpose of maintaining rolling window statistics. The techniques include a novel representation of a computation as a DAG, and adaptations of ideas from econometrics. As a consequence, we expand the range of anomaly detection, forecasting, and time series classification that can be performed in the streaming setting. Our methods were motivated by use cases from infrastructure monitoring, and certain UX and platform requirements for a data-intensive monitoring product. The talk will provide statistics background and business context.