Transaction Profiles: Streaming Analytics Mini Models

Blog Post created by Advocate on Dec 8, 2017

Raise your hand if you knew Kalman filters underlie the rocket guidance systems that were used in the moon landings. It’s true, recursive estimations can be made based on Kalman filtering.


This algorithm uses measurements observed over time to adaptively estimate variables on the fly and in real-time; this is illustrated in the formula below.


Kalman Filters.png

It is critical that we maintain ‘state’, which simply can be viewed as the past iternative estimate of variables where we do not maintain the history of transaction, but of adaptive estimate and update estimates, to get the current ‘read’ or ‘state’ of the variables defining an entity we monitor. A profile state is continually augmented by incoming streaming data, allowing real-time adaption of the state of an entity; this is at the heart of behavioral analytics.


The state contains many variables and can also be applied to our terrestrial lives, not just space travel. The result is an accurate real-time estimate of variables tracking behavior, where each variable is essentially a mini-model changing in real-time. This provides understanding of the trajectory of behaviors and how these trajectories are changing, one example is the behavior of a credit card customer. These mini-models are then fed into progressively more complex machine learning algorithms to generate final scores.

mini models.png

This method allows real-time reactive understanding of customers, financial accounts, computer intrusion, and marketing propensity (to name just a few), and yes, also rocket guidance. Have you taken advantage of streaming data in any of your analytic models?