Automatic detection and prediction of anomalies in complex IT systems

During the last few years, businesses across geographies and industries have faced an explosion of IT complexity driven by the implementation of new digital businesses and the drive for digitalization. Technology vendors have responded by providing public cloud platforms and the introduction of microservices.

This has allowed businesses the tools to implement new solutions faster making applications no longer monolith applications deployed to a few physical servers. Now, applications consist of complex combinations of custom applications, packaged vendor applications, microservices, and containers with dependencies spanning industry value chains. Hence, any performance degradation in even a single component (out of thousands) can have ripple effects that disrupt a full value chain and massive financial consequences. 

Our objectives are to

• find a way to combine numerical time series data and text-based log events to gain insight into the performance of IT systems and alert about potential problems and their root cause;
• do it in a completely automatic and general way so the framework can be adapted and applied to any kind of business, time series, and log data, without additional tuning. 

FoU Challenges for NR in the project: 

– Multivariate anomaly detection algorithms using time series data across a vast array of IT processes; 

– Unsupervised event (log) analysis and prediction using natural language processing; and  

– Combination of time series and event (log) data to detect and predict anomalies.