Subject: anomaly detection, log data, system dependability, AIOps, deep learning
Year: 2022
Type: Proceedings
Title: Leveraging Log Instructions in Log-based Anomaly Detection
Author: Bogatinovski, Jasmin
Author: Madjarov, Gjorgji
Author: Nedelkoski, Sasho
Author: Cardoso, Jorge
Author: Kao, Odej
Abstract: Artificial Intelligence for IT Operations (AIOps) describes the process of maintaining and operating large IT systems using diverse AI-enabled methods and tools for, e.g., anomaly detection and root cause analysis, to support the remediation, optimization, and automatic initiation of self-stabilizing IT activities. The core step of any AIOps workflow is anomaly detection, typically performed on high-volume heterogeneous data such as log messages (logs), metrics (e.g., CPU utilization), and distributed traces. In this paper, we propose a method for reliable and practical anomaly detection from system logs. It overcomes the common disadvantage of related works, i.e., the need for a large amount of manually labeled training data, by building an anomaly detection model with log instructions from the source code of 1000+ GitHub projects. The instructions from diverse systems contain rich and heterogenous information about many different normal and abnormal IT events and serve as a foundation for anomaly detection. The proposed method, named ADLILog, combines the log instructions and the data from the system of interest (target system) to learn a deep neural network model through a two-phase learning procedure. The experimental results show that ADLILog outperforms the related approaches by up to 60% on the F1 score while satisfying core non-functional requirements for industrial deployments such as unsupervised design, efficient model updates, and small model sizes.
Publisher: IEEE
Relation: 2022 IEEE International Conference on Services Computing (SCC)
Identifier: oai:repository.ukim.mk:20.500.12188/23156
Identifier: http://hdl.handle.net/20.500.12188/23156
