Abstract:
System logs play a critical role in diagnosing, securing, and optimising an IT system’s operations.
However, the traditional log analysis methods like rule-based systems lack the capability
to handle the increasing volume, heterogeneity and complexity of log data. In this research,
the scalability and adaptability problems of log analysis were addressed by proposing an automated
framework powered by LLM models. BERT and GPT-2 were chosen as robust baselines
to systematically assess LLM-based log analysis, despite the existence of more recent LLMs.
The objective was to evaluate how well these models can extract structured templates from unstructured
logs and detect anomalies with minimum human intervention. This study focused
on diverse IT log structures that include system, application and security logs. Methodology
involved the collection of more than 32,000 log messages from 16 public sources, including
LogHub, GitHub and Kaggle. Data preprocessing included an extended pipeline consisting of
timestamp normalisation, noise removal, structural parsing, domain normalisation, tokenisation
and padding. The log entries were encoded into template labels through TF-IDF and embeddings,
with both models fine-tuned over transformer learning methods. Isolation Forest was
chosen for anomaly detection as it is effective at handling high-dimensional log data and can
identify rare anomalies without the need for labelled samples. Further, it successfully separates
unusual patterns in large-scale heterogeneous logs compared to clustering or statistical techniques.
The comparative analysis reveals that BERT performed exceptionally well (96.61%)
in structured logs, while GPT-2 was more resilient (97.44%) in unstructured data, even though
GPT-2 had a higher overall accuracy. These findings suggest that these models can automate
log analysis, and each of them can excel in a different situation. This study presents applied
usefulness in implementing LLMs for log parsing and anomaly detection to provide real-time,
scalable solutions that require minimal manual intervention. On the theoretical level, it expands
transformer models beyond typical NLP domains and on the practical level, it integrates them
into enterprise IT monitoring. Future works include ensemble models, Explainable AI and integration
with real-time streaming.
