As part of my Master’s Thesis on Log Anomaly Detection using Large Language Models (LLMs), I’ve started delving deep into Rust, a systems programming language that’s known for its safety and performance. Rust will play a crucial role in my project, as I plan to use it to create a low-level log collector that will feed a dataset for anomaly detection. Here’s a bit about why Rust fits into this picture and what I’ve learned so far.

Why Rust for My Thesis?

My objective is to create a custom crate that will collect system logs, specifically targeting Linux environments. I chose Rust because of several key features:

  • Memory safety: Rust ensures safe memory management without the need for a garbage collector, which is essential when dealing with high-performance, low-level tasks.
  • Concurrency: Since log collection may involve parallel operations, Rust’s ownership and borrowing mechanisms will help avoid common concurrency issues.
  • Systems programming capabilities: Rust provides low-level access to system resources (e.g., /proc), allowing for granular interaction with the file system, which is crucial for capturing logs and identifying potential Advanced Persistent Threats (APTs).

My Approach

The idea is to avoid using existing crates that handle system monitoring and instead build everything from scratch. This will not only deepen my understanding of the Linux OS, but it will also give me complete control over the log collection process, which is tailored to my project’s specific needs.

Here are a few key aspects of what I’ll be doing:

  • Interacting with the Linux file system: I’ll focus on reading from /proc and other directories to capture detailed system metrics.
  • APT detection: The crate will attempt to identify suspicious patterns that may be indicative of an APT attack. These patterns will then be fed into an LLM for deeper anomaly detection analysis.
  • No external crates: While there are many useful crates out there, my goal is to build my own from the ground up. This will give me a better understanding of both Rust and Linux’s inner workings.

Current Learning Focus: Ownership and Borrowing

At the moment, I’m particularly focused on Rust’s ownership and borrowing mechanisms, which form the foundation of the language’s memory safety. By mastering these concepts, I’ll ensure that my crate manages memory efficiently, without unexpected behaviors such as race conditions or memory leaks, which are common in low-level system programming.