Understanding Linux logs system
Weclome back again to the series of “security Analysis 101” where we understand logs and there pattern by using multiple utilities and tools.
In this article, i will help you to go through multiple log types of linux system, where are they stored? and how we can filter them and what commands we are going to use.
Lets start with introduction to the Linux logs system.
Introduction
Linux is an open-source operating system comes in different flavours / distrubutions. You may be familiar with the various flavors of Linux, including Ubuntu, Centos, and Red Hat Enterprise Linux (RHEL). They are different in GUI or softwares and needs but there core is Common, that is Linux.
Logs storing sytem
All these operating systems have a logging framework installed to monitor the system and its services. The logging framework for Linux includes a set of directories, files, services, and commands that administrators can use. It helps you to structure the logs and help to easily find them.
The logs are one of the most important part of troubleshooting. As security analyst I filter the logs and go through them to see where issue occurs and at what time, it also show location and file where is might occured.
Basics of Linux logs:
We will go through the basics of Linux logs: the common Linux logging framework, the locations of these log files, and the different types of logging daemons and protocols (such as syslog and rsyslog). Finally, we’ll review some common Linux log commands to read and search through the logs on a system.
Common Linux Logs and Their Locations
With the Linux logs pattern, you will find logs located under the “/var/log” directory, with files and directories for each service
- “/var/log/syslog” (Debian) or “/var/log/messages” (RHEL): In this log file, you can find messages from services like mail, kernel, authentication, and cron.
- /var/log/auth.log (Debian) or /var/log/secure (RHEL): This file contains authentication logs for both successful and failed login attempts on the system.
- /var/log/wtmp : This file contains a history of all user login and logout activity for auditing users’ activity on the system.
- /var/log/lastlog : Similar to the wtmp audit file, this log file tracks users’ last logins. This is a binary file you can read via the lastlog command.
- /var/log/kern.log : This file contains log messages produced by the kernel before being passed to the system logging service (such as rsyslog) for further processing.
- /var/log/cron: The cron service runs as an orchestrator to start tasks at scheduled intervals. Messages from this service — such as when a cron job has started and if any errors occurred during its execution — can be found in this log file.
your application-specific logs are made available under subdirectories in /var/log . For example, if you are running an Apache web server on your Debian-based Linux system, you will find log files under the /var/log/apache2 directory. You would find separate files for each log stream in this directory, such as the access.log or the error.log
What is Syslog and what is use of it ?
Syslog is a network-based logging protocol that monitors your systems and applications. This protocol provides a standard way for services and applications to report their logs. That way, they can be processed and redirected as needed.
Standardized message format:
The syslog protocol provides a message format defined by the RFC 5424 standard. In this format, common event information is defined, such as the timestamp, hostname, and the name of the application that produced the message. There are 0–23 code i show only normal we use through.
________________________________________________________________
| Facilities Code | Keyword | Description |
|-----------------|---------|----------------------------------|
| 0 | kern | Kernel messages |
| 1 | user | User-level messages |
| 2 | mail | Mail system |
| 3 | daemon | System daemons |
| 4 | auth | Security/authorization messages |
| 5 | syslog | Messages generated internally by syslogd |
| 9 | cron | Clock daemon |
| 11 | ftp | FTP daemon |
----------------------------------------------------------------Basic Commands for Linux Logging :
As an Security Researcher i commonly go through logs whenever i faced suspicious behaviour of my machine such as it take more resource that it require for same software before. There are some commands that i personaly useful during my analysis or solving challenges like sherlock on Hackthebox , commands are listed below:
- cat : Short for concatenate, which allows you to view the contents of one or more files in the terminal.
- more : Similar to cat utility, this command reads the content of files in the terminal. However, this utility will interactively display it one page at a time to the user for an easier manual reading experience.
- less : Much like the more utility, this command displays a single terminal screen of content at a time, allowing for easier navigation of large text files.
- tail : By default, tail displays the last ten lines written to a file. Using the follow option ( -f or — follow ) allows you to monitor the file As new lines are written, they are printed to the user’s terminal.
- head : This utility is the opposite of the tail command, fetching the beginning lines of a file. By default, head will display the first ten lines of a file.
- grep : This command allows you to parse input text using filters and regex to find specific patterns in the text. It is useful for searching and manipulating text in scripts or automation.
Using these commands you can find result you want, for example:
cat /var/log/cron | grep "foo"Advanced logging operations can also be done with other commands like awk , cut , and advanced grep filters, allowing you to gain more insight into what happens on your system.
I hope you like this blogs and let me what should i add more in further blogs so we can grow together with by polishing our skills.
Ok, see you next time! with new content with new spirit !!!
