Security Analysis 101 : Common types of logs

Whats up security people , I came back with some techy topic while browsing over internet we visit unknow site and as security researcher we start perform attacks on random site thats the way most people start there journey then we came to know its illegal activity, each of your request is being recorded on server.

This image is being taken from kapwing.com

Yes its logs that shows your activities to administrator . Further administrator can analyse these log and may take legal action against you, next while doing such stuff remember twice.

Now big questions arises that,

What is logs ?

Logs are records of events or activities generated by applications, systems, and devices. They capture essential information such as timestamps, user actions, system errors, and other significant events.

Common Types of logs Formats:

here, I will give you simple overview of log formats, what are there types and examples.

ok lets uderstand, what is mean by format ? A log format defines how the contents of a log file should be interpreted.

• log content structured or unstructed.
• log data binary or text.
• type of encoding used in log file
• how records are delimited or can say seperated.

So, it cocluded that 3 types of categories of logs :

1. structured : built in parsers can be read by humans and machines

2. unstructured : no built-in parser but still easy for humans to read

3. semi-structured : It have a schema or pattern, making it possible for machines to read too.

Applications usually define their available log format (example JSON or CSV). For hardwares devices manufactures usually define the log types to be used.

There are mainly 6 types of logs formats :

1. JSON

JSON usually uses UTF-8 encoding at rest and in transit, which makes it accessible by both *nix and Windows operating systems. There are no restrictions on the quantity or type of fields you can include.

Example logs:

{<!-- --><p>"timestamp": "2022-07-29T02:03:45.293Z",</p><p>"message": "User Jane.Doe has logged in",</p><p>"log": {<!-- --></p><p>"level": "info",</p><p>"file": "auth.c",</p><p>"line": 66,</p><p>},</p><p>"user": {<!-- --></p><p>"name": "jane.doe",</p><p>"id": 235</p><p>},</p><p>"event": {<!-- --></p><p>"success": true</p><p>}</p><p>}</p>

2. Windows Event logs

It contain data relating to events that occur on the Windows operating system. Security, application, system, and DNS events are some examples of Windows Event logs, and they all use the same log format.

Example:

An account was successfully logged on.<p>Subject:</p><p>Security ID: SYSTEM</p><p>Account Name: DESKTOP-LLHJ389$</p><p>Account Domain: WORKGROUP</p><p>Logon ID: 0x3E7</p><p>Logon Information:</p><p>Logon Type: 7</p><p>Restricted Admin Mode: -</p><p>Virtual Account: No</p><p>Elevated Token: No</p><p>Impersonation Level: Impersonation</p><p>New Logon:</p><p>Security ID: AzureADRandyFranklinSmith</p><p>Account Name: rsmith@montereytechgroup.com</p><p>Account Domain: AzureAD</p><p>Logon ID: 0xFD5113F</p><p>Linked Logon ID: 0xFD5112A</p><p>Network Account Name: -</p><p>Network Account Domain: -</p><p>Logon GUID: {00000000-0000-0000-0000-000000000000}</p><p>Process Information:</p><p>Process ID: 0x30c</p><p>Process Name: C:WindowsSystem32lsass.exe</p><p>Network Information:</p><p>Workstation Name: DESKTOP-LLHJ389</p><p>Source Network Address: -</p><p>Source Port: -</p><p>Detailed Authentication Information:</p><p>Logon Process: Negotiate</p><p>Authentication Package: Negotiate</p><p>Transited Services: -</p><p>Package Name (NTLM only): -</p><p>Key Length: 0</p>

3. Common Event Format (CEF)

It is an open, text-based log format used by security-related devices and applications. It is used when collecting and aggregating data by SIEM and log management systems. CEF logs use UTF-8 encoding and include a common prefix, a CEF header, and a variable extension that contains a list of key-value pairs.

Here is an example entry that uses CEF:

CEF:0|Trend Micro|Deep Security Manager||600|User Signed In|3|src=10.52.116.160 suser=admin target=admin msg=User signed in from 2001:db8::5

4. NCSA Common Log Format (CLF)

It is one of the oldest log formats used by web servers. It’s a standardized, text-based log file with a fixed format, which means you can’t customize the fields.

Example:

127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326

5. Extended Log Format (ELF):

ELF is used by web applications. It is similar to CLF but contains more information and flexibility over which fields are used. Fields are separated by white space, and a hyphen represents a missing field.

Example:

#Version: 1.0
#Date: 12-Jan-1996 00:00:00
#Fields: time cs-method cs-uri
00:34:23 GET /foo/bar.html
12:21:16 GET /foo/bar.html
12:45:52 GET /foo/bar.html
12:57:34 GET /foo/bar.html

6. W3C Extended Log File Format:

W3C is a highly customizable log format used by Windows IIS servers. You can configure which fields to include, helping to reduce the size of the log files and keep only relevant information. available fields includes:

Timestamp, Client IP, Server IP, URI-Stem, HTTP Status Code, Bytes Sent, Bytes Received, Time Taken ,Version.

Example:

#Software: Internet Information Services 6.0<p>#Version: 1.0</p><p>#Date: 2001-05-02 17:42:15</p><p>#Fields: time c-ip cs-method cs-uri-stem sc-status cs-version</p><p>17:42:15 172.16.255.255 GET /default.htm 200 HTTP/1.0</p>

Now, thats enought for today. so many things are there to study deeply we just try to give overview of it . if you are more iterested you can check below blog written by Crowdstrike @officals.

Ok, see you next time! with new content with new spirit !!!