Syslog Fundamentals: Protocol Anatomy and rsyslog Configuration

The "lingua franca" of the security log ecosystem—from protocol formats to hands-on configuration, master the underlying principles of log collection

Introduction: Why can't security products do without syslog?

In modern security infrastructure,SIEM(Security Information and Event Management),HIDS(Host Intrusion Detection System),NIDS(Network Intrusion Detection System) all need to centrally collect logs from various devices. And syslog is precisely the "lingua franca" of this ecosystem:

• Firewall, router, switch → syslog

• Linux server's sshd,sudo,cron→ syslog

• Bastion host, database audit → syslog

As a security developer, understanding the details of the syslog protocol is Correctly parse logs to avoid missed alarms and false positives. the foundation of.

• Introduction: Why can't security products do without syslog?
• In-depth dissection of the syslog protocol format
  • Full format vs original format
  • PRI calculation and inverse solution (Golang parsing example)
  • HEADER: Timestamp and hostname
  • MSG：TAG + CONTENT
• Facility 与 Priority：安全日志的分类与分级
  • Safety-related Facility
  • Mapping from Priority to alarm severity
• rsyslog Configuration in Practice
  • Basic configuration structure
  • Common configurations for security products
  • Security implications of log files
• Golang Fundamentals Explained: Complete Example
• Practical Key Points for Security Developers
  • Common parsing pitfalls
  • Compliance requirements for log collection
• Summary
• Reference resources

In-depth dissection of the syslog protocol format

syslog has two main RFC standards:RFC 3164(traditional format, BSD-syslog) and RFC 5424(enhanced format). In security scenarios, both coexist, and compatibility handling is required during parsing.

Full format vs original format

A typical RFC 3164 syslog message:

<34>Oct 11 22:14:15 mymachine su: 'su root' failed for lonvick on /dev/pts/8

Break it down into:

Part	Example	meaning
PRI	<34>	Facility + Priority encoded value
HEADER	Oct 11 22:14:15 mymachine	Timestamp + Hostname
MSG	su: 'su root' failed...	Label (TAG) + Content (CONTENT)

PRI calculation and inverse solution (Golang parsing example)

PRI =Facility × 8 + Priority

• Facility Range: 0-23 (device type, e.g. kern=0, auth=4, daemon=3)

• Priority Range: 0-7 (severity level, 0=emerg, 3=err, 6=info)

Example calculation:<34>→34 / 8 = 4 surplus 2→ Facility=4(auth), Priority=2(crit)

Golang parsing demo:

package main

import (
    "fmt"
    "strconv"
    "strings"
)

// parsePRI parses the PRI field and returns facility and priority
func parsePRI(priStr string) (facility, priority int, err error) {
    priStr = strings.Trim(priStr, "<>")
    pri, err := strconv.Atoi(priStr)
    if err != nil {
        return 0, 0, err
    }
    facility = pri / 8
    priority = pri % 8
    return facility, priority, nil
}

func main() {
    facility, priority, _ := parsePRI("<34>")
    fmt.Printf("Facility: %d (auth), Priority: %d (crit)\n", facility, priority)
}

HEADER: Timestamp and hostname

RFC 3164 timestamp format:Mmm dd hh:mm:ss(No year; needs to be supplemented during safe parsing)

import "time"

// parseRFC3164Time parses an RFC 3164 timestamp (the year must be supplemented)
func parseRFC3164Time(timestamp string, currentYear int) (time.Time, error) {
    layout := "Jan 2 15:04:05"
    t, err := time.Parse(layout, timestamp)
    if err != nil {
        return time.Time{}, err
    }
    return time.Date(currentYear, t.Month(), t.Day(),
        t.Hour(), t.Minute(), t.Second(), 0, time.Local), nil
}

hostname: May be a short hostname or FQDN, used to identify the log source.

MSG：TAG + CONTENT

• TAG: It is usually the process name, such as su,sshd,kernel

• CONTENT: Specific log content

// splitTagContent split TAG and CONTENT
func splitTagContent(msg string) (tag, content string) {
    parts := strings.SplitN(msg, ":", 2)
    if len(parts) == 2 {
        return parts[0], strings.TrimSpace(parts[1])
    }
    return msg, ""
}

Facility 与 Priority：安全日志的分类与分级

Safety-related Facility

Facility	value	security implications
kern	0	Kernel log: network filtering, SELinux, iptables reject
auth	4	Authentication logs: login, su, sudo (non-privileged)
authpriv	10	Privileged authentication logs: SSH login, password change
daemon	3	Daemon: Security incidents of various services
local0~local7	16-23	Custom: Application-layer security logs

Mapping from Priority to alarm severity

Priority	value	Security alert level	Typical events
emerg	0	🔴 Severe	System unavailable
alert	1	🔴 High risk	Requires immediate action (e.g., root being brute-forced)
crit	2	🟠 High risk	Critical service failure (e.g., PAM crash)
err	3	🟡 Medium risk	Authentication failed, permission error
warning	4	🟡 Low risk	Suspicious behavior but not fatal
notice	5	🔵 Information	Important but normal operation
info	6	🔵 Debugging	Regular access logs
debug	7	⚪ Ignore	Debug information

rsyslog Configuration in Practice

Basic configuration structure

rsyslog configuration file /etc/rsyslog.conf Core grammar:

# Module loading
module(load="imudp")      # Load UDP input module
input(type="imudp" port="514")

# Template definition
template(name="DynamicFile" type="string"
         string="/var/log/%HOSTNAME%/%PROGRAMNAME%.log")

# Selector rules
auth,authpriv.*         /var/log/auth.log
kern.*                  /var/log/kern.log
*.emerg                 :omusrmsg:*   # Emergency message broadcast to all users

Common configurations for security products

1. Route by facility, preserving the original format.

# /etc/rsyslog.conf
auth,authpriv.*         /var/log/auth.log
kern.*                  /var/log/kern.log
daemon.*                /var/log/daemon.log
*.emerg                 :omusrmsg:*

2. Template configuration: compatible with RFC 5424 (high-precision timestamps)

# High-precision timestamp template
template(name="RFC5424" type="string"
         string="%timestamp:::date-rfc3339% %hostname% %syslogtag%%msg%\n")
$ActionFileDefaultTemplate RFC5424

3. Recommended configuration for secure collection

# Ensure logs are not lost
module(load="imuxsock")          # Local socket input
$SystemLogRateLimitInterval 0    # Disable rate limiting (use with caution in production)

# Remote reception (enabled when the security product acts as a log center)
module(load="imtcp")
input(type="imtcp" port="514")

Security implications of log files

file path	Recorded security incidents
/var/log/auth.log	SSH login success/failure, sudo execution, user creation/deletion
/var/log/kern.log	iptables deny, SELinux deny, kernel vulnerability alert
/var/log/syslog	Comprehensive logs, including services such as cron and mail
/var/log/btmp	Failed login attempts (binary format, using lastb Read)
/var/log/wtmp	User login/logout history (last Command)
/var/log/secure	Authentication logs for the RedHat/CentOS series

Golang Fundamentals Explained: Complete Example

The following is a demo framework that reads raw syslog from TCP and parses it:

package main

import (
    "bufio"
    "fmt"
    "net"
    "strconv"
    "strings"
    "time"
)

// SyslogMessage represents the parsed syslog message
type SyslogMessage struct {
    Facility  int
    Priority  int
    Severity  string
    Timestamp time.Time
    Hostname  string
    Tag       string
    Content   string
    Raw       string
}

// Mapping from severityMap Priority values to severity level strings
var severityMap = map[int]string{
    0: "emerg", 1: "alert", 2: "crit", 3: "err",
    4: "warning", 5: "notice", 6: "info", 7: "debug",
}

// parseSyslog parses raw syslog messages (simplified RFC 3164 version)
func parseSyslog(raw string) (*SyslogMessage, error) {
    if len(raw) < 4 || raw[0] != '<' {
        return nil, fmt.Errorf("invalid syslog format")
    }

    // Extract PRI
    end := strings.Index(raw, ">")
    if end == -1 {
        return nil, fmt.Errorf("missing PRI terminator")
    }
    pri, _ := strconv.Atoi(raw[1:end])
    facility := pri / 8
    priority := pri % 8

    // Remaining part
    rest := strings.TrimSpace(raw[end+1:])

    // Simple split: timestamp + hostname + message
    parts := strings.SplitN(rest, " ", 3)
    if len(parts) < 3 {
        return nil, fmt.Errorf("invalid syslog header")
    }

    // Parse timestamp (format: Mmm dd hh:mm:ss)
    timestampStr := parts[0] + " " + parts[1]
    timestamp, _ := parseRFC3164Time(timestampStr, time.Now().Year())

    // Hostname and message
    hostname := parts[2]

    return &SyslogMessage{
        Facility:  facility,
        Priority:  priority,
        Severity:  severityMap[priority],
        Timestamp: timestamp,
        Hostname:  hostname,
        Raw:       raw,
    }, nil
}

// parseRFC3164Time parses RFC 3164 format timestamps
func parseRFC3164Time(timestamp string, currentYear int) (time.Time, error) {
    layout := "Jan 2 15:04:05"
    t, err := time.Parse(layout, timestamp)
    if err != nil {
        return time.Time{}, err
    }
    return time.Date(currentYear, t.Month(), t.Day(),
        t.Hour(), t.Minute(), t.Second(), 0, time.Local), nil
}

// handleConnection handles a single TCP connection
func handleConnection(conn net.Conn) {
    defer conn.Close()
    scanner := bufio.NewScanner(conn)
    for scanner.Scan() {
        raw := scanner.Text()
        msg, err := parseSyslog(raw)
        if err != nil {
            fmt.Printf("Parse error: %v\n", err)
            continue
        }
        fmt.Printf("[%s] [%s] Facility=%d Host=%s\n",
            msg.Timestamp.Format("2006-01-02 15:04:05"),
            msg.Severity, msg.Facility, msg.Hostname)
    }
}

func main() {
    ln, err := net.Listen("tcp", ":5514")
    if err != nil {
        panic(err)
    }
    defer ln.Close()

    fmt.Println("🚀 Syslog collector listening on :5514")
    for {
        conn, err := ln.Accept()
        if err != nil {
            continue
        }
        go handleConnection(conn)
    }
}

Practical Key Points for Security Developers

Common parsing pitfalls

Trap	solution
Timestamp without year	Context determination (log file write time vs event time)
Multi-line messages (such as stack traces)	State machine parsing is required; it cannot be simply split by line.
Non-standard format (such as network devices)	Configure devices to use a standard format, or write an adapter
UDP packet loss	Production environments use TCP or RELP protocols.

Compliance requirements for log collection

Standards such as Classified Protection 2.0, GDPR, and PCI-DSS have explicit requirements for logs:

• Integrity: Logs cannot be tampered with (consider signing or secure transmission)

• Availability: Logs must not be lost (TCP + local cache)

• Confidentiality: Transport encryption (TLS, detailed in Part 2)

Summary

This article builds a foundational knowledge of syslog for security developers, from the underlying protocol to hands-on configuration:

• ✅ Understood PRI = Facility × 8 + Priority, can manually parse and construct

• ✅ Differentiate the timestamp format differences between RFC 3164 / 5424

• ✅ Master rsyslog's core configuration syntax and routing strategies

• ✅ Implement a basic syslog TCP receiver in Golang

Next article preview: <Syslog in Practice: Golang Security Product Integration Guide》 will delve into production-grade topics such as TLS certificate generation, high-performance parsing libraries, and SIEM integration, and provide a complete runnable demo.

Reference resources

• RFC 3164 - The BSD syslog Protocol

• RFC 5424 - The Syslog Protocol

• rsyslog Official Documentation

• systemd Log Management in Practice

Series Articles

• Syslog Fundamentals: Protocol Anatomy and rsyslog Configuration(This article)

• Syslog in Practice: Golang Security Product Integration Guide

• Syslog Comparison: A Guide to Choosing Between syslog and journald Logging