how to manage journalctl logging system on rhel 7

How To Manage and Use Journalctl Logging System on RHEL 7

i | P a g e

Table of Contents Overview ....................................................................................................................................................... 1

Applies To ...................................................................................................................................................... 1

Pre-requisites ................................................................................................................................................ 1

journald.conf – Configurable Values Table ............................................................................................... 1

journald.conf – Attribute Purpose Table .................................................................................................. 2

Default Setting – Configuration File – journald.conf ................................................................................ 3

journalctl – List Entries – Old Top ............................................................................................................. 4

journalctl – List Entries – New Top ........................................................................................................... 4

journalctl – List Entries – Tail .................................................................................................................... 5

journalctl – List Entries – Tail N Lines ........................................................................................................ 5

journalctl – List Entries – Tail Real-time .................................................................................................... 6

journalctl – List Entries – From a Time ...................................................................................................... 6

journalctl – List Entries – Current Boot ..................................................................................................... 6

journalctl – List Entries – Kernel Logs ....................................................................................................... 7

journalctl – List Entries – Between Timeframes ....................................................................................... 7

journalctl – List Entries – From Today ....................................................................................................... 7

journalctl – List Entries – From Yesterday ................................................................................................ 8

Filter Message – By UID (User ID) ............................................................................................................. 8

Filter Message – By GID (Group ID) .......................................................................................................... 8

Filter Message – By PID (Process ID) ......................................................................................................... 9

Filter Message – By Unit (service) ............................................................................................................. 9

Filter Message – By Unit (service) - Verbose .......................................................................................... 10

Filter Message – By Unit (service) - Debugging ...................................................................................... 11

Filter Message – By Unit from today ...................................................................................................... 11

Filter Message – By Hours ago ................................................................................................................ 12

Filter Message – By Minutes ago ............................................................................................................ 12

Filter Message – By Seconds ago ............................................................................................................ 12

Filter Message – By DateTime................................................................................................................. 13

Filter Message – By Syslog Priority ......................................................................................................... 13

Filter Message – By Priority Name ...................................................................................................... 13

Filter Message – By Priority Number .................................................................................................. 13


ii | P a g e

Filter Message – By Executable ........................................................................................................... 14

Journalctl – Service Management ............................................................................................................... 14

Start system-journald.............................................................................................................................. 14

Stop system-journald .............................................................................................................................. 14

Restart system-journald .......................................................................................................................... 14

Status system-journald ........................................................................................................................... 15

Verify Corruption .................................................................................................................................... 15

Disk Space Consumed ............................................................................................................................. 15


1 | P a g e

Overview

The guides will help in understanding the RHEL / CentOS 7 logging system “journalctl”. Journald Daemon

centralizes the management of logs regardless of where the messages are originated.

Applies To

RHEL 7, CentOS 7

Pre-requisites

None

journald.conf – Configurable Values Table

Configuration Attribute Configurable Values

Storage "volatile", "persistent", "auto" and "none"

Compress "yes", "no"

Seal "yes", "no"

SplitMode "login", "uid" and "none"

SyncIntervalSec User defined

RateLimitInterval User defined

RateLimitBurst User defined

SystemMaxUse Customizable, applies when Storage set to persistent

SystemKeepFree Customizable, applies when Storage set to persistent

SystemMaxFileSize Customizable, applies when Storage set to persistent

RuntimeMaxUse Customizable, applies when Storage set to persistent

RuntimeKeepFree Customizable, applies when Storage set to persistent

RuntimeMaxFileSize Customizable, applies when Storage set to persistent

MaxRetentionSec User defined

MaxFileSec User defined

ForwardToSyslog "yes", "no"

ForwardToKMsg "yes", "no"

ForwardToConsole "yes", "no"

TTYPath /dev/console

MaxLevelStore "emerg", "alert", "crit", "err", "warning", "notice", "info", "debug" or integer value from 0 – 7

MaxLevelSyslog "emerg", "alert", "crit", "err", "warning", "notice", "info", "debug"

MaxLevelKMsg or integer value from 0 – 7

MaxLevelConsole "emerg", "alert", "crit", "err", "warning", "notice", "info", "debug"


2 | P a g e

journald.conf – Attribute Purpose Table

Configuration Attribute Purpose

Storage Where to store journal data, for persistent data storage, create /run/log/journal/

Compress Takes a boolean value. If enabled (the default)

Seal Takes a boolean value. If enabled (the default)

SplitMode Controls whether to split up journal files per user

SyncIntervalSec Configures the rate limiting that is applied to all messages generated on the system

RateLimitInterval Configures the rate limiting that is applied to all messages generated on the system. If, in the time interval defined by RateLimitInterval=, more messages than specified in RateLimitBurst= are logged by a service, all further messages within the interval are dropped until the interval is over RateLimitBurst

SystemMaxUse Enforce size limits on the journal files stored. The options prefixed with "System" apply to the journal files when stored on a persistent file system, more specifically /var/log/journal. The options prefixed with "Runtime" apply to the journal files when stored on a volatile in-memory file system, more specifically /run/log/journal. The former is used only when /var is mounted, writable, and the directory /var/log/journal exists. Otherwise, only the latter applies.

SystemKeepFree

SystemMaxFileSize

RuntimeMaxUse

RuntimeKeepFree

RuntimeMaxFileSize

MaxRetentionSec The maximum time to store journal entries. This controls whether journal files containing entries older then the specified time span are deleted.

MaxFileSec The maximum time to store entries in a single journal file before rotating to the next one.

ForwardToSyslog Control whether log messages received by the journal daemon shall be forwarded to a traditional syslog daemon, to the kernel log buffer (kmsg), to the system console, or sent as wall messages to all logged-in users.

ForwardToKMsg

ForwardToConsole

ForwardToWall

TTYPath Change the console TTY to use if ForwardToConsole=yes

MaxLevelStore Controls the maximum log level of messages that are stored on disk, forwarded to syslog, kmsg, the console or wall (if that is enabled) MaxLevelSyslog

MaxLevelKMsg

MaxLevelConsole

MaxLevelWall


3 | P a g e

Default Setting – Configuration File – journald.conf

Logs are controlled by file “journald.conf” which is located in the folder “/etc/systemd/”.

[Journal]

Storage=

…….

The below snippet is default setting configuration file “journald.conf” for journalctl daemon.


4 | P a g e

journalctl – List Entries – Old Top

List all journal entry that is in the system will be displayed within a pager. The oldest entries will be

displayed at top.

journalctl

journalctl – List Entries – New Top

List all journal entry that is in the system will be displayed within a pager. The newest entries will be

displayed at top.

journalctl --reverse


5 | P a g e

journalctl – List Entries – Tail

List all journal entry that is in the system will be displayed. Last 10 lines would be displayed, similar to

running tail /var/log/messages.

journalctl -n

journalctl – List Entries – Tail N Lines

List all journal entry that is in the system will be displayed. Last 15 lines would be displayed, similar to

running tail /var/log/messages -n 15.

journalctl -n 15


6 | P a g e

journalctl – List Entries – Tail Real-time

List all journal entry that is in the system will be displayed in real-time. Last 15 lines would be displayed,

similar to running tail /var/log/messages -f.

journalctl -f

journalctl – List Entries – From a Time

List all journal entries that is in the system starting from specific time onwards.

journalctl --since 02:50

journalctl – List Entries – Current Boot

List all journal entries that is in the system starting from current boot.

journalctl -b


7 | P a g e

journalctl – List Entries – Kernel Logs

List all journal entries pertaining to that of kernel, that is in the system will be displayed.

journalctl -k

journalctl – List Entries – Between Timeframes

List all journal entries that is in the system starting from specific time onwards and until a specific time.

journalctl --since 02:00 --until 02:50

journalctl – List Entries – From Today

List all journal entries that is in the system for today only.

journalctl --since=today


8 | P a g e

journalctl – List Entries – From Yesterday

List all journal entries that is in the system for yesterday onwards and till now.

journalctl --since=yesterday

Filter Message – By UID (User ID)

List all journal entries that is in the system for a User ID.

journalctl _UID=1000

Filter Message – By GID (Group ID)

List all journal entries that is in the system for a Group ID.

journalctl _GID=1000


9 | P a g e

Filter Message – By PID (Process ID)

List all journal entries that is in the system for Process ID.

journalctl _PID=1

Filter Message – By Unit (service)

List all journal entries that is in the system for a unit name (service).

journalctl -u httpd.service


10 | P a g e

Filter Message – By Unit (service) - Verbose

List all journal entries that is in the system for a unit name (service) in verbose mode.

journalctl -f -o verbose UNIT=httpd.service


11 | P a g e

Filter Message – By Unit (service) - Debugging

List all journal entries that is in the system for a unit name (service) for debugging.

journalctl -f -u httpd.service -l

Filter Message – By Unit from today

Alternatively, you can filter messages by today, yesterday, since and until as well.

journalctl -u httpd.service --since=today


12 | P a g e

Filter Message – By Hours ago

You can filter messages by hours, minutes and seconds elapsed as well.

journalctl -u httpd.service --since "20 hours ago"

Filter Message – By Minutes ago

You can filter messages by minutes, hours and seconds elapsed as well.

journalctl -u httpd.service --since "1460 min ago"

Filter Message – By Seconds ago

you can filter messages by seconds, hour and minutes elapsed as well.

journalctl -u httpd.service --since "40 sec ago"


13 | P a g e

Filter Message – By DateTime

you can also filter messages by, from datetime to from datetime using the datetime format "yyyy-mm-dd

hh:mm:ss" elapsed as well.

journalctl -u httpd --since "2015-11-16 23:15:00" --until "2015-11-17 23:20:00"

Filter Message – By Syslog Priority

List all journal entries that is in the system by syslog priority.

Value Severity

0 Emergency

1 Alert

2 Critical

3 Error

4 Warning

5 Notice

6 Informational

7 Debug

Filter Message – By Priority Name

journalctl -p err

or

Filter Message – By Priority Number

journalctl -p 3


14 | P a g e

Filter Message – By Executable

journalctl /usr/bin/dbus-daemon

journalctl /usr/lib/systemd/systemd

Journalctl – Service Management

Start system-journald

systemctl start systemd-journald

Stop system-journald

systemctl stop systemd-journald

Note: You will not be able to stop journalctl service, because of dependency with systemd

Restart system-journald

systemctl restart systemd-journald


15 | P a g e

Status system-journald

systemctl status systemd-journald

Verify Corruption

Check the journal file for internal consistency.

journalctl --verify

Disk Space Consumed

To check disk space consumed by all the journals files.

journalctl --disk-usage

how to manage journalctl logging system on rhel 7

Technology