fedora.linux_system_roles.logging

CI Testing

Table of Contents

Background

Logging role is an abstract layer for provisioning and configuring the logging system. Currently, rsyslog is the only supported provider.

In the nature of logging, there are multiple ways to read logs and multiple ways to output them. For instance, the logging system may read logs from local files, or read them from systemd/journal, or receive them from the other logging system over the network. Then, the logs may be stored in the local files in the /var/log directory, or sent to Elasticsearch, or forwarded to other logging system. The combination between the inputs and the outputs needs to be flexible. For instance, you may want to inputs from journal stored just in the local file, while inputs read from files stored in the local log files as well as forwarded to the other logging system.

To satisfy such requirements, logging role introduced 3 primary variables logging_inputs, logging_outputs, and logging_flows. The inputs are represented in the list of logging_inputs dictionary, the outputs are in the list of logging_outputs dictionary, and the relationship between them are defined as a list of logging_flows dictionary. The details are described in Logging Configuration.

Requirements

This role is supported on RHEL/CentOS-7, RHEL/CentOS-8 and Fedora distributions.

Definitions

Logging Configuration

Brief overview

Logging role allows to have variables logging_inputs, logging_outputs, and logging_flows with additional options to configure logging system such as rsyslog.

Currently, the logging role supports four types of logging inputs: basics, files, ovirt, and remote. And four types of outputs: elasticsearch, files, forwards, and remote_files. To deploy configuration files with these inputs and outputs, specify the inputs as logging_inputs and the outputs as logging_outputs. To define the flows from inputs to outputs, use logging_flows. The logging_flows has three keys name, inputs, and outputs, where inputs is a list of logging_inputs name values and outputs is a list of logging_outputs name values.

This is a schematic logging configuration to show log messages from input_nameA are passed to output_name0 and output_name1; log messages from input_nameB are passed only to output_name1.

---
- name: a schematic logging configuration
  hosts: all
  roles:
    - fedora.linux_system_roles.logging
  vars:
    logging_inputs:
      - name: input_nameA
        type: input_typeA
      - name: input_nameB
        type: input_typeB
    logging_outputs:
      - name: output_name0
        type: output_type0
      - name: output_name1
        type: output_type1
    logging_flows:
      - name: flow_nameX
        inputs: [input_nameA]
        outputs: [output_name0, output_name1]
      - name: flow_nameY
        inputs: [input_nameB]
        outputs: [output_name1]

Variables

Logging_inputs options

logging_inputs: A list of following dictionary to configure inputs.

Logging_outputs options

logging_outputs: A list of following dictionary to configure outputs.

Logging_flows options

Security options

These variables are set in the same level of the logging_inputs, logging_output, and logging_flows.

Server performance optimization options

These variables are set in the same level of the logging_inputs, logging_output, and logging_flows.

Other options

These variables are set in the same level of the logging_inputs, logging_output, and logging_flows.

Update and Delete

Due to the nature of ansible idempotency, if you run ansible-playbook multiple times without changing any variables and options, no changes are made from the second time. If some changes are made, only the rsyslog configuration files affected by the changes are recreated. To delete any existing rsyslog input or output config files generated by the previous ansible-playbook run, you need to add “state: absent” to the dictionary to be deleted (in this case, input_nameA and output_name0). And remove the flow dictionary related to the input and output as follows.

logging_inputs:
  - name: input_nameA
    type: input_typeA
    state: absent
  - name: input_nameB
    type: input_typeB
logging_outputs:
  - name: output_name0
    type: output_type0
    state: absent
  - name: output_name1
    type: output_type1
logging_flows:
  - name: flow_nameY
    inputs: [input_nameB]
    outputs: [output_name1]

If you want to remove all the configuration files previously configured, in addition to setting state: absent to each logging_inputs and logging_outputs item, add logging_enabled: false to the configuration variables as follows. It will eliminate the global and common configuration files, as well.

logging_enabled: false
logging_inputs:
  - name: input_nameA
    type: input_typeA
    state: absent
  - name: input_nameB
    type: input_typeB
    state: absent
logging_outputs:
  - name: output_name0
    type: output_type0
    state: absent
  - name: output_name1
    type: output_type1
    state: absent
logging_flows:
  - name: flow_nameY
    inputs: [input_nameB]
    outputs: [output_name1]

Configuration Examples

Standalone configuration

  1. Deploying basics input reading logs from systemd journal and implicit files output to write to the local files.
    ```yaml
    —
  2. Deploying basics input reading logs from systemd unix socket and files output to write to the local files.
    ```yaml
    —
  3. Deploying basics input reading logs from systemd journal and files output to write to the individually configured local files.
    ```yaml
    —
  4. Deploying files input reading logs from local files and files output to write to the individually configured local files.
    ```yaml
    —
  5. Deploying files input reading logs from local files and files output to write to the local files based on the property-based filters.
    ```yaml
    —

Client configuration

  1. Deploying basics input reading logs from systemd journal and forwards output to forward the logs to the remote rsyslog.
    ```yaml
    —
  2. Deploying files input reading logs from a local file and forwards output to forward the logs to the remote rsyslog over tls. Assuming the ca_cert, cert and key files are prepared at the specified paths on the control host. The files are deployed to the default location /etc/pki/tls/certs/, /etc/pki/tls/certs/, and /etc/pki/tls/private, respectively.
    ```yaml
    —

Server configuration

  1. Deploying remote input reading logs from remote rsyslog and remote_files output to write the logs to the local files under the directory named by the remote host name.
    ```yaml
    —
  2. Deploying remote input reading logs from remote rsyslog and remote_files output to write the logs to the configured local files with the tls setup supporting 20 clients. Assuming the ca_cert, cert and key files are prepared at the specified paths on the control host. The files are deployed to the default location /etc/pki/tls/certs/, /etc/pki/tls/certs/, and /etc/pki/tls/private, respectively.
    ```yaml
    —

Client configuration with Relp

Deploying basics input reading logs from systemd journal and relp output to send the logs to the remote rsyslog over relp.

---
- name: Deploying basics input and relp output
  hosts: clients
  roles:
    - fedora.linux_system_roles.logging
  vars:
    logging_inputs:
      - name: basic_input
        type: basics
    logging_outputs:
      - name: relp_client
        type: relp
        target: logging.server.com
        port: 20514
        tls: true
        ca_cert_src: /path/to/ca.pem
        cert_src: /path/to/client-cert.pem
        private_key_src: /path/to/client-key.pem
        pki_authmode: name
        permitted_servers:
          - '*.server.com'
    logging_flows:
      - name: flow
        inputs: [basic_input]
        outputs: [relp_client]

Server configuration with Relp

. Deploying relp input reading logs from remote rsyslog and remote_files output to write the logs to the local files under the directory named by the remote host name.

---
- name: Deploying remote input and remote_files output
  hosts: server
  roles:
    - fedora.linux_system_roles.logging
  vars:
    logging_inputs:
      - name: relp_server
        type: relp
        port: 20514
        tls: true
        ca_cert_src: /path/to/ca.pem
        cert_src: /path/to/server-cert.pem
        private_key_src: /path/to/server-key.pem
        pki_authmode: name
        permitted_clients:
          - '*.client.com'
          - '*.example.com'
    logging_outputs:
      - name: remote_files_output
        type: remote_files
    logging_flows:
      - name: flow
        inputs: [relp_server]
        outputs: [remote_files_output]

Port and SELinux

SELinux is only configured to allow sending and receiving on the following ports by default:

syslogd_port_t        tcp   514, 20514
syslogd_port_t        udp   514, 20514

If other ports need to be configured, you can use linux-system-roles/selinux to manage SELinux contexts.

Providers

Rsyslog - This documentation contains rsyslog specific information.

Tests

Automated CI-tests - This documentation shows how to execute CI tests in the tests directory as well as how to debug when the test fails.