socklog - network logging

Logging via network connection cannot be made reliable. There is always a possibility for failures. The network connection itself may be down or the receiving machine has crashed for example. So there must be a decision, what to do in such cases.

The socklog network logging concept

logging is still done locally through multilog
the log is transmitted through a network connection (if possible) when multilog decides that current is big enough, using multilog's processor feature.
since this transmission cannot be made reliable, there is a tool tryto that limits the time to try this transmission. See below for examples.
if the transmission of current fails, the log will be saved locally, multilog remembers the failure in its state and tryto includes the error messages into the transmission on the next run, so the remote machine gets the information of log transmission failures.

There are no restrictions on how to transmit the log data, a separate process of your choice will do the work, e.g. tcpclient or netcat. This modularity lets you easily insert authentication, compression, encryption and other things.

Example setup

Log Server (machine receiving log data)

Setup a socklog-ucspi-tcp service as described in Configuration with the following socklog-ucspi-tcp/run and socklog-ucspi-tcp/log/run scripts:

socklog-ucspi-tcp/run:

  #!/bin/sh
  PORT=10116
  exec 2>&1
  exec softlimit -m 2000000 \
    envuidgid log tcpserver -vUHR -l0 0 $PORT socklog ucspi TCPREMOTEIP

socklog-ucspi-tcp/log/run:

  #!/bin/sh
  LOGDIR=/var/log/socklog-remote
  exec 2>&1
  exec setuidgid log multilog ${LOGDIR}/all \
    -* +'10.0.0.236:*' ${LOGDIR}/10.0.0.236

You will then find all log data from remote hosts that was successfully transmitted in ${LOGDIR}/all/. Log data from 10.0.0.236 will also be saved in ${LOGDIR}/10.0.0.236/.

Log client (machine sending log data)

Change the socklog configuration to use a processor to transmit the log data:

socklog-unix/log/run:

  #!/bin/sh
  LOGDIR=/var/log/socklog
  LOGSERVERIP=10.0.0.16
  PORT=10116
  exec setuidgid log multilog s4096 n20 \
    !"tryto -pv tcpclient -v $LOGSERVERIP $PORT sh -c 'cat >&7'" \
    ${LOGDIR}/main

and restart the service:

  # svc -t /service/socklog-unix/log

On each rotation of multilog's current, the data will be transmitted to $LOGSERVERIP:$PORT using tryto and tcpclient, failures will be noticed and notified on the next run.

Gerrit Pape <pape@smarden.org>

$Id: network.html,v 1.12 2002/08/07 21:45:07 pape Exp $