2009-12-17

Remote Apache logging with syslog: is there anything better?

We're doing a pilot to do remote logging for some Apache logs (possibly other services in the future). We've heard of remote syslog capability (and since syslog is on all Linux system) we're giving it a try.

The configuration is quite simple:
  1. [srv] prepare a machine to do the log server
  2. [srv] open it's rsyslogd to receive messages on UDP (or TCP) port
  3. [srv] log the selected facility.level to a log file
  4. [cli] forward the above facility.level to @srv
Ok, easy and it works. Unfortunately we face pretty fast the limitation of syslog:
  1. only 8 facilities for users custom log (local0-local7)
  2. only 8 levels for logs severity
this is a big block (only 64 combinations, if you agree to do some "dirty" stuff) if you want to log remotely several services on several different platform on the syslog server.

rsyslog is quite flexible and it allows you to filter messages based on the tag in them, and log in different files, but it's still something very "home-made" and not that professional.

I don't we're the only one needing a remote logging tool, and while syslog is the classic solution, is very bind to the system logs and not to the applications logs: any suggestions for this task? I'd like to hear how you solved this task, possibly without a custom tool, but using something standard.

5 comments:

Ivan Ristić said...

Assuming you want to keep your logging servers on the same network as the web servers, try mod_log_spread.

Antoine said...

Could you use non standard port ?

Sandro Tosi said...

@Ivan: thanks for the suggestion: we'll look into it for apache log. Sadly, it's only for apache (while we might also considering logging other/custom services).

@Antoine: sure, we can use any port (if not already reserved) we need: it's just for logging inside the datacenter lan.

Justin said...

Scribe?

Sandro Tosi said...

@Justin: oh god, scribe is damn cool! it's just a bit difficult to compile (we use also non-debian systems) but it seems to be exactly what we need. Thanks a lot for this suggestion.