nginx redislog module

This is a high-performance nginx module for logging http requests into Redis database. Turn your Redis database into log file storage using this module!

Ever wanted to centralize your log files on one machine or organize your logfiles better? This module is for you!

Redis is a simple yet fast database server that perfectly fits for storing log files. This module links together a fast web server with a fast database server:

• you can define multiple log destinations (redis servers)
• you can configure the module to keep daily, hourly, as well as monthly and yearly log files
• any nginx variable can be used as log file name (e.g. domain, subdomain, location, etc)
• restarting nginx in order to rotate logs is not required anymore!

The module keeps persistant connections to Redis database servers, reconnecting if necessary.

Developed and tested with nginx version 1.3.x, might work with earlier as well.

How to use

Download sources and unpack the archive:

$ unzip nginx-redislog-module-1.0.0.zip

Configure nginx with additional module:

cd [nginx installation directory]
./configure --add-module=[path to redislog module]
make
make install

error_log  logs/error.log  debug;

events {
    worker_connections  1024;
}

redislog test localhost;

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    server {
        listen       80;
        server_name  localhost;

        access_redislog test "$server_name$redislog_yyyymmdd";
    }
}

redislog

syntax: redislog <peer name> <hostname>[:<port>] [<password>]

default: none

severity: mandatory

context: main configuration file

Specifies the name of a log peer, its redis hostname and optional password. Once specified, the peer name can be used in access_redislog directives. hostname can be both domain name and IP address of a destination redis database. A password needs to be specified in plain text as long as requirepass redis configuration directive is used.

access_redislog

syntax: access_redislog *<off>|<peer name>* *<key template>* *[<arguments...>]*

default: none

severity: optional

context: http, server, location

Enables or disables logging into specified redis peer. Once enabled at certain level, same logging settings are inherited by subsequent levels. off argument disables logging into redis for specific location. Multiple peers are allowed at the same location. Log messages are then sent into all specified peers. key template specifies a template for redis database key where log messages will be appended to. Besides standard nginx variable following variables are allowed:

• $redislog_yyyy -- 4 digit number of current year
• $redislog_yyyymm -- 4 digit number of current year plus 2 digit number of current month
• $redislog_yyyymmdd -- current year, month and day
• $redislog_yyyymmddhh -- current year, month, day and hour

As of version 1.0.1 this directive supports following arguments:

command -- Specifies redis command that will be used to save log messages (APPEND is default). Any command can be used, following are most useful: RPUSH, LPUSH, SADD, SREM, EVALSHA predefinedsha1. Use arg1 in order to specify the first argument of commands with 2 argument (see below).

arg1 -- Specifies the first argument for command with 2 argument, e.g. EVALSHA.

format -- Specifies a log format that will be used to format log messages (default is 'combined').

if -- Specifies an expression for conditional logging. If set, a request will be logged only if this expression evaluates to true (any non-empty value except for literal "0").

ifnot -- Specifies an expression for conditional logging. If set, a request will be logged only if this expression evaluates to false (any empty value or literal "0").

Example: accessredislog test $servername$redislog_yyyymmdd;

accessredislog test $servername$redislogyyyymmdd ifnot=$uidgot; # Logs only unique visitors

Creates a key for each hostname and each day.

Use redis GET command to retrieve logging data.