$_ BSDHowTo.ch
How To... Why not..? Scripts Patches RSS logo

How to log client IP in httpd behind relayd

Last update: 2019-05-12

Introduction

You have set up relayd(8) in front of your httpd(8) server(s). Now you check /var/www/logs/access.log and realize that the only client IP you see in it is the IP of your relayd(8) server. In this post I show you a way to deal with this situation.

Problem and solution description

If you run a reverse proxy in front of a HTTP server, the server will log the IP address of the reverse proxy as source IP for all requests. The original information is not lost, it is logged by the reverse proxy. Matching the entries in the proxy log with the entries in the server log is tricky and error prone. It would be better if the server logs the original connection information too.

The industry has developed an informal pseudo-standard for this: The X-Forwarded-{For|Port|By} HTTP pseudo headers. A reverse proxy can store information like IP and port number in these headers and the HTTP server can use the headers for logging.

As of OpenBSD 6.6 httpd(8) comes with a new log format called forwarded. It extends the existing format combined by appending two extra fields to each line:

  1. The content of X-Forwarded-For header
  2. The content of X-Forwarded-Port header

If any of the two is missing or empty, a dash (-) is printed to the access log. To use the new log format you must adapt your httpd.conf(5) file. If you have a line like

log combined

you can simply change it to

log forwarded

Details about the configuration format including the new option log forwarded are described in httpd.conf(5).

Configuration of relayd(8)

You must tell relayd(8) - or whatever reverse proxy you use - to set the two headers X-Forwarded-For and X-Forwarded-By in the connections it passes to httpd(8). Else you will not see any entries in your access log. Edit your relayd.conf(5) and add these two lines to the appropriate http protocol block:

match request header set "X-Forwarded-For" value "$REMOTE_ADDR"
match request header set "X-Forwarded-Port" value "$REMOTE_PORT"

Using log analyzers

The new format works for human readers, but automated log analyzers like Webalizer and GoAccess do not know about this custom log format. With a small awk script you can transform the forwarded format to combined without losing the information about the client IP.

#!/usr/bin/awk -f

{
    ip = $(NF - 1)
    sub(/127.0.0.1/, ip)
    for (i = 2; i <= NF - 2; i++) {
        printf("%s ", $i)
    }
    printf("\n")
}

convert.awk

The script reads the client IP from the second-last field of the line and replaces the proxy IP - in this case 127.0.0.1 - with the client IP. It then prints all fields of the line except the first one and the last two to stdout.

You can run the script like this:

$ awk -f convert.awk access.log > combined.log

If you use newsyslog(8) to rotate your access.log you should cut the first line which contains a notification about the last time the log file was rotated. Just modify the above command like this:

$ awk -f convert.awk access.log | sed '1d' > combined.log

You can now feed the new file combined.log to the log analyzer you trust.