[Apache] Understanding access log format settings

Hello

This is Miyazaki from the System Solutions Department

In this article, I would like to give a brief introduction to formatting Apache access logs

Server Settings

OS version

[root@Webserver local]# cat /proc/version Linux version 4.9.51-10.52.amzn1.x86_64 (mockbuild@gobi-build-64010) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Fri Sep 29 01:16:19 UTC 2017

Apache version

[root@localhost]# httpd -v Server version: Apache/2.4.27 (Amazon)

The Apache configuration file is written in /etc/httpd/conf/httpd.conf, depending on the OS

Apache access log output destination

The output destination for the access log is determined by CustomLog

[root@Webserver ~]# less /etc/httpd/conf/httpd.conf CustomLog "logs/access_log" combined

CustomLog uses relative paths by default.
These relative paths are represented by the "ServerRoot" field.

The "ServerRoot" setting is also written in this httpd.conf file

[root@Webserver ~]# less /etc/httpd/conf/httpd.conf ServerRoot "/etc/httpd"

In other words, the location of CustomLog is

Since it's ServerRoot/logs/access_log,
it becomes /etc/httpd/logs/acces_log.

[root@Webserver ~]# ls -l /etc/httpd/ total 12 drwxr-xr-x 2 root root 4096 Oct 27 13:36 conf drwxr-xr-x 2 root root 4096 Oct 27 13:36 conf.d drwxr-xr-x 2 root root 4096 Oct 27 13:36 conf.modules.d lrwxrwxrwx 1 root root 14 Oct 27 13:36 logs -> /var/log/httpd lrwxrwxrwx 1 root root 24 Oct 27 13:36 modules -> /usr/lib64/httpd/modules lrwxrwxrwx 1 root root 14 Oct 27 13:36 run -> /var/run/httpd

/etc/httpd/logs/ has been replaced with /var/log/httpd.
This means the location of the access log is now /var/log/httpd/access_log.

[root@Webserver ~]# ls -l /var/log/httpd total 8 -rw-r--r-- 1 root root 1285 Oct 27 13:44 access_log -rw-r--r-- 1 root root 1832 Oct 27 15:02 error_log

Apache access log format

The format of the Apache access log is determined by the LogFormat section in /etc/httpd/conf/httpd.conf

[root@Webserver ~]# less /etc/httpd/conf/httpd.conf LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %b" common

For details on custom log formats, please refer to the official Apache documentation below:
http://httpd.apache.org/docs/2.4/mod/mod_log_config.html#formats

The last part of LogFormat, such as "combined" or "common", is called the nickname

The CustomLog, which is the output destination for the access logs mentioned earlier, also contains the nickname, and
the LogFormat format with the same nickname will be output to the access logs.

CustomLog "logs/access_log" combined

This CustomLog has a combined nickname,

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined LogFormat "%h %l %u %t \"%r\" %>s %b" common

The access log will be output in the format of the LogFormat above

In fact, the access log with nickname in combined format looks like this:

[root@Webserver ~]# less /var/log/httpd/access_log XXX.XXX.XXX.XXX - - [27/Oct/2017:04:44:01 +0000] "GET / HTTP/1.1" 403 4891 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"

Next, let's edit the CustomLog and LogFormat settings in /etc/httpd/conf/httpd.conf and view the access log

[root@Webserver ~]# vi /etc/httpd/conf/httpd.conf #LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined #LogFormat "%h %l %u %t \"%r\" %>s %b" common LogFormat "%h %l %u %t" test #CustomLog "logs/access_log" combined CustomLog "logs/access_log" test

Let's examine the access logs after removing everything after %t in the LogFormat and setting the nickname to "test".
What kind of output will we see?

[root@Webserver ~]# less /var/log/httpd/access_log 200.XXX.XXX.20 - - [27/Oct/2017:05:05:36 +0000]

As per the LogFormat format, up to %t, which is the time the request was received, is displayed

summary

I looked into the formatting of access logs, something I had been wondering about myself.
Actually editing httpd.conf and changing the displayed items, then
checking the access logs, made it much easier to understand.

That's all