Extracting unique page visits from the access.log

Luca Nerlich, tech
Back

Table of Contents

Option 1 - Visitors

Hey there! In this post, I am going to quickly explain, how one can find the count of unique visitors on any server running nginx.

This how-to has been tested on Ubuntu Server 18.04.2 LTS.

To reach this goal, we are going to use the tool Visitors. We will have to download the sourcecode, compile it to a usable program and feed it our access.log[^1].

Lets go.

  1. Create a new directory where you want to place visitors.

  2. cd into this directory and download the sourcecode. At the time of writing, the current version is 0.7.

    cd ~/your/path && wget http://www.hping.org/visitors/visitors-0.7.tar.gz
  3. This archive needs to be unzipped..

    tar -xzvf visitors-0.7.tar.gz
  4. The sourcecode needs to be compiled.

    make
    • If make fails or you are unable to execute it, your machine needs the essential build tools.

      # as sudo
      apt-get install build-essential
  5. We are now able to let visitors parse the access.log[^1] and generate a report.html file.

    ~/your/path/visitors_0.7/visitors /var/log/nginx/access.log > report.html

If done correctly, Visitors will have written its report into the report.html file inside its root folder. cd ~/your/path/visitors_0.7/.

Since you are most likely on a commandline only machine, you will need a handy solution to view this .html file. For this, we will be using Lynx - a text only webbrowser.

All common distributions will have Lynx in its repository. Therefore one can easily install it via the following command:

apt-get install lynx

Feeding Lynx our report.html, gives you the opportunity to step through the report, using your keyboard.

The page will look similar to this: lynx example

Text written in Green, represent links.

Using your keyboard, right will follow this link, whereas left will go back. up and down let you navigate your cursor. Quit with Q.

As you can see in the above screenshot, lynx will analyse and group its results by day, month and pages as well as other files. Feel free to explore!

[^1]: Default location /var/log/nginx/access.log

Option 2 - GoAccess Dashboard

GoAccess

zcat -f /var/log/nginx/access.log.*.gz |
goaccess /var/log/nginx/access.log -
-o /var/www/goaccess/report.html
--log-format=COMBINED
--html-report-title=my-dashboard

Thanks for reading!

luca

CC BY-NC 4.0  ©   Luca NerlichImprint + Privacy