1. Please take a little time for this simple survey! Thank you for participating!
    Dismiss Notice
  2. Dear Pleskians, please read this carefully! New attachments and other rules Thank you!
    Dismiss Notice
  3. Dear Pleskians, I really hope that you will share your opinion in this Special topic for chatter about Plesk in the Clouds. Thank you!
    Dismiss Notice

Solution for last months Awstats

Discussion in 'Plesk for Linux - 8.x and Older' started by balu, Apr 17, 2007.

  1. balu

    balu Guest

    0
     
    So, is this the solution you all are looking for?

    (Hm, looks like I can not attach a file here? So here is the complete script:)

    Code:
    #!/bin/sh
    ## The following script is meant to allow the viewing of last months awstats
    ## information on Plesk servers. It creates the statistics for all found 
    ## awstats monthly data and a frameset to allow to choose a month from the
    ## generated ones.
    ## 
    ## The main idea behind this file is to avoid any changes to the installed Plesk.
    ## So I'm misusing e.g. the at_domains_index.html which is first in Apaches 
    ## DirectoryIndex configuration on Plesk servers to display the frameset, e.g.
    ## 
    ## It was written on a Debian 3.1 (Sarge) and will probably not work on other
    ## distributions because of different paths, scripts, etc.
    ##
    ## BEWARE: This works on my box, but it might crash your system, delete all 
    ##         existing files on it and make you loose your job. So
    ##         USE AT YOUR OWN RISK. I'M NOT RESPONSIBLE FOR ANY DAMAGE THIS
    ##         SCRIPT DOES ON YOUR SERVERS. USE IT WITH CARE AND MAKE SURE YOU
    ##         UNDERSTAND WHAT YOU ARE DOING.
    ##         
    ##         The script is fairly new and should be called "alpha" in this state,
    ##         but it seems to works fine here.
    ##
    ## Notes:
    ##      - The script only loops through all virtual hosts in /var/www/vhosts/
    ##        and does not check wether awstats is enabled for the host in Plesk 
    ##        (it is for all domains on the machine this script was written for)
    ##      - It does not update the awstats data from logfiles, but uses the 
    ##        ones that were created by awstats in the daily runs
    ##      - it is probably best used by putting it into /etc/cron.monthly/
    ##        (don't forget to make it executable ;)
    ##      - it is to be run as root, so have a look at the BEWARE above
    ##
    ## 2007-04-17 - initial release 
    ##              by Thomas "Balu" Walter <walter@bartels-schoene.de>
    
    # set some variables
    buildStaticPages="/usr/share/doc/awstats/examples/awstats_buildstaticpages.pl"
    webstatDir='statistics/webstat'
    
    # if you want to translate the title of the frameset or the text in front 
    # of the select box, update the following variables which are only used for the
    # first creation of the files
    tlStatsFor="Statistics for"
    tlShowStatsFor="Show statistics for"
    tlThisMonth="this month"
    
    # awstats on debian needs this to override configdir
    export AWSTATS_ENABLE_CONFIG_DIR=1
    
    # to avoid generating of this months stats which is done by cron.daily
    thisMonth=`date +%m`
    thisYear=`date +%Y`
    
    # loop through all virtual hosts
    # TODO it would probably be better to fetch the awstats enabled hosts directly
    #      from the database, but what the heck
    for vhostDir in /var/www/vhosts/*
    do
        # get only the virtual hosts name
        vhostBaseName=`basename $vhostDir`
    
        # don't create statistics for those
        if [[ "$vhostBaseName" == "default" ]]; then 
            continue 
        fi
        if [[ "$vhostBaseName" == "chroot" ]]; then
            continue
        fi
    
        # create the frameset file 
        # I'm misusing at_domains_index.html here which is meant to be used for 
        # something else, but is first in Apaches DirectoryIndex setting in plesk
        framesFile="$vhostDir/$webstatDir/at_domains_index.html"
        echo "<html><head><title>$tlStatsFor $vhostBaseName</title>" > $framesFile
        echo '</head><frameset rows="38,*">' >> $framesFile
        echo '<frame src="monthchooser.html" name="monthchooser" scrolling="no" noresize="noresize" />' >> $framesFile
        echo '<frame src="index.html" name="statistics" />' >> $framesFile
        echo "</frameset></html>" >> $framesFile
    
        # create the month chooser frame 
        frameFile="$vhostDir/$webstatDir/monthchooser.html"
        echo '<html><head><title>Statistics</title></head><body>' > $frameFile
        echo '<body style="font: 11px verdana, arial, helvetica, sans-serif; background: #ececec;">' >> $frameFile
        echo '<form style="margin: 0; padding: 0;" action="#">' >> $frameFile
        echo $tlShowStatsFor >> $frameFile
        echo ' <select onChange="top.statistics.location.href = this.value;">' >> $frameFile
        echo '<option value="index.html">'"$tlThisMonth"'</option>' >> $frameFile
        echo '<!-- add next month here -->' >> $frameFile
        echo '</select></form></body></html>' >> $frameFile
    
        ## loop through all last months statistics stored by awstats
        for monthFile in $vhostDir/$webstatDir/awstats*.$vhostBaseName-http.txt
        do
            monthFileBaseName=`basename $monthFile`
    
            # figure out year and month from the filename (might be solved more elegant...)
            month=`echo $monthFileBaseName | sed "s%awstats\(..\)\(....\).$vhostBaseName-http.txt%\1%"`
            year=`echo $monthFileBaseName | sed "s%awstats\(..\)\(....\).$vhostBaseName-http.txt%\2%"`
    
            if [[ $month == $thisMonth && $year == $thisYear ]]; then
                continue
            fi
    
            # check if the stats for the month were already generated
            if [[ ! -e "$vhostDir/$webstatDir/$year-$month" ]]; then
    
               # if not create the stats directory
               mkdir "$vhostDir/$webstatDir/$year-$month"
    
    	   # uncomment this if you want to see anything
               # echo "Building for $vhostBaseName $year-$month"
    
                # and build the statistics for the month
                $buildStaticPages \
                    -config=$vhostBaseName-http \
                    -staticlinks \
                    -configdir=/opt/psa/etc/awstats \
                    -month=$month \
                    -year=$year \
                    -awstatsprog=/usr/lib/cgi-bin/awstats.pl \
                    -dir=$vhostDir/$webstatDir/$year-$month >/dev/null 2>&1
    
                # link index.html to the newly generated stats main page
                ln -sf "$vhostDir/$webstatDir/$year-$month/awstats.$vhostBaseName-http.html" \
                       "$vhostDir/$webstatDir/$year-$month/index.html"
            fi
    
            # add the statistics to the monthchooser
            sed 's%<!-- add next month here -->%<!-- add next month here -->'"\n"'<option value="'$year'-'$month'/index.html">'$year'-'$month'</option>%' \
                "$vhostDir/$webstatDir/monthchooser.html" > "$vhostDir/$webstatDir/monthchooser.html.tmp"
            mv "$vhostDir/$webstatDir/monthchooser.html.tmp" "$vhostDir/$webstatDir/monthchooser.html"
        done
    done
    Balu
     
  2. eugenevdm

    eugenevdm Silver Pleskian

    30
    68%
    Joined:
    Nov 11, 2003
    Messages:
    611
    Likes Received:
    0
    Wow fantastic! I haven't tried it yet but it looks promising.

    I would be nice if one day you could update the script to also check if AWStats is enabled on the domain.
     
  3. toneworks

    toneworks Guest

    0
     
    changes for RHEL/CentOS 4 + some fixes

    Hello! Thank you for the great script.
    I changed some things to make it work on our Plesk 8.1.1 on RHEL/CentOS 4.
    Maybe someone can diff the 2 versions, and create one version with configurations for different OSses. I don't have time for that today.

    EDIT: I updated the code, now you can configure it to use full urls in the month dropdown. I needed it because we display the statistics within an other frameset.

    Code:
    #!/bin/sh
    ## The following script is meant to allow the viewing of last months awstats
    ## information on Plesk servers. It creates the statistics for all found 
    ## awstats monthly data and a frameset to allow to choose a month from the
    ## generated ones.
    ## 
    ## The main idea behind this file is to avoid any changes to the installed Plesk.
    ## So I'm misusing e.g. the at_domains_index.html which is first in Apaches 
    ## DirectoryIndex configuration on Plesk servers to display the frameset, e.g.
    ## 
    ## It was written on a Debian 3.1 (Sarge) and will probably not work on other
    ## distributions because of different paths, scripts, etc.
    ##
    ## BEWARE: This works on my box, but it might crash your system, delete all 
    ##         existing files on it and make you loose your job. So
    ##         USE AT YOUR OWN RISK. I'M NOT RESPONSIBLE FOR ANY DAMAGE THIS
    ##         SCRIPT DOES ON YOUR SERVERS. USE IT WITH CARE AND MAKE SURE YOU
    ##         UNDERSTAND WHAT YOU ARE DOING.
    ##         
    ##         The script is fairly new and should be called "alpha" in this state,
    ##         but it seems to works fine here.
    ##
    ## Notes:
    ##      - The script only loops through all virtual hosts in /var/www/vhosts/
    ##        and does not check wether awstats is enabled for the host in Plesk 
    ##        (it is for all domains on the machine this script was written for)
    ##      - It does not update the awstats data from logfiles, but uses the 
    ##        ones that were created by awstats in the daily runs
    ##      - it is probably best used by putting it into /etc/cron.monthly/
    ##        (don't forget to make it executable ;)
    ##      - it is to be run as root, so have a look at the BEWARE above
    ##
    ## 2007-04-17 - initial release 
    ##              by Thomas "Balu" Walter <walter@bartels-schoene.de>
    ##
    ## 2007-04-19 - customized for Plesk 8.1 on RHEL/CentOS 4 and some fixes
    ##              by Toneworks Studios <jeroen@toneworks.nl>
    
    # set some variables
    buildStaticPages="/usr/bin/awstats_buildstaticpages.pl"
    webstatDir='statistics/webstat'
    webstatRoot='plesk-stat/webstat'
    awstats_configdir='/usr/local/psa/etc/awstats'
    awstats_prog='/var/www/awstats/awstats.pl'
    vhostsdir='/home/httpd/vhosts' # no tailing slash
    forceupdate=0
    useFullUrls=0
    
    # if you want to translate the title of the frameset or the text in front 
    # of the select box, update the following variables which are only used for the
    # first creation of the files
    tlStatsFor="Statistics for"
    tlShowStatsFor="Show statistics for:"
    tlThisMonth="this month"
    
    # awstats on debian needs this to override configdir
    export AWSTATS_ENABLE_CONFIG_DIR=1
    
    # to avoid generating of this months stats which is done by cron.daily
    thisMonth=`date +%m`
    thisYear=`date +%Y`
    
    # loop through all virtual hosts
    # TODO it would probably be better to fetch the awstats enabled hosts directly
    #      from the database, but what the heck
    for vhostDir in `find $vhostsdir -type d -maxdepth 1`
    do
        # get only the virtual hosts name
        vhostBaseName=`basename $vhostDir`
        #echo "#### domain \"$vhostBaseName\""
    
        # don't create statistics for those
        if [[ "$vhostBaseName" == "default" ]]; then 
            continue 
        fi
        if [[ "$vhostBaseName" == "chroot" ]]; then
            continue
        fi
        if [[ "$vhostBaseName" == "" ]]; then
            continue
        fi
                    
        # skip if AWStats is not enabled for this domain
        if [ ! -f "$vhostDir/$webstatDir/awstats.$vhostBaseName-http.html" ]; then
            continue
        fi
    
        fullWebstatRoot=""
        if [ 1 == $useFullUrls ]; then
          fullWebstatRoot="http://$vhostBaseName/$webstatRoot/";
        fi
    
    
        # create the frameset file 
        # I'm misusing at_domains_index.html here which is meant to be used for 
        # something else, but is first in Apaches DirectoryIndex setting in plesk
        framesFile="$vhostDir/$webstatDir/at_domains_index.html"
        echo "<html><head><title>$tlStatsFor $vhostBaseName</title>" > $framesFile
        echo '</head><frameset rows="27,*" border="0" frameborder="0" framespacing="0">' >> $framesFile
        echo '<frame src="monthchooser.html" name="monthchooser" scrolling="no" noresize="noresize" />' >> $framesFile
        echo '<frame src="index.html" name="statistics" />' >> $framesFile
        echo "</frameset></html>" >> $framesFile
    
        # create the month chooser frame 
        frameFile="$vhostDir/$webstatDir/monthchooser.html"
        echo '<html><head><title>Statistics</title></head><body>' > $frameFile
        echo '<body style="font: 12px verdana, arial, helvetica, sans-serif; background: #ececec;" leftmargin="2" topmargin="2" marginwidth="2" marginheight="2">' >> $frameFile
        echo '<form style="margin: 0; padding: 0;" action="#">' >> $frameFile
        echo '<table border=0 cellspacing=0 cellpadding=0><tr><td valign=middle style="font: 12px verdana, arial, helvetica, sans-serif;">' >> $frameFile
        echo $tlShowStatsFor >> $frameFile
        echo '&nbsp;</font></td><td valign=middle>' >> $frameFile
        echo ' <select onChange="parent.statistics.location.href = this.value;">' >> $frameFile
        echo '<option value="'$fullWebstatRoot'index.html">'"$tlThisMonth"'</option>' >> $frameFile
        echo '<!-- add next month here -->' >> $frameFile
        echo '</select>' >> $frameFile
        echo '</td></tr></table>' >> $frameFile
        echo '</form></body></html>' >> $frameFile
    
        ## loop through all last months statistics stored by awstats
        for monthFile in `ls $vhostDir/$webstatDir/awstats*.$vhostBaseName-http.txt 2>/dev/null`
        do
            monthFileBaseName=`basename $monthFile`
    
            # figure out year and month from the filename (might be solved more elegant...)
            month=`echo $monthFileBaseName | sed "s%awstats\(..\)\(....\).$vhostBaseName-http.txt%\1%"`
            year=`echo $monthFileBaseName | sed "s%awstats\(..\)\(....\).$vhostBaseName-http.txt%\2%"`
    
            if [[ $month == $thisMonth && $year == $thisYear ]]; then
                continue
            fi
    
            # check if the stats for the month were already generated
            if [[ 1 == $forceupdate || ! -e "$vhostDir/$webstatDir/$year-$month" ]]; then
    
               # if not create the stats directory
               if [ ! -d "$vhostDir/$webstatDir/$year-$month" ]; then
                 mkdir "$vhostDir/$webstatDir/$year-$month"
               fi
    
     	         # uncomment this if you want to see anything
               #echo "Building for $vhostBaseName $year-$month"
    
                # and build the statistics for the month
                $buildStaticPages \
                    -config=$vhostBaseName-http \
                    -staticlinks \
                    -configdir=$awstats_configdir \
                    -month=$month \
                    -year=$year \
                    -awstatsprog=$awstats_prog \
                    -dir=$vhostDir/$webstatDir/$year-$month >/dev/null 2>&1
    
                # link index.html to the newly generated stats main page
                ln -sf "$vhostDir/$webstatDir/$year-$month/awstats.$vhostBaseName-http.html" \
                       "$vhostDir/$webstatDir/$year-$month/index.html"
            fi
    
            # add the statistics to the monthchooser
            sed 's%<!-- add next month here -->%<!-- add next month here -->'"\n"'<option value="'$fullWebstatRoot$year'-'$month'/index.html">'$year'-'$month'</option>%' \
                "$vhostDir/$webstatDir/monthchooser.html" > "$vhostDir/$webstatDir/monthchooser.html.tmp"
            mv "$vhostDir/$webstatDir/monthchooser.html.tmp" "$vhostDir/$webstatDir/monthchooser.html"
        done
    done
    
    Kind regards,
    Jeroen Vermeulen
     
  4. balu

    balu Guest

    0
     
    I'm not sure if that is still needed. The note was added before I changed the script to loop through the statistic summaries that awstats generates ($vhostDir/$webstatDir/awstats*.$vhostBaseName-http.txt).

    So if there are no awstats*-http.txt files found it won't run to create the awstats html pages.

    toneworks, I will have a look at your changes. Is there some kind of file like /etc/debian_version on RHEL/CentOS 4 that might help automagically figuring out the distro?

    Balu
     
  5. toneworks

    toneworks Guest

    0
     
    Balu:
    [root@server ~]# cat /etc/redhat-release
    CentOS release 4.4 (Final)
     
  6. zonathen12

    zonathen12 Guest

    0
     
    Worked like a charm

    Thanks I had to change some paths but worked like a charm. Thank god for helpful people such as yourself!
     
  7. lgeoffrey

    lgeoffrey Guest

    0
     
    Hello,

    I'm running debian 3.1 plesk 8.1

    I ran the script and got this in the drop menu of each domain

    -this month
    -awstats*.domain.com-http.txt-awstats*.domain.com-http.txt-awstats*.domain.com-http.txt-awstats*.domain.com-http.txt

    Anybody had the same problem ?

    Thank you
    Geoffrey.
     
  8. captainpovey

    captainpovey New Pleskian

    22
    23%
    Joined:
    Jun 7, 2007
    Messages:
    5
    Likes Received:
    0
    Another thumbs up from me!

    Brilliant stuff!
    I was thinking I'd have to install a copy of awstats for each domain ... but this has done the job.
    Many, many thanks for saving me a lot of time.

    For what its worth, here are my path settings on a UK 1&1 Business Server (FC4)

    Code:
    buildStaticPages="/usr/share/awstats/awstats_buildstaticpages.pl"
    webstatDir='statistics/webstat'
    webstatRoot='plesk-stat/webstat'
    awstats_configdir='/usr/local/psa/etc/awstats'
    awstats_prog='/var/www/cgi-bin/awstats/awstats.pl'
    vhostsdir='/var/www/vhosts' # no tailing slash
    
     
  9. futureweb

    futureweb Regular Pleskian

    28
    23%
    Joined:
    Feb 4, 2007
    Messages:
    186
    Likes Received:
    4
    Location:
    Tirol, Austria
    hi,

    thank you very much for the script, it works wonderful!
    A shame that SWSoft supplies AWStats in a way that cannot be used for customers at all.

    Don't forget, if you have SSL Domains, that you have to run those logs separat with the webstat-ssl directory, and changing the loop that checks if awstats is enabled (or file exist) from http to https.

    Thanks
    Patrick
     
  10. nickthephreak

    nickthephreak Guest

    0
     
    Hi,

    First of all also a thank you very much from me! this script helped me a lot!!!!

    the version from balu i could not get running on my suse servers, but the version of toneworks worked like a charm with some changed paths.

    thought i share it with you:

    for my suse 9.3 server i used this paths:

    Code:
    buildStaticPages="/usr/share/doc/packages/awstats/tools/awstats_buildstaticpages.pl"
    webstatDir='statistics/webstat'
    webstatRoot='plesk-stat/webstat'
    awstats_configdir='/usr/local/psa/etc/awstats'
    awstats_prog='/srv/www/cgi-bin/awstats.pl'
    vhostsdir='/var/www/vhosts' # no tailing slash
    forceupdate=0
    useFullUrls=0
    and for my suse 10.0 i used this paths:
    Code:
    buildStaticPages="/usr/share/doc/packages/awstats/tools/awstats_buildstaticpage$
    webstatDir='statistics/webstat'
    webstatRoot='plesk-stat/webstat'
    awstats_configdir='/usr/local/psa/etc/awstats'  
    awstats_prog='/srv/www/cgi-bin/awstats.pl'
    vhostsdir='/srv/www/vhosts' # no tailing slash
    forceupdate=0
    useFullUrls=0

    Greets
    NICK
     
  11. rethus

    rethus Guest

    0
     
    i got the following error after strt the script:



    Anny idea whats wrong?
    Code:
    find: warning: you have specified the -maxdepth option after a non-option argument -type, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it).  Please specify options before other arguments. 
     
  12. nickthephreak

    nickthephreak Guest

    0
     
    i had the same error message. though the script finished despite of that and the stats were showing as expected, i didn't bother.

    anyway i would also be interested in an explanation of the error message.

    greets

    nick
     
  13. rethus

    rethus Guest

    0
     
    I've found a solution.

    only change the Line

    Code:
    for vhostDir in `find $vhostsdir -type d -maxdepth 1`
    with:
    Code:
    for vhostDir in `find $vhostsdir -maxdepth 1 -type d`
    Now the Script retun no error, but it dosnt work.

    if i look to my statistic-Page i see no Pulldownmenue
     
  14. Spazholio@

    Spazholio@ Guest

    0
     
    Here's an odd one. I get this when I run the script, amongst all the text that flies by:

    Main HTML page is 'awstats.<domain>.com-http.html'.
    ./awstats.sh: line 158: -dir=/var/www/vhosts/<domain>.com/statistics/webstat/2007-07: No such file or directory

    That directory exists. And I'm running the script from within /root/, so, for some odd reason, all the files are written there! I've got hundreds of awstats files in there. But in the dir that's listed above, I see:

    lrwxrwxrwx 1 root root 90 Aug 3 09:25 index.html -> /var/www/vhosts/<domain>.com/statistics/webstat/2007-07/awstats.<domain>.com-http.html

    Any ideas?
     
  15. udoersam

    udoersam New Pleskian

    24
     
    Joined:
    Apr 9, 2007
    Messages:
    14
    Likes Received:
    0
    Hello.

    I had the situation on my 1&1 root server based on Suse Linux 10.1.

    The program worked fine, but nothing happened to the subdirectories.

    My solution:
    I enabled the option forceupdate=1
    awstats_configdir='srv/www/cgi-bin/awstats.pl'

    And then I run the program again and now it works fine.

    After the 1st run you can disable the forceupdate-option.

    Thank you Balu for this great soultion


    udoersam
     
  16. davidnewcomb

    davidnewcomb Guest

    0
     
    Awstats Monthly History Software and instructions

    The trouble with the solution above is that it does not tackle the root of the problem, it is trying to re-create the statistics pages from scratch.

    The solution below fixes the root of the problem (statistics files being in the wrong place), works with domains that have awstats enabled, works with customers who are enabled and most importantly does not impact on the system's normal method of operation.

    I've been running it for a couple of months and it does everything it says on the tin:

    http://www.bigsoft.co.uk/blog/index.php/2008/07/08/plesk-awstat-forgets-the-monthly-statist
     
Loading...