Awstats - previous months :(

[bsandor]

I have a problem with awstats. The problem is well known!!! I already read a lot about it! I checked all the threads in this forum! I already read the KB articles about it... for example: http://kb.odin.com/en/5550

I sent letters to the operators at Parallels and I also checked other sites on the web. There are scripts in perl I can download......

I also made a little investigation because I felt that I am on my own AGAIN!!!

I tried the scripts. I checked the configuration of my server. I checked the version number of awstats.

I can make the previous months stats with a script written by smart guys or by my hand but it does nothing, because next time the statistics process runs it deletes all the directories and files in the statistics/webstat folder.

I give a try. I copied the current folder (2009-07) and rename it to 2009-06. I also add the lines to the nav.html to get the new month in the combobox on the top of the awstats page. It is cool. Working fine. Then I run the daily process for one domain:

/opt/psa/admin/sbin/statistics --calculate-one --domain-name=DOMAIN_NAME

and HOPP it deletes the new folder I made by my hand and it creates a new nav.html with only the current month.

It means:

- awstats did the task I got the result from the log files
- but the /opt/psa/admin/sbin/statistics command deleted everything and created the current month

I do not really understand this. If the daily process will delete everything in that folder than what should I do? Run the process after the daily stat proc to reproduce the previous month stats every day?

It is a joke!

I know that there are a lot of smart guys here. Let's try to figure out something! I think the problem is with the /opt/psa/admin/sbin/statistics command because it deletes the already made folders and stats.

I will continue digging. If I will find something I will post it here.

Please post your opinion about it! Thank you in advanced.

Plesk 9.2.1 Debian 4.0 Awstats 6.5

 

[Flagg]

bsandor,

Just to make sure, you did set up Settings -> Settings of Server Statistics -> Retain Web and traffic statistics to be more than a month?

 

[bsandor]

Dear Flagg,

yes I set it to 12 month. I set it a few month ago.

 

[Flagg]

bsandor,

Are these settings checked in Domains -> <domain.tld> -> Statistics -> Summary Report -> View Traffic History on affected domains? Is the stat_ttl field (default retain time) set appropriately in the Plesk database? Following article related:

http://kb.odin.com/en/6246

 

[bsandor]

YOOO!

This was unset! I hope this was the reason! Thank you very very much!

On the other hand. I wrote 3 letters to the support. I gave my IP and let them in! None of the checked this. None of them told me anything about it!

Thank you for your help again! You solved my biggest problem...

 

[Flagg]

bsandor,

Although this article relates to Plesk 8.x, this also works on Plesk 9 (tested this, to be sure). It would be great, if you could post feedback on if this works in your case.

 

[bsandor]

Dear Flagg,

I will let you know the result on the 1st of aug. If I want to generate the previous month I sholuld use this KB article:
http://kb.odin.com/en/5550

I tried to bulid the missing months. It made the dirs (example: 2009-01). It creted the html files but when I checked with the browser there were 0-s everywhere. No data.

I have the log-s for the previous months.

 

[Flagg]

bsandor,

Just to check, you did run statistics for the affected domains, as in step 6 of the article you quote? The situation you have now looks very much like the pages were created, as per step 5, but the actual data was not gathered by the statistics utility.

 

[bsandor]

Dear Flagg,

yes. I made a backup before I tried to reproduce the stats.

Then I tried both ways. The first when I use the txt-s in the webstat directory then when I remove it.

At the end I ran the statistics for the affected domain. I got the pages but there were only zeros.

But the last step only responsible for the nav.html is not it?

 

[bsandor]

I checked the script again. I deleted the existing 2009-07 dir and the files for the report. I only kept the awstatsMMYYYY*.txt files in the directory. Then I run the script again.

The result is. I had 2008-08 .... 2009-07 folders. All of them with the needed html files. But only the last one 2009-07 has data. But that data contains strange numbers. For example 20 visitors. It seems that the script uses the apache log-s to generate the riports not the TXT files.

How can I tell the script to use the TXT files?

 

[bsandor]

Dear Flagg,

I also checked the following. I deleted all the awstats database files (awstatsMMYYYY*.txt) from the folder. And I run again the script. It did the same. I got a report with 0-s without any error messages. It seems that the script do not use the database files!!! But without useing it it will not be able to reproduce the stats becuse I do not have the apache log files...

 

[bsandor]

Dear Flagg,

I have the result. The settings are ok. Now I have the stats for 2009.07 and today a new folder was created for 2009.08. So I the half of the problem is solved.

Now I have the previous month but I still have to create the stats before 2009.07. As I wrote it before it seems that the procedure in this article http://kb.odin.com/en/5550 do not use the awstatsMMYYYY*.txt files I have.

Any suggestions appreciated! I am still looking for the solutions if I found something I will post it!

 

[toolate]

bsandor,

Following is a script that I adapted from KB article 5550 to build previous months stats from the existing .txt files. You could adjust the 'for file in' line to process only the files that you need.

Please note that on my system (8.6.0 on CentOS 5.2) running the 'statstics' command always deletes stats for the month 4 months prior to the current month. Runnings stats on my system today would delete the stats for 2009-04 but leave any stats before that date untouched. This I have not been able to track down and find a resolution for. It may be specific to my installation. I have not seen any posts about that type of problem.

Hope this code helps. As always backup and test before using in production environment.

#!/bin/bash

# Script to process all existing awstats files for a give domain. 
# Adapted from Plesk KB article: http://kb.odin.com/en/5550

vhost_name=${1}

# Get paths and such
AWSTATS_BIN_D=`grep ^AWSTATS_BIN_D /etc/psa/psa.conf | awk '{print $2}'`
HTTPD_VHOSTS_D=`grep ^HTTPD_VHOSTS_D /etc/psa/psa.conf | awk '{print $2}'`
PRODUCT_ROOT_D=`grep ^PRODUCT_ROOT_D /etc/psa/psa.conf | awk '{print $2}'`
awstats=${AWSTATS_BIN_D}/awstats.pl
awstats_gen_opts="-staticlinks -configdir=${PRODUCT_ROOT_D}/etc/awstats -config=${vhost_name}-http"

echo "Rebuilding Stats for:" ${vhost_name}

cd $HTTPD_VHOSTS_D/$vhost_name/statistics/webstat
for file in awstats*.txt
do
   m=${file:7:2}
   y=${file:9:4}
   echo Processing: $m - $y
   mkdir ${HTTPD_VHOSTS_D}/${vhost_name}/statistics/webstat/$y-$m;

   dest_dir=$HTTPD_VHOSTS_D/$vhost_name/statistics/webstat/$y-$m ;
   $awstats $awstats_gen_opts -month=$m -year=$y -output > $dest_dir/awstats.${vhost_name}-http.html ;
   ln -s ${dest_dir}/awstats.${vhost_name}-http.html $dest_dir/index.html ;
   for output in alldomains allhosts lasthosts unknownip allrobots lastrobots session urldetail urlentry urlexit osdetail unknownos refererse refererpages keyphrases keywords errors404 ; do
     $awstats $awstats_gen_opts -month=$m -year=$y -output=$output > $dest_dir/awstats.${vhost_name}-http.$output.html ;
   done ;
done

# Step 6
echo "Running statistics to rebuild navigation menu..."
$PRODUCT_ROOT_D/admin/sbin/statistics --calculate-one --domain-name=$vhost_name

 

[HostaHost]

I checked that the stat_ttl value is correct on several of our servers, some 8.6 up from older versions, some 8.6 native install, some 9.3 native install, and none of the servers preserve previous months in awstats. Any other ideas?

 

[Jllynch]

Thanks so much toolate that script worked great!