• If you are still using CentOS 7.9, it's time to convert to Alma 8 with the free centos2alma tool by Plesk or Plesk Migrator. Please let us know your experiences or concerns in this thread:
    CentOS2Alma discussion

Awstats and Plesk again

Code:
--- maillogconvert.pl	2004-09-14 04:48:07.000000000 +0930
+++ /usr/local/awstats/tools/maillogconvert.pl	2005-10-20 12:38:04.205717080 +0930
@@ -27,7 +27,7 @@
 $Debug
 %mail %qmaildelivery
 $help
-$mode $year $lastmon $Debug
+$mode $year $lastmon $Debug $filterdomain
 $NBOFENTRYFOFLUSH
 $MailType
 %MonthNum
@@ -178,6 +178,7 @@
 	if ($param{$key} =~ /(^|-|&)debug=([^&]+)/i) { $Debug=$2; shift; next; }
 	if ($param{$key} =~ /^(\d+)$/) { $year=$1; shift; next; }
 	if ($param{$key} =~ /^(standard|vadmin)$/i) { $mode=$1; shift; next; }
+	if ($param{$key} =~ /^(.*\..*)$/i) { $filterdomain=$1; shift; next; }
 }
 if ($mode ne 'standard' and $mode ne 'vadmin') { $help = 1; }
 
@@ -325,10 +326,10 @@
 		# sendmail: Sep 30 04:21:32 halley sendmail[3161]: g8U2LVi03161: ruleset=check_rcpt, arg1=<[email protected]>, relay=moon.partenor.fr [10.0.0.254], reject=550 5.7.1 <[email protected]>... Relaying denied
 
 		# sendmail:	Jan 10 07:37:48 smtp sendmail[32440]: ruleset=check_relay, arg1=[211.228.26.114], arg2=211.228.26.114, relay=[211.228.26.114], reject=554 5.7.1 Rejected 211.228.26.114 found in dnsbl.sorbs.net
-		# sendmail: Jan 10 07:37:08 smtp sendmail[32439]: ruleset=check_relay, arg1=235.Red-213-97-175.pooles.rima-tde.net, arg2=213.97.175.235, relay=235.Red-213-97-175.pooles.rima-tde.net [213.97.175.235], reject=550 5.7.1 Mail from 213.97.175.235 refused. Rejected for bad WHOIS info on IP of your SMTP server - see http://www.rfc-ignorant.org/
+		# sendmail: Jan 10 07:37:08 smtp sendmail[32439]: ruleset=check_relay, arg1=235.Red-213-97-175.pooles.rima-tde.net, arg2=213.97.175.235, relay=235.Red-213-97-175.pooles.rima-tde.net [213.97.175.235], reject=550 5.7.1 Mail from 213.97.175.235 refused. Rejected for bad WHOIS info on IP of your SMTP server - see http://www.rfc-ignorant.org/
 		# sendmail: Jan 10 17:15:42 smtp sendmail[12770]: ruleset=check_relay, arg1=[63.218.84.21], arg2=63.218.84.21, relay=[63.218.84.21], reject=553 5.3.0 Rejected - see http://spamhaus.org/
 		my ($mon,$day,$time,$id,$ruleset,$arg,$relay_s,$code)=m/(\w+)\s+(\d+)\s+(\d+:\d+:\d+)\s+[\w\-\.\@]+\s+(?:sendmail|sm-mta)\[\d+\][:\s]*(.*?):\sruleset=(\w+),\s+arg1=(.*),\s+relay=(.*),\s+(reject=.*)/;
-		# sendmail: Jan 10 18:00:34 smtp sendmail[5759]: i04Axx2c005759: Milter: data, reject=511 Virus found in email!
+		# sendmail: Jan 10 18:00:34 smtp sendmail[5759]: i04Axx2c005759: Milter: data, reject=511 Virus found in email!
 		if (! $mon) { ($mon,$day,$time,$id,$ruleset,$code)=m/(\w+)\s+(\d+)\s+(\d+:\d+:\d+)\s+[\w\-\.\@]+\s+(?:sendmail|sm-mta)\[\d+\]:\s+(.*?):\s\w+:\s(\w+),\s+(reject=.*)/; }
 		$mailid=(! $id && $mon?'999':$id);	# id not provided in log, we take '999'
 		if ($mailid) {
@@ -630,7 +631,9 @@
 
 		# If we can
 		if ($canoutput) {
-			&OutputRecord($mail{$mailid}{'year'}?$mail{$mailid}{'year'}:$year,$mail{$mailid}{'mon'},$mail{$mailid}{'day'},$mail{$mailid}{'time'},$mail{$mailid}{'from'},$to,$mail{$mailid}{'relay_s'},$mail{$mailid}{'relay_r'},$code,$mail{$mailid}{'size'},$mail{$mailid}{'forwardto'},$mail{$mailid}{'extinfo'});
+			if ($mail{$mailid}{'from'} =~ /$filterdomain$/ || $to =~ /$filterdomain$/) {
+				&OutputRecord($mail{$mailid}{'year'}?$mail{$mailid}{'year'}:$year,$mail{$mailid}{'mon'},$mail{$mailid}{'day'},$mail{$mailid}{'time'},$mail{$mailid}{'from'},$to,$mail{$mailid}{'relay_s'},$mail{$mailid}{'relay_r'},$code,$mail{$mailid}{'size'},$mail{$mailid}{'forwardto'},$mail{$mailid}{'extinfo'});
+			}
 			# Delete mail with generic unknown id (This id can by used by another mail)
 			if ($mailid eq '999') {
 				debug(" Delete mail for id=$mailid",3);

Code:
--- awstats.pl	2005-02-25 07:32:23.000000000 +1030
+++ /usr/local/awstats/wwwroot/cgi-bin/awstats.pl	2005-10-01 15:14:43.000000000 +0930
@@ -5632,6 +5632,7 @@
 	exit 2;
 }
 $SiteConfig||=&Sanitize($ENV{'SERVER_NAME'});
+
 #$ENV{'SERVER_NAME'}||=$SiteConfig;	# For thoose who use __SERVER_NAME__ in conf file and use CLI.
 $ENV{'AWSTATS_CURRENT_CONFIG'}=$SiteConfig;
 
@@ -5730,6 +5731,12 @@
 	exit 0;	
 }
 
+my $Domain = $ENV{'SERVER_NAME'};
+$Domain =~ s/^([^\.]+)//g;
+if (! ($SiteConfig =~ /^([^\.]+).$Domain/i)) {
+	error("Server '".$ENV{"SERVER_NAME"}."' is not allowed to access statistics of this domain/config.");
+}
+
 # Security check
 if ($AllowAccessFromWebToAuthenticatedUsersOnly && $ENV{'GATEWAY_INTERFACE'}) {
 	if ($Debug) { debug("REMOTE_USER=".$ENV{"REMOTE_USER"}); }
@@ -6068,7 +6075,7 @@
                 $LogFormat = $fixField[0];
                 &DefinePerlParsingFormat();
             }
-        }
+        }
 
 		# Parse line record to get all required fields
 		if (! (@field=map(/$PerlParsingFormat/,$line))) {
 
Almost have AWstats working..sort of..

Ok ..so I did the easy to follow install procedure given by ivsdigitel.
Then I used the awesome setup_awstats.sh file by mian in this thread and I set it up under Event Manager on domain creation with
<new_domain_name> <new_ssl_support> parameter (note that mian's post says <new_domain> it should be <new_domain_name> ).

I created a new domain and here is what happens
It creates the directories

/home/httpd/vhosts/mydomain/statistics/awstats/

when I go to http://mydomain.com/awstats/ i get the the dreaded
FORBIDDEN msg...

Now..when I ran

/usr/local/awstats/tools/awstats_updateall.pl now

I get lots of errors..an excerpt below . . .

Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=http.mydomain -configdir="/etc/awstats"' to update config http.mydomain
Update for config "/etc/awstats/awstats.http.mydomain.conf"
With data in log file "/home/httpd/vhosts/mydomain/statistics/logs/access_log.processed"...
Error: Couldn't open server log file "/home/httpd/vhosts/mydomain/statistics/logs/access_log.processed" : No such file or directory
Setup ('/etc/awstats/awstats.http.mydomain.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

here's the thing.. I have no /logs folder under the statistics folders..only awstats..hence the error

my awstats.http.mydomain.conf file looks like this

------------------
Include "/etc/awstats/awstats.model.conf"

DirData="/home/httpd/vhosts/mydomain/statistics/awstats/"
LogFile="/home/httpd/vhosts/mydomain/statistics/logs/access_log.processed"
LogType=W
LogFormat=1
SiteDomain="mydomain"
HostAliases="mydomain mydomain"

----------------
Now obviously I can see the log file parameter shows my non-existent directory..but I don't know enough about AWstats to know if its trying to read or write this file

what did I forget to do ? or what should I edit or change ?

Thanks to any who can help in advance :)

Winter
 
have you run the Plesk statistics program (usually runs in crontab each night), it takes care of rotating the access_log to access_log.processed.. if you only just created the domain and statistics has not run yet this would be one cause of a non-existant file
 
Originally posted by mian
have you run the Plesk statistics program (usually runs in crontab each night), ...
Don't think thats it...the i created the domain over 30 hours ago...
so worst case scenario should have run 6 hours ago...

Shouldn't the "/home/httpd/vhosts/mydomain/statistics/logs/ "directory exist ? Was it supposed to be created during AWStats install... or is it creazted the first time the drob job is run? the /httpd/ dir didn't exist either (under home) until I ran setup_awstats.sh...

am I missing some symbolic link or something ? where are the "access_log.processed" supposed to be anyway...

thanks

winter
 
the /httpd/ dir didn't exist either (under home) until I ran setup_awstats.s
If you're server defaulted to putting domain docroots under something other than /home/httpd/vhosts/ then you should have probably modified the script to reflect that.

On some OS's with fresh Plesk installs, the default docroot is /var/www/vhosts , so if this is the case, then you should have gone through the script(s) to make sure that the paths were correct for your particular server.
 
Originally posted by ShadowMan
If you're server defaulted to putting domain docroots under something other than /home/httpd/vhosts/ then you should have probably modified the script to reflect that.

On some OS's with fresh Plesk installs, the default docroot is /var/www/vhosts , so if this is the case, then you should have gone through the script(s) to make sure that the paths were correct for your particular server.

Your right... I am running Vituozzo with Fedorca Core2 OS template.

I changed the setup_awstats.sh to reflect that

so every instance of /home/httpd/vhosts/ was changed to /var/www/vhosts/ in the file...

Still having some issues...
before the file under event manager creates the /home/httpd/vhosts/mydomain/statistics/awstats/ directory

but since i made the changes..now it does not create the /var/www/vhosts/mydomain/statistics/awstats/ directory

I know its executing because i can see the /etc/awstats/mydomain.conf files being created...

when I run setup_awstats.sh from the cmd line however...

/usr/local/psa/bin/awstats.sh mydomain 0

it does create the directory...why wouldn't it work from the event manager ???

it still doesn't work even after that...i run the

/usr/local/awstats/tools/awstats_updateall.pl now

and get no errors (after the stats program has run of course to create the access_log.processed files it looks for)

but when I try to go to

http://mydomain/awstats/

I still get forbidden msg/..

there are only text files in the /var/www/vhosts/mydomain/statistics/awstats/ folder no html files..

the files are named

awstats112005.ftp.mydomaint.txt
awstats112005.http.mydomaint.txt
awstats112005.https.mydomaint.txt

any ideas ?...

thanks

winter
 
there are only text files in the /var/www/vhosts/mydomain/statistics/awstats/ folder no html files..
Yes, that is what should be in this directory. Not html files.
http://mydomain/awstats/

I still get forbidden msg/..
You get the forbidden message since that is not the proper URL. Try browsing to
http://mydomain/plesk-stat

or the SSL side (https:).

Event Manager: Instead of basing it on 'Domain Creation', try 'Physical Hosting Created', until this point, no domain directory structure exists, so the other script will not be able to create additional dir's.
 
DUDE!!!! we have a saying where I come from..
"Your a bear!!!"

its means you rock!!! :)

I am getting closer..you were right about the event manager thing. When the domain is created no all the proper folders are now created along with it.

You were also right about the path thing..

I now see the menu frame on the top of the screen allowing for AWStats on one side and webalizer on the other...

however the main frame just under says

"The page can not be found" :(

clicking on the web button for example on the awstats side tried to go to the url..

'https://mydomain/awstatscgi/awstats.pl?config=http.mydomain'

but like i said "page not found"

clicking on anything on the awstats side gives the same error

the webalizer side works fine...

I am SOOOO close now :D I can feel it...:)

would it have something to do with permissions and the awstats.pl file?
My cgi-bin folder is empty...

any ideas ?

thanks

winter
 
The index.html file which is displayed is supposed to call up a frame showing 'menu.html' which should also be in the

/var/www/vhosts/domain.tld/statistics

directory. Both of these .html files are owned by root : root, and the perms are:

-rw-r--r--

I am wondering if there was still some leftover garbage from running the installer with the wrong paths... ?

The menu.html file (again) calls up a series of frames.

/awstatscgi is an alias for /usr/local/awstats/wwwroot/cgi-bin (at least on my OS), but check your /var/www/vhosts/domain.tld/conf/awstats.conf to see what it is aliased on yours.

#ll /usr/local/awstats/wwwroot/
drwxr-xr-x root root 4096 cgi-bin

#ll /usr/local/awstats/
drwxr-xr-x root root 4096 wwwroot

In each of the directories under the domain's ~/statistics directory, make sure there is an index.html file owned by root : psaserv, perms 755 (I don't know why it has to be 755 but they are). Dir names:

anon_ftpstat
ftpstat
webstat
webstat-ssl
(these are root : root and 755)
 
my cgi-bin

#ll /usr/local/awstats/wwwroot/

drwxr-xr-x 5 root root 1024 Nov 7 03:06 cgi-bin
-------
my wwwroot

#ll /usr/local/awstats/

drwxr-xr-x 7 root root 1024 Nov 8 01:08 wwwroot

well...here is what I have for the statistics dir...

drwxr-xr-x 2 root root 1024 Nov 8 04:41 anon_ftpstat
drwxr-xr-x 2 root root 1024 Nov 8 05:13 awstats
drwxr-xr-x 2 root root 1024 Nov 8 04:56 ftpstat
-rw-r--r-- 1 root root 521 Nov 8 04:41 index.html
drwxr-xr-x 2 root root 1024 Nov 8 05:23 logs
-rw-r--r-- 1 root root 1833 Nov 8 04:41 menu.html
drwxr-xr-x 2 root root 1024 Nov 8 04:56 webstat
drwxr-xr-x 2 root root 1024 Nov 8 05:12 webstat-ssl

in the begining of your post you mention that both index.html and menu.html are supposed to be root :root -rw-r--r-- which is 644

but at the end you say that menu.html is supposed to be root: psaserv -rwxr-xr-x which is 755

which is it ?

i tried both (lol) still no go.. :(

my /var/www/vhosts/mydomain/conf/awstats.conf file contains

Alias /awstatscss /usr/local/awstats/wwwroot/css/
Alias /awstatsicons /usr/local/awstats/wwwroot/icon/
Alias /awstatscgi /usr/local/awstats/wwwroot/cgi-bin/

<Directory /usr/local/awstats/wwwroot/cgi-bin/>
<IfModule mod_perl.c>
<Files ~ (\.pl$)>
SetHandler perl-script
PerlHandler ModPerl::Registry
Options ExecCGI
allow from all
PerlSendHeader On
</Files>
</IfModule>
</Directory>

looks in order no ?

just one last thing i noticed... in my /etc/awstats/ directory

i have the five .conf files created by me creating the domain

awstats.mail.mydomain.conf
awstats.https.mydomain.conf
awstats.http.mydomain.conf
awstats.ftp.mydomain.conf
awstats.anonftp.mydomain.conf

and i also have these other files . . .

awstats.mail..conf, awstats.https..conf, awstats.http..conf, awstats.ftp..conf, awstats.anonftp..conf

which i am assuming are default files created during install ... and then one more file

awstats.model.conf

now at the beginning of all my domain created .conf files is the line

Include "/etc/awstats/awstats.model.conf"


looking in this file (excerpt)

LogFile="/var/log/httpd/mylog.log"

LogType=W
LogFormat=1
LogSeparator=" "
SiteDomain=""
HostAliases="localhost 127.0.0.1 REGEX[myserver\.com$]"
DNSLookup=2
DirData="/var/lib/awstats"
DirCgi="/awstats"
DirIcons="/awstatsicons"
AllowToUpdateStatsFromBrowser=0
AllowFullYearView=2
....

this first line ..

LogFile="/var/log/httpd/mylog.log"

there is no such file in that dir..what should it be ? or am I looking in the wrong place...

btw..thanks for all your help so far :)

winter
 
you can remove awstats.mail..conf files, they must have been created when no domain was specified. awstats.model.conf is the only one needed, it's included by the rest so you can set common config values in it.

ignore the non existant logdir too as that will be overridden with the path to the log files in each config file

change DirCgi to /awstatscgi
 
ok.. I changed the DirCgi to /awstatscgi ..
still no go..
getting error 404 file not found...

i even restarted the server :(

I don't think it's seeing awstats.pl .. :(

but my alias' are correct
 
check the domains error log in the statistics dir, its the best source of information
 
Originally posted by Wintermute
I don't think it's seeing awstats.pl
Code:
{Tue Nov 08 03:13:15 Bash 2.05b.0 /usr/local/awstats/wwwroot/cgi-bin}
[root@ns2 cgi-bin]# ll
total 548
drwxr-xr-x    5 root     root         4096 Oct  9 20:26 .
drwxr-xr-x    7 root     root         4096 Oct  9 20:26 ..
-rwxr-xr-x    1 root     root         5407 Feb 25  2005 awredir.pl
-rwxr-xr-x    1 root     root       527395 Feb 25  2005 awstats.pl
drwxr-xr-x    5 root     root         4096 Oct  9 20:26 lang
drwxr-xr-x    2 root     root         4096 Oct  9 20:26 lib
drwxr-xr-x    3 root     root         4096 Oct  9 20:26 plugins
make sure the awstats.pl has the executable bits set.
 
ok...3 days on this..ouch...

here is what I found out...

I see the menu.html on the top and click one of the link on the awstats side..say the web one...

the url it goes to

"http://mydomain/awstatscgi/awstats.pl?config=http.mydomain"

I get an error 404- file not found

if i check the error log is shows...

[Wed Nov 09 05:29:40 2005] [error] [client 10.10.10.1] File does not exist: /var/www/vhosts/mydomain/httpdocs/awstatscgi, referer: "http://mydomain/plesk-stat/menu.html"

Is it looking for the awstatscgi aliased dir under httpdocs dir??
the only thing in that dir are my .html files . . .

if this is the case what file do i need to edit so I can get it to point to the right place ?

man..what a chore...

thanks ;)
winter
 
i went through the entire setup_awstats.sh file to see if i missed anything.. (maybe a wrong dir again)

uhm...
I don't have a vhost.conf or vhost_ssl.conf file under my

/var/www/vhosts/mydomain/conf directory..

only files in there are

awstats.conf
httpd.include
httpd.include.new
webalizer.conf

isn't this a problem? (lol)

winter
 
yeh you should have a vhost.conf or vhost_ssl.conf which atleast has Include awstats.conf, this will definately be your problem as when awstats.conf isnt included the Alias's and mod_perl stuff isnt being loaded by Apache and thus /awstatscgi wont work

all your work is not going unappreciated tho :) i updated my script to automatically grab the path from psa.conf to handle different locations and updated the new_domain_name, just gotta make one to delete the awstats config files when physical hosting is deleted so awstats_updateall.pl doesnt keep trying to run stats on them then will post again
 
EUREKA!!!!!!!!!!

i created the vhost.conf file and after an httpd bounce...

IT WORKED!!!!

Thanks to everyone who helped out on this and other threads especially ..ivsdigitel, Proftiger, mian and Shadowman...

just one thing (lol) :) :) :)

does the setup_awstats.sh file create the vhost.conf file ?
because in my case it did not...

If it was already supposed to be there..it wasn't..
I suppose I could edit the file to create it no ?

mian...looking forward to you next script post... :)

especially the one to delete the files once the domain has been deleted :)

...and I am sure I will bug you about the maillogconvert.pl thing once I get some sleep :D

Wow..I feel like i just graduated from something... ;)

Thanks again guys!!

Winter
 
hey..while I got this working (finally- thanks to everyone for their help) ..

I was wondering why the vhost.conf file does not exist for any new domains created...

Does the setup_awstats.sh script create it or just check for it ?

thanks
 
U are right !
The whole if statements don;t work for some reason ....
Anyway anyone got any advice on when the best time is to run the cronjob ?
Cause I think webalizer deletes the logs ?
 
Back
Top