1. Please take a little time for this simple survey! Thank you for participating!
    Dismiss Notice
  2. Dear Pleskians, please read this carefully! New attachments and other rules Thank you!
    Dismiss Notice
  3. Dear Pleskians, I really hope that you will share your opinion in this Special topic for chatter about Plesk in the Clouds. Thank you!
    Dismiss Notice

Awstats and Plesk again

Discussion in 'Plesk for Linux - 8.x and Older' started by plmerlin, Sep 29, 2005.

  1. plmerlin

    plmerlin Guest

    0
     
    Hello,

    I installed Awstats according: http://forum.sw-soft.com/showthread.php?s=&threadid=23954 Thanks to ivsdigitel and ProfiTiger :)

    on a Fedora 2 box powered by Plesk 7.5.4

    The install went great. SSH test, no problem at all.

    Browser wyse http://mydomain.com/awstats/ gives me a 403 forbidden with [Thu Sep 29 20:33:53 2005] [error] [client x.x.x.x] Symbolic link not allowed: /home/httpd/vhosts/mydomain/httpdocs/awstats

    When I add the Options +FollowSymLinks I get a neat and nice 500 error :(

    Any idea?
     
  2. mian

    mian Guest

    0
     
    just finished revising my setup on FC2, the problem is because of suExec, you can get around it by using mod_perl to run awstats as so

    you can put this in event manager with the parameters <new_domain_name> <new_ssl_support> or to call it manually ./setup_awstats.sh domain.com 0 or 1 for if they have SSL support enabled. this is because if the user has SSL support enabled Plesk sets their statistics up on the SSL domain otherwise on the non-SSL domain.

    it creates 5 config files in /etc/awstats for each domain (mail, https, http, ftp, anonftp). i patched my awstats maillogconvert.pl so it will only get the relevant entries for the users domain, you will have todo the same if you wish todo mail stats.

    each config file includes awstats.model.conf so you can configure generic entries in there. I use
    DirCgi="/awstatscgi"
    DirIcons="/awstatsicons"
    StyleSheet="/awstatscss/awstats_default.css"
    the reason for using /awstatscgi directory is to avoid the worms which automatically look for /awstats/awstats.pl to exploit it

    awstats files are logged to the users statistics/awstats directory.

    it uses the log.processed file to avoid missing any entries so you should crontab awstats_updateall.pl to run 30-60 minutes after the Plesk statistics. this way Plesk statistics will run, cycle the current log files to .processed then awstats will process those files.

    an index.html and menu.html is created in the users statistics directory with a top frame offering links to awstats and webalizer log files so the user can use http(or https if they have SSL enabled)://domain.com/plesk-stat/
     
  3. mian

    mian Guest

    0
     
    setup_awstats.sh
    Code:
    #!/bin/bash
    
    domain=$1
    ssl_support=$2
    
    # make awstats directory
    mkdir -p /home/httpd/vhosts/$domain/statistics/awstats/
    
    # remove previously added awstats to avoid dupe
    if [ -e /home/httpd/vhosts/$domain/conf/vhost_ssl.conf ]; then
            cat /home/httpd/vhosts/$domain/conf/vhost_ssl.conf |grep -v "Include /home/httpd/vhosts/$domain/conf/awstats.conf" > /home/httpd/vhosts/$domain/conf/vhost_ssl.conf.tmp
            mv /home/httpd/vhosts/$domain/conf/vhost_ssl.conf.tmp /home/httpd/vhosts/$domain/conf/vhost_ssl.conf
    fi
    
    # add awstats to SSL domain if SSL was specified
    if [ "$ssl_support" -eq 1 ] && [ -e /home/httpd/vhosts/$domain/conf/vhost_ssl.conf ]; then
    cat >> /home/httpd/vhosts/$domain/conf/vhost_ssl.conf <<EOF
    Include /home/httpd/vhosts/$domain/conf/awstats.conf
    EOF
    fi
    
    # remove previously added awstats to avoid dupe
    if [ -e /home/httpd/vhosts/$domain/conf/vhost.conf ]; then
            cat /home/httpd/vhosts/$domain/conf/vhost.conf |grep -v "Include /home/httpd/vhosts/$domain/conf/awstats.conf" > /home/httpd/vhosts/$domain/conf/vhost.conf.tmp
            mv /home/httpd/vhosts/$domain/conf/vhost.conf.tmp /home/httpd/vhosts/$domain/conf/vhost.conf
    fi
    
    # add awstats to non-SSL domain if SSL is disabled
    if [ "$ssl_support" -eq 0 ] && [ -e /home/httpd/vhosts/$domain/conf/vhost.conf ]; then
    cat >> /home/httpd/vhosts/$domain/conf/vhost.conf <<EOF
    Include /home/httpd/vhosts/$domain/conf/awstats.conf
    EOF
    fi
    
    # generate apache awstats config
    cat > /home/httpd/vhosts/$domain/conf/awstats.conf <<EOF
    Alias /awstatscss /usr/local/awstats/wwwroot/css/
    Alias /awstatsicons /usr/local/awstats/wwwroot/icon/
    Alias /awstatscgi /usr/local/awstats/wwwroot/cgi-bin/
    
    <Directory /usr/local/awstats/wwwroot/cgi-bin/>
            <IfModule mod_perl.c>
            <Files ~ (\.pl$)>
                    SetHandler perl-script
                    PerlHandler ModPerl::Registry
                    Options ExecCGI
                    allow from all
                    PerlSendHeader On
            </Files>
            </IfModule>
    </Directory>
    EOF
    
    # generate awstats mail config
    cat > /etc/awstats/awstats.mail.$domain.conf <<EOF
    Include "/etc/awstats/awstats.model.conf"
    
    DNSLookup=0
    DirData="/home/httpd/vhosts/$domain/statistics/awstats/"
    SiteDomain="mail.$domain"
    LogFile="/usr/local/awstats/tools/maillogconvert.pl vadmin $domain < /usr/local/psa/var/log/maillog.processed |"
    LogType=M
    LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd"
    LevelForBrowsersDetection=0
    LevelForOSDetection=0
    LevelForRefererAnalyze=0
    LevelForRobotsDetection=0
    LevelForWormsDetection=0
    LevelForSearchEnginesDetection=0
    LevelForFileTypesDetection=0
    ShowMenu=1
    ShowSummary=HB
    ShowMonthStats=HB
    ShowDaysOfMonthStats=HB
    ShowDaysOfWeekStats=HB
    ShowHoursStats=HB
    ShowDomainsStats=0
    ShowHostsStats=HBL
    ShowAuthenticatedUsers=0
    ShowRobotsStats=0
    ShowEMailSenders=HBML
    ShowEMailReceivers=HBML
    ShowSessionsStats=0
    ShowPagesStats=0
    ShowFileTypesStats=0
    ShowFileSizesStats=0
    ShowBrowsersStats=0
    ShowOSStats=0
    ShowOriginStats=0
    ShowKeyphrasesStats=0
    ShowKeywordsStats=0
    ShowMiscStats=0
    ShowHTTPErrorsStats=0
    ShowSMTPErrorsStats=1
    EOF
    
    # generate awstats anonymous ftp config
    cat > /etc/awstats/awstats.anonftp.$domain.conf <<EOF
    Include "/etc/awstats/awstats.model.conf"
    
    DirData="/home/httpd/vhosts/$domain/statistics/awstats/"
    SiteDomain="ftp.$domain"
    LogFile="/home/httpd/vhosts/$domain/statistics/logs/xferlog_anonymous.processed"
    LogType=F
    LogFormat="%time3 %other %host %bytesd %url %other %other %method %other %logname %other %code %other %other"
    LogSeparator="\s"
    NotPageList=""
    LevelForBrowsersDetection=0
    LevelForOSDetection=0
    LevelForRefererAnalyze=0
    LevelForRobotsDetection=0
    LevelForWormsDetection=0
    LevelForSearchEnginesDetection=0
    ShowLinksOnUrl=0
    ShowSummary=UVHB
    ShowMonthStats=UVHB
    ShowDaysOfMonthStats=HB
    ShowDaysOfWeekStats=HB
    ShowHoursStats=HB
    ShowDomainsStats=HB
    ShowHostsStats=HBL
    ShowAuthenticatedUsers=HBL
    ShowRobotsStats=0
    ShowEMailSenders=0
    ShowEMailReceivers=0
    ShowSessionsStats=1
    ShowPagesStats=PBEX
    ShowFileTypesStats=HB
    ShowFileSizesStats=0
    ShowBrowsersStats=0
    ShowOSStats=0
    ShowOriginStats=0
    ShowKeyphrasesStats=0
    ShowKeywordsStats=0
    ShowMiscStats=0
    ShowHTTPErrorsStats=0
    
    ShowSMTPErrorsStats=0
    EOF
    
    # generate awstats ftp config
    cat > /etc/awstats/awstats.ftp.$domain.conf <<EOF
    Include "/etc/awstats/awstats.model.conf"
    
    DirData="/home/httpd/vhosts/$domain/statistics/awstats/"
    SiteDomain="ftp.$domain"
    LogFile="/home/httpd/vhosts/$domain/statistics/logs/xferlog_regular.processed"
    LogType=F
    LogFormat="%time3 %other %host %bytesd %url %other %other %method %other %logname %other %code %other %other"
    LogSeparator="\s"
    NotPageList=""
    LevelForBrowsersDetection=0
    LevelForOSDetection=0
    LevelForRefererAnalyze=0
    LevelForRobotsDetection=0
    LevelForWormsDetection=0
    LevelForSearchEnginesDetection=0
    ShowLinksOnUrl=0
    ShowSummary=UVHB
    ShowMonthStats=UVHB
    ShowDaysOfMonthStats=HB
    ShowDaysOfWeekStats=HB
    ShowHoursStats=HB
    ShowDomainsStats=HB
    ShowHostsStats=HBL
    ShowAuthenticatedUsers=HBL
    ShowRobotsStats=0
    ShowEMailSenders=0
    ShowEMailReceivers=0
    ShowSessionsStats=1
    ShowPagesStats=PBEX
    ShowFileTypesStats=HB
    ShowFileSizesStats=0
    ShowBrowsersStats=0
    ShowOSStats=0
    ShowOriginStats=0
    ShowKeyphrasesStats=0
    ShowKeywordsStats=0
    ShowMiscStats=0
    ShowHTTPErrorsStats=0
    ShowSMTPErrorsStats=0
    EOF
    
    # generate awstats http config
    cat > /etc/awstats/awstats.http.$domain.conf <<EOF
    Include "/etc/awstats/awstats.model.conf"
    
    DirData="/home/httpd/vhosts/$domain/statistics/awstats/"
    LogFile="/home/httpd/vhosts/$domain/statistics/logs/access_log.processed"
    LogType=W
    LogFormat=1
    SiteDomain="www.$domain"
    HostAliases="$domain www.$domain"
    EOF
    
    # generate awstats ssl https config
    cat > /etc/awstats/awstats.https.$domain.conf <<EOF
    Include "/etc/awstats/awstats.model.conf"
    
    DirData="/home/httpd/vhosts/$domain/statistics/awstats/"
    LogFile="/home/httpd/vhosts/$domain/statistics/logs/access_ssl_log.processed"
    LogType=W
    LogFormat=1
    SiteDomain="www.$domain"
    HostAliases="$domain www.$domain"
    UseHTTPSLinkForUrl="/"
    EOF
    
    # generate index.html frameset page
    cat > /home/httpd/vhosts/$domain/statistics/index.html <<EOF
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Frameset//EN" "http://www.w3.org/TR/html4/frameset.dtd">
    <html>
    <head>
    <title>$domain Statistics</title>
    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
    </head>
    
    <frameset rows="30,*" frameborder="NO" border="0" framespacing="0">
      <frame src="menu.html" name="topFrame" scrolling="NO" noresize >
      <frame src="/awstatscgi/awstats.pl?config=http.$domain" name="mainFrame">
    </frameset>
    <noframes><body>
    </body></noframes>
    </html>
    EOF
    
    # generate top menu for frame with links to awstats & webalizer stats
    cat > /home/httpd/vhosts/$domain/statistics/menu.html <<EOF
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
    <html>
    <head>
    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
    <style type="text/css">
    BODY { font-family: Tahoma, Verdana, Arial; font-size: 8pt; }
    A { color: #404040; text-decoration: none; font-weight: bold; font-size: 8pt; }
    A:hover { text-decoration: none; color: #808080; }
    </style>
    </head>
    
    <body>
    
    <table width="100%" cellpadding="0" cellspacing="0"><tr><td align="left">
    [ <b>awstats</b> ]
    <a href="/awstatscgi/awstats.pl?config=anonftp.$domain" target="mainFrame" title="Anonymous FTP Statistics for $domain">anonymous ftp</a> <small>|</small>
    <a href="/awstatscgi/awstats.pl?config=ftp.$domain" target="mainFrame" title="FTP Statistics for $domain">ftp</a> <small>|</small>
    <a href="/awstatscgi/awstats.pl?config=mail.$domain" target="mainFrame" title="Mail Statistics for $domain">mail</a> <small>|</small>
    <a href="/awstatscgi/awstats.pl?config=https.$domain" target="mainFrame" title="SSL Statistics for $domain">ssl</a> <small>|</small>
    <a href="/awstatscgi/awstats.pl?config=http.$domain" target="mainFrame" title="Web Statistics for $domain">web</a>
    </td><td align="right">
    [ <b>webalizer</b> ]
    <a href="/anon_ftpstat" target="mainFrame" title="Anonymous FTP Statistics for $domain">anonymous ftp</a> <small>|</small>
    <a href="/ftpstat" target="mainFrame" title="FTP Statistics for $domain">ftp</a> <small>|</small>
    <a href="/webstat-ssl" target="mainFrame" title="SSL Statistics for $domain">ssl</a> <small>|</small>
    <a href="/webstat" target="mainFrame" title="Web Statistics for $domain">web</a>
    </td></tr></table>
    
    </body>
    </html>
    EOF
    
    # make sure vhost.conf or vhost_ssl.conf is included by httpd.include
    /usr/local/psa/admin/bin/websrvmng --vhost-name=$domain
    
     
  4. plmerlin

    plmerlin Guest

    0
     
    Impressive :) it works perfectly
     
  5. Cranky

    Cranky Guest

    0
     
    That's quite a neat script. I've not tested it yet, but it sure looks like it should work.
     
  6. plmerlin

    plmerlin Guest

    0
     
    I get a weird behavior:

    Every morning at 3:08am the Include /home/httpd/vhosts/mydomain.com/conf/awstats.conf is removed from the http.conf file.

    Any idea from where it's coming?
    Thanks
     
  7. Whistler

    Whistler Guest

    0
     
    Maybe because you should put it in your vhost.conf file instead?
     
  8. plmerlin

    plmerlin Guest

    0
     
    I tried that to but it got removed the next day at 3:08am
     
  9. Whistler

    Whistler Guest

    0
     
    Hmm... what file are you using? What utility besides the PSA - /path_psa/admin/sbin/statistics utility are you running at 3.08?

    As far as I know - no PSA native utility will touch your vhost.conf (and vhost_ssl.conf) files.
     
  10. jamesyeeoc

    jamesyeeoc Guest

    0
     
    4PSA addons can and will modify the vhost.conf and vhost_ssl.conf files. If he is running any of their stuff, he would have to put his changes into the vhost_in.conf or vhost_ssl_in.conf files and run the 4PSA utility:

    /usr/local/psa/admin/bin/pdomain <domain_name>
     
  11. Whistler

    Whistler Guest

    0
     
    I know... That's why I'm asking about what stuff is runned other than Plesks own utilities as these won't mess with the vhost* files.
     
  12. plmerlin

    plmerlin Guest

    0
     
    I'm just running Plesk. I didn't purchase 4PSA yet.

    Also I noticed that the http.include changed only on that site. Not the others.

    Really weird.
    Got it... the site was ftp backup by Plesk FTP backup schedule.
    The other one wasn't.
     
  13. Whistler

    Whistler Guest

    0
     
    Do you by any chance read what I'm writing?

    You have to put your custom changes in a vhost.conf file and not the httpd.include - as this file is overwritten by Plesk when ever a update is needed...

    Please read the manual! It's not strange that the httpd.include is updated - it's supposed to!
     
  14. plmerlin

    plmerlin Guest

    0
     
    Yes we read what you're saying and we tried to put Include /home/httpd/vhosts/mydomain.com/conf/awstats.conf in the vhost.conf which is an empty file but it doesn't work. I get a nice 500 error.
     
  15. mian

    mian Guest

    0
     
    you shouldnt have to put include awstats.conf anywhere yourself, the script does it all.. check crontab and see what you have running at 3:08am
     
  16. Whistler

    Whistler Guest

    0
     
    Well, then maybe we should consentrate on finding what you've got wrong in the vhost.conf, awstats.conf or your cgi-bin - as you won't fix your problem by adding it in httpd.conf (yes, for now - but every time a change is made to that domain - your custom changes will be lost!).

    What does the apache error log say? What's the exact content of your vhost.conf?

    As far as I would guess - you probably haven't set the correct permissions on a CGI-file or something like that.
     
  17. plmerlin

    plmerlin Guest

    0
     
    The include wasn't when I ran the script so I had to put it manually.
    There nothing running at 3:08 or close to 3:08am.
     
  18. AceMan

    AceMan Guest

    0
     
    In what way did you patch maillogconvert.pl?

    I'm guessing you added another CLI argument, and then are // reg matching for the domain w/in the line in the while?


    Is there a way to pipe content to maillogconvert.pl instead of < ?

    I'm thinking grep "domain.com" maillog.processed.

    I'd rather not patch maillogconvert.pl if I don't have to.
     
  19. mian

    mian Guest

    0
     
    it may work the problem is not every line contains the domain name and maillogconvert processes multiple lines gathering bits of information from each by the message id so if you used grep you would filter out important lines because they don't contain the domain name however they are needed and are recognised only by mailid. have to get ready for work now but will make a patch later, it's only like 5 lines.
     
  20. mian

    mian Guest

    0
     
    see attached maillog.diff.txt which takes a domain to filter (to/from) addresses as the paramater as per the setup_awstats.sh

    the second patches awstats.pl to ensure you can only view configs matching this SERVER_NAME to prevent users viewing other domains statistics by specifying ?config=http.otherdomain.com
     
Loading...