GoAccess taking huge RAM allocations & running long times

[aFrI]

TITLE

GoAccess taking huge RAM allocations & running long times

PRODUCT, VERSION, OPERATING SYSTEM, ARCHITECTURE

Plesk Obsidian 18.0.79 Update #4 (on Debian Linux 12.15)

PROBLEM DESCRIPTION

Good day,

As GoAccess was introduced as the new stats tool, I was switching from the former configured AWStats. As I was checking our monitorings I recognized periods of huge RAM allocations since then - today I went to the server and found GoAccess seems to be running a bit wild, screenshot of htop:

As I have absolutely no experience with GoAccess, I would like to ask if that may be considered normal - or if the set parameters are producing the situation.

I would believe a statistics-tool running on a daily basis should possibly not have so much hunger for RAM..

I am switching back to AWStats for now - I am open for delivering more exact details if needed or test mitigations.

STEPS TO REPRODUCE

Switch from AWStats to GoAccess for a Plesk site.

ACTUAL RESULT

High allocations in RAM from the GoAccess process & steadily allocating one core with 100% for hours.

EXPECTED RESULT

I can just go from my expectations, but little knowledge of the GoAccess product: I would believe such tooling should not use such amounts of RAM.

ANY ADDITIONAL INFORMATION

(DID NOT ANSWER QUESTION)

YOUR EXPECTATIONS FROM PLESK SERVICE TEAM

Help with sorting out

 

[aFrI]

I just tested to give a sigterm, but none of the two running processes are considering it..

short strace on the one process which seems to still iterate over a log file:



Update: And I am stupid, and just did find the other similar topic (GoAccess taking huge RAM allocations & running long times) - so this can be closed as a duplicate - sorry for me not realizing this before posting.. *sigh

 

[Sebahat.hadzhi]

Hello, @aFrI . We indeed had a similar complaint recently under this thread. I am assuming that's the one you had in mind in your second reply. Could you please confirm if you also have website(s) with dynamic URLs (e.g. search, filtering, e-commerce)? We do have an existing task to review the behavior, I just want to determine if the usage spike is caused by the same reason in your case. Thank you in advance.

 

[aFrI]

Yes, our site has PHP functionality working on different GET parameters to offer content, so I would consider us having dynamic URLs. It is an online forum.

 

[goa]

GoAccess dev here. I came across this thread and thought I'd share a bit of context that may help.

In GoAccess v1, memory usage is driven more by retained cardinality than by raw request count. With keep-last=90, up to 90 distinct log dates are retained, but there is no fixed row or memory limit. Repeated requests aggregate efficiently, while unique IP addresses, user agents, URLs, hosts, referrers, and virtual hosts (vhosts) create additional hash entries.

Based on my measurements using a typical Apache/Nginx combined log, 45 GiB of memory could correspond to anywhere from roughly 40 million highly unique requests to approximately 1.7 billion requests with substantial repetition. I recommend upgrading from 1.10.2 to 1.11, which includes a major storage redesign. Typical memory savings are around 15–20%, with potentially greater reductions when unique visitor data is the primary driver of memory usage.

If query strings contain per-request identifiers, using -q can reduce memory usage dramatically by grouping requests by path. Disabling unneeded high-cardinality panels can also significantly reduce memory consumption.

To refine the estimate, please share the exact command or configuration, the number of retained valid requests, unique visitors, unique files/URLs, unique hosts, and unique referrers, as well as whether --list-agents, geoip, or --date-spec=hr/min are enabled.

 

[aFrI]

@goa thanks for the insights and recommendations. From my side I dont need more accurate estimates, but more of a fixture from Plesk as provider of the platform making use of GoAccess. So I do not really want to investigate the exact requests logged further - except those details would help you to improve GoAccess, then I would invest more time to help.

@Sebahat.hadzhi goa's post may be relevant for the Plesk team.