• If you are still using CentOS 7.9, it's time to convert to Alma 8 with the free centos2alma tool by Plesk or Plesk Migrator. Please let us know your experiences or concerns in this thread:
    CentOS2Alma discussion

Issue Very high Memory high use and very low process

Antrax1

Basic Pleskian
Hi,
I´m a little confuse with memory uses, my server have very high Memory use but have very low process, ¿is it a normal situation?

Server config:
Ubuntu ‪Ubuntu 18.04.5 LTS
Maria 1:10.2.36+maria~bionic
Php-fpm 7.3
2 cores
8 ram
1 site wordpress
1 site magento 2.3.6

My.cnf:




Code:
# MariaDB database server configuration file.
#
# You can copy this file to one of:
# - "/etc/mysql/my.cnf" to set global options,
# - "~/.my.cnf" to set user-specific options.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

# This will be passed to all mysql clients
# It has been reported that passwords should be enclosed with ticks/quotes
# escpecially if they contain "#" chars...
# Remember to edit /etc/mysql/debian.cnf when changing the socket location.
[client]
port        = 3306
socket        = /var/run/mysqld/mysqld.sock

# Here is entries for some specific programs
# The following values assume you have at least 32M ram

# This was formally known as [safe_mysqld]. Both versions are currently parsed.
[mysqld_safe]
socket        = /var/run/mysqld/mysqld.sock
nice        = 0

[mysqld]
sql_mode=ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
bind-address = ::ffff:127.0.0.1
local-infile=0
skip-name-resolve
#
# * Basic Settings
#
user        = mysql
pid-file    = /var/run/mysqld/mysqld.pid
socket        = /var/run/mysqld/mysqld.sock
port        = 3306
basedir        = /usr
datadir        = /var/lib/mysql
tmpdir        = /tmp
lc_messages_dir    = /usr/share/mysql
lc_messages    = en_US
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
#
# * Fine Tuning
#
max_connections        = 100
connect_timeout        = 5
wait_timeout        = 31536000
max_allowed_packet    = 32M
thread_cache_size       = 128
#sort_buffer_size    = 4M  Ajustado
bulk_insert_buffer_size    = 16M
tmp_table_size        = 32M
max_heap_table_size    = 32M
table_definition_cache  = -1
join_buffer_size    = 256K
#
# * MyISAM
#
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched. On error, make copy and try a repair.
myisam_recover_options = BACKUP
key_buffer_size        = 128M
#open-files-limit    = 2000
table_open_cache    = 64
myisam_sort_buffer_size    = 512M
concurrent_insert    = 2
#read_buffer_size    = 2M Ajustado
#read_rnd_buffer_size    = 1M Ajustado
#
# * Query Cache Configuration
#
# Cache only tiny result sets, so we can fit more in the query cache.
query_cache_limit        = 128K
query_cache_size        = 0
# for more write intensive setups, set to DEMAND or OFF
query_cache_type        = 0
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error logging goes to syslog due to /etc/mysql/conf.d/mysqld_safe_syslog.cnf.
log_error =  /var/log/mysql/mariadb-error.log
# we do want to know about network errors and such
log_warnings        = 2
#
# Enable the slow query log to see queries with especially long duration
#slow_query_log[={0|1}]
#slow_query_log_file    = /var/log/mysql/mariadb-slow.log
long_query_time = 10
#log_slow_rate_limit    = 1000
log_slow_verbosity    = query_plan

#log-queries-not-using-indexes
#log_slow_admin_statements
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
#server-id        = 1
#report_host        = master1
#auto_increment_increment = 2
#auto_increment_offset    = 1
#log_bin            = /var/log/mysql/mariadb-bin
log_bin_index        = /var/log/mysql/mariadb-bin.index
# not fab for performance, but safer
#sync_binlog        = 1
#expire_logs_days    = 10
#max_binlog_size         = 100M
# slaves
#relay_log        = /var/log/mysql/relay-bin
#relay_log_index    = /var/log/mysql/relay-bin.index
#relay_log_info_file    = /var/log/mysql/relay-bin.info
#log_slave_updates
#read_only
#
# If applications support it, this stricter sql_mode prevents some
# mistakes like inserting invalid dates etc.
#sql_mode        = NO_ENGINE_SUBSTITUTION,TRADITIONAL
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
default_storage_engine    = InnoDB
innodb_buffer_pool_size    = 4G
innodb_buffer_pool_instances = 4
innodb_log_buffer_size    = 8M
innodb_file_per_table    = 1
innodb_open_files    = 400
innodb_io_capacity    = 400
innodb_flush_method    = O_DIRECT
innodb_log_file_size    = 512M
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem

#
# * Galera-related settings
#
[galera]
# Mandatory settings
#wsrep_on=ON
#wsrep_provider=
#wsrep_cluster_address=
#binlog_format=row
#default_storage_engine=InnoDB
#innodb_autoinc_lock_mode=2
#
# Allow server to accept connections on all interfaces.
#
#bind-address=0.0.0.0
#
# Optional setting
#wsrep_slave_threads=1
#innodb_flush_log_at_trx_commit=0

[mysqldump]
quick
quote-names
max_allowed_packet    = 32M

[mysql]
#no-auto-rehash    # faster start of mysql but no tab completion

[isamchk]
key_buffer        = 16M

#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#
!includedir /etc/mysql/conf.d/
 

Attachments

  • Sin título-1.jpg
    Sin título-1.jpg
    287.5 KB · Views: 32
Nothing strange to see in the chart. 8 GB memory is not really a lot of RAM. Most of it is used by cache. That is the normal Linux behavior. Active memory seems to be around 2 GB in your system, 6 GB cached. These will be freed automatically when needed.

Example for a system with 256 GB RAM:
Code:
[root@<sysname>]# free
              total        used        free      shared  buff/cache   available
Mem:      263859984     5236120     2810408      489628   255813456   256824104
Swap:     134217724       80384   134137340

Same situation there: Almost all of the memory seems to be used, but almost all of that usage is cache only. No problem.
 
Ok, but sometimes I have mysql alerting memory:

mysql.jpg
And sometimes I have a high load server and crash to memory lack

mysql_2.jpg
 
There is always a chance that websites require more resources than the server offers.

Your innodb_buffer_pool_size is large compared to the total RAM available. Maybe you would want to limit it?

You could also examine what the database server is actually serving, e.g. look into the processlist (SQL command "showprocesslist") or you could try to use a script like MySQLTuner (major/MySQLTuner-perl) to get a better insight what your database actually needs.

If you do not have enough RAM sometimes, you could analyze what is using all the RAM at the given time. Normally it will be processes like PHP-FPM or httpd or apache2. You could consider adding more RAM or using a swap partition for such cases so that at least enough "virtual" RAM is available just in case the system reaches a limit.
 
thank you for your anwer.

Adjusted key_buffer_size with this code:
Code:
SELECT CONCAT(ROUND(KBS/POWER(1024,  IF(PowerOf1024<0,0,IF(PowerOf1024>3,0,PowerOf1024)))+0.4999),  SUBSTR(' KMG',IF(PowerOf1024<0,0,  IF(PowerOf1024>3,0,PowerOf1024))+1,1))  recommended_key_buffer_size FROM  (SELECT LEAST(POWER(2,32),KBS1) KBS  FROM (SELECT SUM(index_length) KBS1  FROM information_schema.tables  WHERE engine='MyISAM' AND  table_schema NOT IN ('information_schema','mysql')) AA ) A,  (SELECT 2 PowerOf1024) B;

Also adjusted to innodb_buffer_pool_size = 1G
And reduce memory from 7G to 5G

This server is focused to magento 2.3 and I see some memory hungry ram processes:
Code:
USER            CPU    MEM        VIRTUAL    RESIDENT    
elasticsearch    0.4    17.0    3.8 GB    1.4 GB    
rabbitmq        0.4    1.0        2.2 GB    84.9 MB    

CONSUMERS RABBITMQ:

xxxxxx        0.0    1.2    448.4 MB    102.7 MB    ?    S    Jan11    00:00:23    /opt/plesk/php/7.3/bin/php /var/www/vhosts/AAAAAAA.COM/httpdocs/bin/magento queue:consumers:start inventory.mass.update --single-thread --max-messages=10000
    
xxxxxx        0.0    1.2    448.2 MB    102.1 MB    ?    S    Jan11    00:00:23    /opt/plesk/php/7.3/bin/php /var/www/vhosts/AAAAAAA.COM/httpdocs/bin/magento queue:consumers:start product_action_attribute.update --single-thread --max-messages=10000
    
xxxxxx        0.0    1.2    448.2 MB    102.2 MB    ?    S    Jan11    00:00:23    /opt/plesk/php/7.3/bin/php /var/www/vhosts/AAAAAAA.COM/httpdocs/bin/magento queue:consumers:start product_action_attribute.website.update --single-thread --max-messages=10000
    
xxxxxx        0.0    1.2    448.2 MB    101.3 MB    ?    S    Jan11    00:00:23    /opt/plesk/php/7.3/bin/php /var/www/vhosts/AAAAAAA.COM/httpdocs/bin/magento queue:consumers:start exportProcessor --single-thread --max-messages=10000
    
xxxxxx        0.0    1.2    448.2 MB    101.5 MB    ?    S    Jan11    00:00:23    /opt/plesk/php/7.3/bin/php /var/www/vhosts/AAAAAAA.COM/httpdocs/bin/magento queue:consumers:start inventory.source.items.cleanup --single-thread --max-messages=10000
    
xxxxxx        0.0    1.2    448.2 MB    101.5 MB    ?    S    Jan11    00:00:23    /opt/plesk/php/7.3/bin/php /var/www/vhosts/AAAAAAA.COM/httpdocs/bin/magento queue:consumers:start inventory.reservations.update --single-thread --max-messages=10000
    
xxxxxx        0.0    1.2    448.2 MB    101.5 MB    ?    S    Jan11    00:00:23    /opt/plesk/php/7.3/bin/php /var/www/vhosts/AAAAAAA.COM/httpdocs/bin/magento queue:consumers:start codegeneratorProcessor --single-thread --max-messages=10000
    
xxxxxx        0.0    1.2    446.2 MB    101.2 MB    ?    S    Jan11    00:00:23    /opt/plesk/php/7.3/bin/php /var/www/vhosts/AAAAAAA.COM/httpdocs/bin/magento queue:consumers:start inventory.reservations.cleanup --single-thread --max-messages=10000

Is it possible to adjust these memory requirements?
 

Attachments

  • memory_2.jpg
    memory_2.jpg
    145.3 KB · Views: 6
Back
Top