fail2ban hangs after making change

Refer to KB http://kb.sp.parallels.com/en/122407

Symptoms

After enabling a jail, fail2ban service can be neither restarted nor stopped. The service status is shown wrong, without the jails list:

# service fail2ban status
fail2ban-server (pid  3291) is running

/var/log/fail2ban.log shows that it stopped when adding log files to its monitoring pool:

2014-07-27 21:09:25,487 fail2ban.filter [25047]: INFO    Added logfile = /var/www/vhosts/system/dom1.com/logs/proxy_access_log
2014-07-27 21:09:25,985 fail2ban.filter [25047]: INFO    Added logfile = /var/www/vhosts/system/domain.com/logs/proxy_access_ssl_log

Cause

Fail2ban has plesk-apache-badbot and plesk-apache (or other big) jails enabled. That jail forces fail2ban to parse all the access and error logs for each virtual host and Apache’s access log. In case if there are a lot of virtual host access logs, the service hangs by resource over usage trying to parse all of them.

NOTE: When you enable this jail in panel you might see the warning:

Warning: Fail2Ban might not work well if there are many domains and Fail2Ban has to monitor too many log files.

Resolution

The following instruction should be applied in case if there are less then 300 domains, and amount of log files in jail should be reduced:

  1. Kill the stuck process(es) by PID (exercise caution, it might be a good idea to check PIDs which it will kill first by omitting the last part after |):
    # ps aux | grep fail2ban|awk '{print $2}'|xargs kill -9
    
  2. Remove .pid file:
    # rm -f /var/run/fail2ban/fail2ban.pid
    
  3. Reduce the amount of logs to parse for the jail plesk-apache-badbot (or disable the jail altohether). Open file/etc/fail2ban/jail.d/plesk.conf and change the mask of the path to logs from '*access*log' to '*access_log':
    [plesk-apache-badbot]
    
    enabled  = true
    filter   = apache-badbots
    action   = iptables-multiport[name=BadBots, port="http,https,7080,7081"]
    logpath  = /var/www/vhosts/system/*/logs/*access_log
               /var/log/httpd/*access_log
    
  4. If fail2ban service is running, execute fail2ban-client reload. Otherwise start the service.

The below instruction is for big amount of domains (more then 300):

Fail2Ban can use a lot of RAM on the server, in case if it monitors a lot of jails with many log files! Make sure that the server will not experience the out-of-memory condition before applying this solution! If it does – disable some jails.

In case if you have a very big amount of domains on your Plesk server, and the above workaround doesn’t help, you may divide the logs by different jails, so that they are loaded one by one and therefore minimize the amount of logs in one jail. This should help, since issue is caused by a single big jail with a lot of logs, but it won’t be happening when there are a lot of jails with small amount of logs included.

Please use the following commands to create separate jails for domains according to the first name letter\digit:

  1. Get admin email:
    admin_email=`mysql -Ns -uadmin -p\`cat /etc/psa/.psa.shadow\` psa -Ne"select email from clients where login='admin'"`
    
  2. Set plesk-apache jails:
     for i in a b c d e f g h i j k l m n o p q r s t u v w x y z 1 2 3 4 5 6 7 8 9 0;do find /var/www/vhosts/system/$i*/logs/error_log 2>/dev/null 1>/dev/null; found=`echo $?`;if [ $found == "0" ];then echo "[[\"usedns\",\"no\"],[\"logpath\",\"\\/var\\/www\\/vhosts\\/system\\/$i*\\/logs\\/error_log\"],[\"enabled\",\"true\"],[\"filter\",\"apache-auth\"],[\"maxretry\",\"6\"],[\"__source__\",\"jail.d\\/plesk.conf\"],[\"action\",\"iptables-multiport[name=apache, port=\\\"http,https,7080,7081\\\"]\"],[\"ignoreip\",\"127.0.0.1\/8\"],[\"bantime\",\"600\"],[\"destemail\",\"$admin_email\"],[\"findtime\",\"600\"],[\"backend\",\"auto\"]]"|/usr/local/psa/admin/bin/f2bmng --set-jail plesk-apache-$i ;fi;done
    
  3. Set plesk-apache-badbot jails:
     for i in a b c d e f g h i j k l m n o p q r s t u v w x y z 1 2 3 4 5 6 7 8 9 0;do find /var/www/vhosts/system/$i*/logs/error_log 2>/dev/null 1>/dev/null; found=`echo $?`;if [ $found == "0" ];then echo "[[\"usedns\",\"no\"],[\"logpath\",\"\\/var\\/www\\/vhosts\\/system\\/$i*\\/logs\\/*access_log\"],[\"enabled\",\"true\"],[\"filter\",\"apache-badbots\"],[\"maxretry\",\"100\"],[\"__source__\",\"jail.d\\/plesk.conf\"],[\"action\",\"iptables-multiport[name=BadBots, port=\\\"http,https,7080,7081\\\"]\"],[\"ignoreip\",\"127.0.0.1\/8\"],[\"bantime\",\"172800\"],[\"destemail\",\"$admin_email\"],[\"findtime\",\"600\"],[\"backend\",\"auto\"]]" |/usr/local/psa/admin/bin/f2bmng --set-jail plesk-apache-badbot-$i;fi;done
    
  4. In regular plesk-apache-badbot and plesk-apache jails, leave only general error\access logs file paths:

    plesk-apache-badbot:

    /var/log/httpd/*error_log
    

    plesk-apache:

    /var/log/httpd/*access_log
    
  5. On steps 2 and 3 we created jails only if there are some domains matching the first digit\letter of domain name. Otherwise, Fail2ban will not start due to configuration errors. Now we need to set up the script that adds a jail upon new domains creation:
    • Download the attached script , put it on your server, and grant executable permissions:
      wget http://kb.sp.parallels.com/Attachments/kcs-32570/add_jails.sh
      
      chmod +x add_jails.sh
      
    • Create tasks in Plesk event manager with the following parameters:

      Domain created lowest (0) root /root/add_jails.sh <new_domain_name>

      Default domain (the first domain added to a subscription or webspace) created lowest (0) root /root/add_jails.sh <new_domain_name>

  6. Use the first instruction to restart Fail2ban if it hanged.

horde time off

Horde was not reading the default server time (all users were seeing UTC). I set the default in this config file:

/usr/share/psa-horde/imp/config/prefs.local.php

I added this line:

$_prefs[‘timezone’][‘value’] = ‘US/Central’;

export products in magento

Using Magento’s Data-Profiles to Export your entire product database (if this does not work see OPTION 2)

OPTION 1
Navigate to System -> Import/Export -> Data-Profiles
Select Export All Products

Review your options in the Profile Wizard. You may wish to change the name of your export file under File Information -> File Name.

To define which product attributes you wish to export for each product, under Profile Wizard -> Data Format -> Export you can choose to export All Fields, or select specific product attributes you wish to export by selecting Only Mapped Fields and selecting your required attributes.magento-data-profile-data-format

To define a specific range of products you wish to export under Profile Wizard -> Export Filters -> you can filter your products by name, sku, product type, attribute set, price, stock quantity, visibility or status. Save and Continue Edit, and we’re ready to export.

Now that the parameters for your export are set, choose the Run Profile tab and click the Run Profile in Popup button. Your export will start automatically in a pop up window.

Once your export is complete, you now need to retrieve it from your server via FTP.

Once logged into your server, you will find you exported csv file waiting for you inside the var/export folder.

OPTION 2
The above did not work for me, and I found lots of references online saying the same thing. I was able to accomplish the same thing via the command line.

Install this script into main httpdocs directory. You can then specify the profile from OPTION 1 that you want it to run. The export will then be created in the same directory as the script.

Import/Export > Profiles

require_once(‘app/Mage.php’);
umask(0);
Mage::app(‘admin’);

$profile = Mage::getModel(‘dataflow/profile’);
$userModel = Mage::getModel(‘admin/user’);
$userModel->setUserId(0);
Mage::getSingleton(‘admin/session’)->setUser($userModel);
$profile->load($profileId);
if (!$profile->getId()) {
Mage::getSingleton(‘adminhtml/session’)->addError(‘ERROR: Incorrect profile id’);
}

Mage::register(‘current_convert_profile’, $profile);
$profile->run();

echo “EXPORT COMPLETE.\n”;
?>

Warning: the CA certificate does not sign the certificate.

When installing an SSL certificate in Parallels Panel (Plesk) along with an intermediate certificate you receive the error message

Warning: the CA certificate does not sign the certificate.

Generally it is safe to ignore this error. It is just Parallels Panel not being able to follow the complete CA root path. However, I have found that if you append the intermediate certificate to the CA certificate (instead of putting it in the intermediate certificate box), you do not get the error.

Here is the intermediate and root DomainSSL cert for globalsign:

—–BEGIN CERTIFICATE—–
MIIFQjCCBCqgAwIBAgISESFxABpzduLKxXYRi0KcPsx3MA0GCSqGSIb3DQEBCwUA
MGAxCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMTYwNAYD
VQQDEy1HbG9iYWxTaWduIERvbWFpbiBWYWxpZGF0aW9uIENBIC0gU0hBMjU2IC0g
RzIwHhcNMTQwOTI5MTU1OTQxWhcNMTUwOTMwMTU1OTQxWjBKMQswCQYDVQQGEwJV
UzEhMB8GA1UECxMYRG9tYWluIENvbnRyb2wgVmFsaWRhdGVkMRgwFgYDVQQDEw9p
bnRvdGhld2luZC5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCZ
4nwCxV+6qWgZONNxGyMNjdIbEzO+EaGG4TlhYBzsn6h3yDCtat18tWXIQ25J+l03
zSJS2CbfMR7yFr1iKlrNHtdkmvHSOi3fC/FlIjmsJYbP5IQTSF7UoLLt21Vs6T8x
ix/sH6u2rs8aGyfPBbduY607v0r8yqcIJ4j1WczNcsvpixXG9mPKSCEmOQB4QppO
mc7OaIR6UUXunzQFrccZYUP978hzGzgZHXQNSELgkHELNJwYJi7+CJ1KJSqNeOw/
j8B5h+GbhhPrdvOr8NPwjQ3k+3YxevpVBSOFIPrYC8Wz0SLqHcuiI41I7t2lIU8o
bKQfhHvSpF521KQH79BVAgMBAAGjggIKMIICBjAOBgNVHQ8BAf8EBAMCBaAwSQYD
VR0gBEIwQDA+BgZngQwBAgEwNDAyBggrBgEFBQcCARYmaHR0cHM6Ly93d3cuZ2xv
YmFsc2lnbi5jb20vcmVwb3NpdG9yeS8wYwYDVR0RBFwwWoIPaW50b3RoZXdpbmQu
Y29tghNvd2EuaW50b3RoZXdpbmQuY29tghRtYWlsLmludG90aGV3aW5kLmNvbYIc
YXV0b2Rpc2NvdmVyLmludG90aGV3aW5kLmNvbTAJBgNVHRMEAjAAMB0GA1UdJQQW
MBQGCCsGAQUFBwMBBggrBgEFBQcDAjBDBgNVHR8EPDA6MDigNqA0hjJodHRwOi8v
Y3JsLmdsb2JhbHNpZ24uY29tL2dzL2dzZG9tYWludmFsc2hhMmcyLmNybDCBlAYI
KwYBBQUHAQEEgYcwgYQwRwYIKwYBBQUHMAKGO2h0dHA6Ly9zZWN1cmUuZ2xvYmFs
c2lnbi5jb20vY2FjZXJ0L2dzZG9tYWludmFsc2hhMmcycjEuY3J0MDkGCCsGAQUF
BzABhi1odHRwOi8vb2NzcDIuZ2xvYmFsc2lnbi5jb20vZ3Nkb21haW52YWxzaGEy
ZzIwHQYDVR0OBBYEFPfk8s6E7BRobX+cTK8Ao/9z86IHMB8GA1UdIwQYMBaAFOpO
fNSALeUVgYYmjIJtwJikz5cPMA0GCSqGSIb3DQEBCwUAA4IBAQB9Fyabi4Ju+RU4
sWKcfZ6TnSj4dS/nZbKJtD9m2vSt55MVb/gpJMiz1Ytoi1ftj6jUvWiciIMcKNTT
OnpopipBgqvi5QbtcsPvT4cWw0xjxCqqZNPYvduKcHcZm2wFy7xargrn0keNP30+
gTMWBRVUVNECkqExv8tMG27MeVXPNzgz8pGk+qibR0Ysv/IBiqvdA3DGrPxTmuYS
b/lw6n4Ya/BwHcXhzOh63/d+CD+6cg/HWYt88gyycWfzOjtF65lRL1BUCYgsQLRP
ieHEnyzicLW+sWJhtvMgAkxwKpCOswb1puVxuD5QPUedIjVG+u1HtjrK+uQ1MclL
heUZjPom
—–END CERTIFICATE—–
—–BEGIN CERTIFICATE—–
MIIEXDCCA0SgAwIBAgILBAAAAAABMYnGQlgwDQYJKoZIhvcNAQELBQAwTDEgMB4G
A1UECxMXR2xvYmFsU2lnbiBSb290IENBIC0gUjMxEzARBgNVBAoTCkdsb2JhbFNp
Z24xEzARBgNVBAMTCkdsb2JhbFNpZ24wHhcNMTEwODAyMTAwMDAwWhcNMjIwODAy
MTAwMDAwWjBgMQswCQYDVQQGEwJCRTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1z
YTE2MDQGA1UEAxMtR2xvYmFsU2lnbiBEb21haW4gVmFsaWRhdGlvbiBDQSAtIFNI
QTI1NiAtIEcyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqd3MDrPi
MjndSSKoE2mTh4jhDO5xfb2Qh5ZdWfLMs9JYV1f5Ru9sJtg2Qo5+MLMvmj5Tex9u
tqJMRR880xWTHIntPPRX3sq97AaaaiqgGVJ/UdF0OQifq+vXhhMVl642w1RmDlry
oHOFMeOyZBRq/6WijiS7vYVSFaJ57vC17j249H2AvNmQNWW4F6mts5ifoH59bvs/
rXzCG1k2lto3MktLXTUCY47bp89i7swu1I3JvTxqkXKiIqdyLSDR+so32hiY5hYk
cSVLxOV7iVIJAv1ZKwRuygeB1LPa2tvjzICoVgcGfJYIN53bOLZiNJFiB3QBONhy
MOLrkHEmYsBX8wIDAQABo4IBKTCCASUwDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB
/wQIMAYBAf8CAQAwHQYDVR0OBBYEFOpOfNSALeUVgYYmjIJtwJikz5cPMEcGA1Ud
IARAMD4wPAYEVR0gADA0MDIGCCsGAQUFBwIBFiZodHRwczovL3d3dy5nbG9iYWxz
aWduLmNvbS9yZXBvc2l0b3J5LzA2BgNVHR8ELzAtMCugKaAnhiVodHRwOi8vY3Js
Lmdsb2JhbHNpZ24ubmV0L3Jvb3QtcjMuY3JsMD4GCCsGAQUFBwEBBDIwMDAuBggr
BgEFBQcwAYYiaHR0cDovL29jc3AyLmdsb2JhbHNpZ24uY29tL3Jvb3RyMzAfBgNV
HSMEGDAWgBSP8Et/qC5FJK5NUPpjmove4t0bvDANBgkqhkiG9w0BAQsFAAOCAQEA
asQTEbudYOYIgMIwjjpyB5oIB0XIrKrD85MGzbyWwKt7IzfBYFPvJPpKhbGSET9Y
Z3RAFBWwLNQme7DAeGM3MedBUDL/h2VtRi9mFsJ3cjffjB2ynXz79WmNDEGDBeXD
CDhdCaUwUQzemQAYJFATZ9U956HDXbA92Ho3wqPeCzxXJmuY/hfAUdEL8BdEK71i
gPIuTWgRjCLvlHt4p7WbguowPW2xYzOsZvok9lOdICfAXHRETYzo2yBAhQDmbSaf
dzhT8XdnuiJ3yEW84eLFJqSi6NDOgmdMyXg/4ntqs5XUwfpSfUQaAuXxScCedcp3
IAITFUgbBl1wm7duUhuklQ==
—–END CERTIFICATE—–

How to upgrade PHP to PHP 5.4 on CentOS 6 from a remi repository

Parallels Article ID: 115875

1. Install epel and remi repositories:
# wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
# wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm
# sudo rpm -Uvh remi-release-6*.rpm epel-release-6*.rpm

2. Enable remi repository:
# sudo vi /etc/yum.repos.d/remi.repo
In the [remi] section of the file, set the “enabled” option to 1.

3. Upgrade PHP with this command:
# yum install php

4. Set remi section back to 0.

Enable Spamassassin for all existing mailboxes

In order to enable Spamassassin for all existing mailboxes you can use the following script:
#!/bin/sh

#Get the list of mailboxes registered in PPA

mysql -uadmin -p`cat /etc/psa/.psa.shadow` psa -sNe “select mail.mail_name, domains.name from mail left join domains on mail.dom_id=domains.id” | sed -E ‘s/[[:space:]]+/@/g’ > /tmp/mailboxes_list

#Enable spamassassin for mailboxes

while read mailbox
do
/usr/local/psa/bin/spamassassin -u $mailbox -status true
done < /tmp/mailboxes_list