When running a server in the cloud, it is a good idea to at least secure SSH so that brute forcing attempts will automatically be blocked after a set number of incorrect tries. Having set up
fail2ban with
firewalld in Archlinux on a laptop, I figured the process would be pretty much the same for CentOS 7 in a Digital Ocean droplet.
As soon as you login into your new Droplet, I recommend that you follow the steps in
this DO article about creating a regular user account, enabling sudo, disabling root logins over SSH, etc. You should also read the
follow-up article about how to set up
firewalld in CentOS 7 (TL;DR
sudo systemctl enable firewalld,
sudo systemctl start firewalld)
Now you need to install
fail2ban. Although it is not in the default CentOS
yum repositories, it is available through EPEL which is included in the CentOS Extras repository (which is enabled by default).
sudo yum install epel-release
Starting from RHEL/CentOS 7,
iptables and
rsyslog have been replaced with
firewalld and
journalctl (part of
systemd), so you must be sure to
yum install the following packages:
fail2ban-systemd (to ensure journalctl compatibility)
fail2ban-firewalld
ipset (to ban IP's with firewalld)
fail2ban-server
Now
cd into
/etc/fail2ban where you will see the systemwide
fail2ban config file
jail.conf which you can use as a reference but should not edit directly. Instead, in the subdir
/etc/fail2ban/jail.d/ you should make your own config file ending with the
.conf extension (I named mine
local.conf). You will also notice that in this sub-directory there will be two other .conf files named
00-firewalld.conf and
00-systemd.conf which contain the following:
[centjun@juncent7 jail.d]$ cat 00-firewalld.conf
# This file is part of the fail2ban-firewalld package to configure the use of
# the firewalld actions as the default actions. You can remove this package
# (along with the empty fail2ban meta-package) if you do not use firewalld
[DEFAULT]
banaction = firewallcmd-ipset
[centjun@juncent7 jail.d]$ cat 00-systemd.conf
# This file is part of the fail2ban-systemd package to configure the use of
# the systemd journal as the default backend. You can remove this package
# (along with the empty fail2ban meta-package) if you do not want to use the
# journal backend
[DEFAULT]
backend=systemd
In Archlinux, these config files are not included in the
jail.d subdir, so I had to add
banaction=... and
backend=... in
local.conf (in the case of Archlinux, not CentOS7).
The
/etc/fail2ban/jail.d/local.conf which I used in Archlinux is as follows:
[DEFAULT]
bantime = 18000
banaction = firewallcmd-ipset
backend = systemd
sender = fail2ban@example.com
destemail = root
ignoreip = 127.0.0.1 192.168.0.0/16
use_dns = no
maxretry = 21
action = %(action_mwl)s
[sshd]
enabled = true
However when I used the settings above in CentOS 7 and then tried to start the
fail2ban systemd service with
sudo systemctl start fail2ban I got the following errors in
journalctl:
Jul 04 14:36:19 juncent7 fail2ban-client[1674]: ERROR Found no accessible config files for 'action.d/sendmail-whois-lines' under /e...ail2ban
Jul 04 14:36:19 juncent7 fail2ban-client[1674]: ERROR Error in action definition sendmail-whois-lines[name=sshd, dest="centjun", lo...INPUT"]
Jul 04 14:36:19 juncent7 fail2ban-client[1674]: ERROR Errors in jail 'sshd'. Skipping...
Since
fail2ban uses
sendmail as its default mta, I checked to see if
sendmail was installed:
[centjun@juncent7 jail.d]$ rpm -q sendmail
package sendmail is not installed
Instead, I found that
postfix was installed, so I then added
mta = postfix to local.conf, but still got similar errors:
Jul 04 14:45:13 juncent7 fail2ban-client[1715]: ERROR Found no accessible config files for 'action.d/postfix-whois-lines' under /etc/fail2ban
Jul 04 14:45:13 juncent7 fail2ban-client[1715]: ERROR Error in action definition postfix-whois-lines[name=sshd, dest="centjun", log...INPUT"]
Jul 04 14:45:13 juncent7 fail2ban-client[1715]: ERROR Errors in jail 'sshd'. Skipping...
I checked
/etc/fail2ban/action.d/ for the files
postfix-whois-lines and
sendmail-whois-lines but both of the them were nowhere to be found.
Finally I edited the default action in
local.conf to just ban the offending IP instead of sending a mail to root:
action = $(action_)s
Now when starting
fail2ban.service with
systemctl start fail2ban, everything works fine:
[centjun@juncent7 jail.d]$ systemctl status fail2ban
fail2ban.service - Fail2Ban Service
Loaded: loaded (/usr/lib/systemd/system/fail2ban.service; enabled)
Active: active (running) since Sat 2015-07-04 15:08:04 SGT; 44min ago
Docs: man:fail2ban(1)
Process: 1803 ExecStart=/usr/bin/fail2ban-client -x start (code=exited, status=0/SUCCESS)
Main PID: 1806 (fail2ban-server)
CGroup: /system.slice/fail2ban.service
└─1806 /usr/bin/python -Es /usr/bin/fail2ban-server -s /var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail2ban.pid -x -b
[centjun@juncent7 jail.d]$ sudo fail2ban-client status
[sudo] password for centjun:
Status
|- Number of jail: 1
`- Jail list: sshd
Just in case you think it is too much hassle to do minimal server hardening, take a look at just a few of the dozens of unauthorized login attempts recorded by PAM in
journalctl:
Jul 04 15:21:42 juncent7 sshd[1828]: reverse mapping checking getaddrinfo for 107.30.65.218.broad.xy.jx.dynamic.163data.com.cn [218...ATTEMPT!
Jul 04 15:21:43 juncent7 sshd[1828]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.65...ser=root
Jul 04 15:21:43 juncent7 sshd[1828]: pam_succeed_if(sshd:auth): requirement "uid >= 1000" not met by user "root"
Jul 04 15:21:45 juncent7 sshd[1828]: Failed password for root from 218.65.30.107 port 57804 ssh2
Jul 04 15:21:45 juncent7 sshd[1828]: pam_succeed_if(sshd:auth): requirement "uid >= 1000" not met by user "root"
Jul 04 15:21:47 juncent7 sshd[1828]: Failed password for root from 218.65.30.107 port 57804 ssh2
Jul 04 15:21:49 juncent7 sshd[1828]: pam_succeed_if(sshd:auth): requirement "uid >= 1000" not met by user "root"
Jul 04 15:21:51 juncent7 sshd[1828]: Failed password for root from 218.65.30.107 port 57804 ssh2
Jul 04 15:21:54 juncent7 sshd[1828]: Received disconnect from 218.65.30.107: 11: [preauth]
Jul 04 15:21:54 juncent7 sshd[1828]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.65.30.107 user=root
Jul 04 15:21:55 juncent7 sshd[1830]: reverse mapping checking getaddrinfo for 107.30.65.218.broad.xy.jx.dynamic.163data.com.cn [218...ATTEMPT!
Jul 04 15:21:55 juncent7 sshd[1830]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=218.65...ser=root
Jul 04 15:21:55 juncent7 sshd[1830]: pam_succeed_if(sshd:auth): requirement "uid >= 1000" not met by user "root"
Jul 04 15:21:58 juncent7 sshd[1830]: Failed password for root from 218.65.30.107 port 53038 ssh2
Jul 04 15:22:03 juncent7 sshd[1830]: pam_succeed_if(sshd:auth): requirement "uid >= 1000" not met by user "root"
Jul 04 15:22:04 juncent7 sshd[1830]: Failed password for root from 218.65.30.107 port 53038 ssh2
Jul 04 15:22:05 juncent7 sshd[1830]: pam_succeed_if(sshd:auth): requirement "uid >= 1000" not met by user "root"
Jul 04 15:22:06 juncent7 sshd[1830]: Failed password for root from 218.65.30.107 port 53038 ssh2
Jul 04 15:22:07 juncent7 sshd[1830]: Received disconnect from 218.65.30.107: 11: [preauth]
...
When I switched to
root user PAM informed me of a large number of failed login attempts on the root account:
[centjun@juncent7 jail.d]$ su -
Password:
Last login: Fri Jul 3 23:08:03 SGT 2015 from xxx.xxx.xxx.xxx (redacted) on pts/0
Last failed login: Sat Jul 4 15:22:42 SGT 2015 from 218.65.30.107 on ssh:notty
There were 65 failed login attempts since the last successful login.
This large number of unsuccessful logins occurred within just
1 hour of launching my new droplet!
When I ran a reverse DNS on one of the IP's associated with a failed login, it appeared to be
an address from China.
Comments on certain settings in /etc/fail2ban/jail.d/local.conf
ignoreip =
It is a good idea to put your remote IP (the IP of the computer you are connecting from) into
ignoreip above (multiple IP's can be entered in the whitelist, separated with spaces) to avoid getting locked out of your cloud server by an attacker sending packets with a spoofed source header (of course the attacker would have to know your remote IP address).
use_dns = no
It is safer not to use dns to ban hostnames because an attacker can change the PTR of an IP that they control to point to another hostname that could make fail2ban to mistakenly ban valid domains (
http://www.fail2ban.org/wiki/index.php/Hostnames_or_IP_Addresses).
Thoughts about Digital Ocean Droplets
I was pleasantly surprised with my first experience using Digital Ocean (DO) cloud instances. The only other cloud server I have used is AWS EC2 (free usage tier) and I think DO compares favorably. Although DO is missing the "enterprisey" features of AWS, I think DO is much easier to use and better-suited for quickly spinning up a server instance in the cloud. Whereas AWS micro instances only give you 8GB of storage and 1 CPU, DO's smallest instance gives you 1 CPU with 30 GB of SSD!
For me, DO's killer feature is the ability to take a snapshot of an instance (once you have shut it down), delete the Droplet, and then later create a new Droplet from the snapshot you created earlier. The reason this is so awesome is that DO snapshots are free (unlike those on AWS EC2, which take up gigabytes of Elastic Storage) and allow you to use cloud instances on an hourly basis if you are so inclined. Here are some good forum posts about using DO on an hourly basis:
https://www.digitalocean.com/community/questions/pricing-monthly-or-hourly
https://www.digitalocean.com/community/questions/restore-snapshot-after-destroying-a-droplet
On AWS EC2, however, snapshots are destroyed along with an image, so you couldn't easily replicate this DO feature over there.
References:
https://fedoraproject.org/wiki/Fail2ban_with_FirewallD
Fedora wiki about setting up
fail2ban with
firewalld; also explains the meaning of variables in
fail2ban config files
https://wiki.gentoo.org/wiki/Fail2ban#Actions
Gentoo wiki about
fail2ban
