New user's registration have been closed due to high spamming and low trafic on this forum. Please contact forum admins directly if you need an account. Thanks !

SSH brute force attack prevention

Good ideas? Share with us!
mcg
Posts: 36
Joined: 13 Nov 2009, 22:34

SSH brute force attack prevention

Post by mcg »

Hey folks,

A couple of days ago I started getting some unsuccessful login attempts logged into auth.log that eventually blossomed into a full-blown brute-force attack on my SSH port. I noticed it because it was slowing down my DHCP access. Here's just a snippet of what that looks like.

Dec 20 14:29:24 bubba sshd[15412]: Invalid user desktop from 123.134.66.5
Dec 20 14:29:26 bubba sshd[15414]: Invalid user workshop from 123.134.66.5
Dec 20 14:29:29 bubba sshd[15416]: Invalid user mailnull from 123.134.66.5
Dec 20 14:29:31 bubba sshd[15418]: Invalid user nfsnobody from 123.134.66.5
Dec 20 14:29:34 bubba sshd[15420]: Invalid user rpcuser from 123.134.66.5
Dec 20 14:29:36 bubba sshd[15422]: Invalid user rpc from 123.134.66.5

I'm still deciding how to handle this. I can't turn off my external SSH access permanently, and I'd rather not turn off password access within the LAN. So I'm investigating a couple of alternatives:

1) Use separate SSH daemons for each ethernet interface:
http://www.kudos.be/multiple_sshd
This allows me to disable password access over the WAN, but not the LAN. I'll do this for now, I think, but I'd really rather not shut off password access permanently, just in case I'd like to log in from someone else's machine and I don't have my private key handy.

2) Modify iptables to better protect against these kinds of attacks. There are a variety of solutions available on the net. My favorite choices involve setting up a "tarpit" to tie up the hackers' daemons, but I do not know if the kernel that ships with the Bubba supports it. Here's a source:
http://www.la-samhna.de/library/brutessh.html
Of course, the iptables solution can be modified to apply only to eth0.

The problem I have, however, is that the Bubba modifies its iptables configuration from its web interface. What I'd really like to see is one of these basic protection rulesets integrated into that script by default.
asparak
Posts: 173
Joined: 08 Jun 2009, 07:38

Re: SSH brute force attack prevention

Post by asparak »

Not sure if it will solve your issue, but I never run ssh on port 22, for just this reason.

iptables is probably the right route but its difficult with the current Excito implementation. Hopefully this will get addressed at some point in the new year.
mcg
Posts: 36
Joined: 13 Nov 2009, 22:34

Re: SSH brute force attack prevention

Post by mcg »

That's certainly a feasible option for me, although somewhat inconvenient, so I'd probably use it only for the WAN connection coupled with the two-sshd-instance solution I mentioned earlier.
carl
Posts: 474
Joined: 07 May 2008, 04:41

Re: SSH brute force attack prevention

Post by carl »

Actually I my self have been subject to such attack. As i have an prototype BubbaTwo as my router at home with an older more noisier disk, I could actually hear it as it wrote to access lock each second.

What I did to prevent this was to install the package "fail2ban" which will add rules to iptables after some failed attempts. Has had some problem with it though (see later in this post), and the fact it isn't compatible with our network configuration system, is reason I've not written an howto.

But if you feel you really want to try this up, a short howto here for you daredevils there is:):
  • install the package fail2ban (from backports, located in upstream_etch_forclaire)
  • in /etc/fail2ban/jail.conf enable ssh, pam-generic, proftpd, and sasl.
  • in /etc/fail2ban/filter.d/proftpd.conf change the failregex as the shipped doesn't work:

    Code: Select all

    failregex = \(\S+\[<HOST>\]\)[: -]+ USER \S+: no such user found from \S+ \[\S+\] to \S+:\S+
                \(\S+\[<HOST>\]\)[: -]+ USER \S+ \(Login failed\): Incorrect password\.$
                \(\S+\[<HOST>\]\)[: -]+ SECURITY VIOLATION: \S+ login attempted\.$
                \(\S+\[<HOST>\]\)[: -]+ Maximum login attempts \(\d+\) exceeded$
    
I might have missed something, but you should notice in both /var/log/auth.log and /var/log/fail2ban.log if it works (I had some friendly Chinese boxes testing my setup for me)

It did work rather fine for me, but occasionally I get following error:

Code: Select all

2009-12-22 15:55:54,407 fail2ban.actions.action: ERROR  iptables -n -L INPUT | grep -q fail2ban-ssh returned 100
2009-12-22 15:55:54,410 fail2ban.actions.action: ERROR  Invariant check failed. Trying to restore a sane environment
2009-12-22 15:55:54,597 fail2ban.actions.action: ERROR  iptables -D INPUT -p tcp -m multiport --dports ssh -j fail2ban-ssh
I never dug into this much, and the kicking noise was gone :)
but an "/etc/init.d/fail2ban restart" will resolve it though (probably related to the fact our web interface isn't compatible, and will remove all the fail2ban rules).

iptables-save should show following:

Code: Select all

:fail2ban-pam-generic - [0:0]
:fail2ban-proftpd - [0:0]
:fail2ban-sasl - [0:0]
:fail2ban-ssh - [0:0]
-A INPUT -p tcp -m multiport --dports 21,20,990,989 -j fail2ban-proftpd 
-A INPUT -p tcp -m multiport --dports 25,465,143,220,993,110,995 -j fail2ban-sasl 
-A INPUT -p tcp -m multiport --dports 22 -j fail2ban-ssh 
-A INPUT -p tcp -j fail2ban-pam-generic 
And a normal functional session would be noticed like this:
auth.log:

Code: Select all

Dec 20 06:41:30 bubba sshd[23306]: Invalid user mailman from 220.67.126.11
Dec 20 06:41:33 bubba sshd[23308]: Invalid user mailman from 220.67.126.11
Dec 20 06:41:44 bubba sshd[23316]: Invalid user bgz from 220.67.126.11
Dec 20 06:41:47 bubba sshd[23318]: Invalid user bgz from 220.67.126.11
Dec 20 06:41:50 bubba sshd[23320]: Invalid user bgz from 220.67.126.11
Dec 20 06:41:52 bubba sshd[23322]: Invalid user bgz from 220.67.126.11
fail2ban.log

Code: Select all

2009-12-20 06:41:53,955 fail2ban.actions: WARNING [ssh] Ban 220.67.126.11
2009-12-20 06:47:12,476 fail2ban.filter : INFO   Log rotation detected for /var/log/auth.log
2009-12-20 06:51:54,235 fail2ban.actions: WARNING [ssh] Unban 220.67.126.11
i.e. he was banned for 10 minutes (configurable though, but as you can fail yourself, don't choose a too long time :))
If you like this one, perhaps we can work out some easier solution later.
/Carl Fürstenberg, Excito Software Developer
http://www.excito.com
support@excito.com
mcg
Posts: 36
Joined: 13 Nov 2009, 22:34

Re: SSH brute force attack prevention

Post by mcg »

Thanks, Carl. I've gone ahead and tried this. I needed only the SSH support for now, really; I don't use the FTP server or the Email server. Now to wait for those Chinese to try again :)
Puma
Posts: 230
Joined: 29 Sep 2008, 06:30

Re: SSH brute force attack prevention

Post by Puma »

Carl,

This post triggert me to see my Auth.log and my acces.log

Yes they tried me as well!!!

For now I blocked my ssh from wan acces port 22
But I see in my acces.log allso:

67.18.244.106 - - [21/Dec/2009:18:03:46 +0100] "GET /user/soapCaller.bs HTTP/1.1" 404 375 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:46 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:46 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:46 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:47 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:47 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:47 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:47 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:48 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:48 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:48 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:49 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:49 +0100] "GET soapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"
67.18.244.106 - - [21/Dec/2009:18:03:49 +0100] "GET \x97\x04\b\x1a\x97\x04\b*\x97\x04\b:\x97\x04\bJ\x97\x04\bZ\x97\x04\bj\x97\x04\bz\x97\x04\b\x8a\x97\x04\b\x9a\x97\x04\b\xaa\x97\x04\b\xba\x97\x04\b\xca\x97\x04\b\xda\x97\x04\b\xea\x97\x04\b\xfa\x97\x04\bsoapCaller.bs HTTP/1.1" 400 385 "-" "Morfeus Fucking Scanner"

In my auth.log I see lots of hits from all over the world. I hate this.

What can I do about this?

Will there be a configurable web interface to handle this?

Perhaps a table to give only a few IP adresses acces to web / ssh etc.??

Thank you for your how to but i'm not sure if i'm brave enough to do this. IPTable I can change.
Maybe you have some good advice.

Thanks in advance.

Puma
Linux is like a wigwam - no windows, no gates, apache inside!
mcg
Posts: 36
Joined: 13 Nov 2009, 22:34

Re: SSH brute force attack prevention

Post by mcg »

I've installed fail2ban, and sure enough it has successfully thwarted one attack already. Interestingly this one was from a Comcast circuit in Connecticut, not China, but who knows who actually *controls* that bot...

Puma, fail2ban is pretty general. All it does is monitor the logs you specify for the appearance of certain patterns. Each log file can be given a different set of patterns. So, for instance, you could specify the pattern "Morfeus Fucking Scanner" as a "failure". In fact, you don't even need to give that pattern a "grace period". For SSH, you generally allow a certain number of failures to account for mis-typed passwords and other innocent mistakes. But for "Morfeus Fucking Scanner"? I think you can ban that the first time, and leave it banned :)
mcg
Posts: 36
Joined: 13 Nov 2009, 22:34

Re: SSH brute force attack prevention

Post by mcg »

I will say, however, that I don't think fail2ban is a good choice for a Bubba release. The risk of error in dynamically modifying iptables doesn't seem consistent with the desire for stability and consistency in a Bubba server.

In my view, a static set of iptables rules, such as those suggested in the links I've posted above, would be a better choice for SSH attack prevention. It will be easy to add such rules to the existing Bubba web configurator.
Puma
Posts: 230
Joined: 29 Sep 2008, 06:30

Re: SSH brute force attack prevention

Post by Puma »

Thanks for your reply,

Do you think I can just ad the following to the IPTABLES

iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set --name SSH -j ACCEPT

iptables -A INPUT -p tcp --dport 22 -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j LOG --log-prefix "SSH_brute_force "

iptables -A INPUT -p tcp --dport 22 -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j DROP

Will this work properly until EXCITO has solved this with a whitelist eg in the webinterface?

Puma
Linux is like a wigwam - no windows, no gates, apache inside!
mcg
Posts: 36
Joined: 13 Nov 2009, 22:34

Re: SSH brute force attack prevention

Post by mcg »

I think I'm going to try something like this soon. I'm getting the same fail2ban errors that Carl was getting. I don't want to have to periodically reset fail2ban. To me, there's no practical difference between banning IPs with multiple *failures* versus banning IPs with multiple *attempts*. After all, once I get my public keys shipped around, I will only have one or two attempts at a time.
mcg
Posts: 36
Joined: 13 Nov 2009, 22:34

Re: SSH brute force attack prevention

Post by mcg »

OK, this is what I've done to get things working.

First, I went ahead and added a rule right at the top of my INPUT and FORWARD lists to drop INVALID packets:

Code: Select all

-A INPUT -m state --state INVALID -j DROP
This, combined with the rule that accepts all RELATED and ESTABLISHED packets, means that I can be sure that all packets that make it to my ssh rules are NEW. So there's no need to check their state.

Code: Select all

-A INPUT -p tcp -m tcp --dport 22 -m recent --set --name SSH
-A INPUT -p tcp -m tcp --dport 22 -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j LOG --log-prefix "SSH Attack: "
-A INPUT -p tcp -m tcp --dport 22 -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j DROP
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
I drew the first three lines from one of the resources I found on the net: basically, it allows up to three connections in one minute, then it logs and drops the fourth and subsequent connections.

What is new is the last line: since our default action is DROP, I have to ACCEPT any ssh packet that makes it past the first three rules. If you don't do that, you'll never get through.

Anyway, I know this could be cleaned up, but it works for now. One thing in particular that I want to do is slow down the logging. I don't want to log EVERY failed attempt. For now I'm just going to drop logging altogether, but there are some resources out there that show you how to log only a fraction of the attempts.
Puma
Posts: 230
Joined: 29 Sep 2008, 06:30

Re: SSH brute force attack prevention

Post by Puma »

mcg,

Thanks for your input.

I opend port 22 this morning (to get acces myself) and again cracking/hacking began immediately.
See logfile below.

What striked me is that probarbly one and the same hacker changes his IP number every time..... which is bad for a IPtable solution, i think.

IP 92.65.85.105 is from my work.

Maybe it is better (safer) to use a whitelist IP (home, work, friends etc) that can access and otherwise drop the line.

Carl, do you have a solution? Will there be a solution in the next update?

Puma


Dec 24 07:41:46 bubba sshd[6076]: reverse mapping checking getaddrinfo for static.kpn.net failed - POSSIBLE BREAK-IN ATTEMPT!
Dec 24 07:41:50 bubba sshd[6076]: Accepted keyboard-interactive/pam for hdroogers from 92.65.85.105 port 33929 ssh2
Dec 24 07:41:50 bubba sshd[6079]: (pam_unix) session opened for user puma by (uid=0)
Dec 24 07:45:01 bubba CRON[6090]: (pam_unix) session opened for user root by (uid=0)
Dec 24 07:45:08 bubba CRON[6090]: (pam_unix) session closed for user root
Dec 24 07:47:52 bubba sshd[6103]: Invalid user elvis from 62.77.221.54
Dec 24 07:47:52 bubba sshd[6105]: (pam_unix) check pass; user unknown
Dec 24 07:47:52 bubba sshd[6105]: (pam_unix) authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=mail.atkft.hu
Dec 24 07:47:53 bubba sshd[6103]: error: PAM: User not known to the underlying authentication module for illegal user elvis from mail.atkft.hu
Dec 24 07:47:53 bubba sshd[6103]: Failed keyboard-interactive/pam for invalid user elvis from 62.77.221.54 port 49323 ssh2
Dec 24 07:50:01 bubba CRON[6109]: (pam_unix) session opened for user root by (uid=0)
Dec 24 07:50:08 bubba CRON[6109]: (pam_unix) session closed for user root
Dec 24 07:55:01 bubba CRON[6113]: (pam_unix) session opened for user root by (uid=0)
Dec 24 07:55:08 bubba CRON[6113]: (pam_unix) session closed for user root
Dec 24 08:00:01 bubba CRON[6117]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:00:07 bubba CRON[6117]: (pam_unix) session closed for user root
Dec 24 08:03:48 bubba sshd[6121]: Invalid user elvis from 58.247.222.163
Dec 24 08:03:48 bubba sshd[6123]: (pam_unix) check pass; user unknown
Dec 24 08:03:48 bubba sshd[6123]: (pam_unix) authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=58.247.222.163
Dec 24 08:03:50 bubba sshd[6121]: error: PAM: User not known to the underlying authentication module for illegal user elvis from 58.247.222.163
Dec 24 08:03:50 bubba sshd[6121]: Failed keyboard-interactive/pam for invalid user elvis from 58.247.222.163 port 57080 ssh2
Dec 24 08:05:01 bubba CRON[6124]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:05:08 bubba CRON[6124]: (pam_unix) session closed for user root
Dec 24 08:09:01 bubba CRON[6128]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:09:01 bubba CRON[6128]: (pam_unix) session closed for user root
Dec 24 08:10:01 bubba CRON[6141]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:10:08 bubba CRON[6141]: (pam_unix) session closed for user root
Dec 24 08:15:01 bubba CRON[6145]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:15:08 bubba CRON[6145]: (pam_unix) session closed for user root
Dec 24 08:17:01 bubba CRON[6154]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:17:01 bubba CRON[6154]: (pam_unix) session closed for user root
Dec 24 08:20:01 bubba CRON[6157]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:20:07 bubba CRON[6157]: (pam_unix) session closed for user root
Dec 24 08:25:02 bubba CRON[6161]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:25:08 bubba CRON[6161]: (pam_unix) session closed for user root
Dec 24 08:27:28 bubba sshd[6165]: Invalid user eric from 84.246.69.21
Dec 24 08:27:29 bubba sshd[6167]: (pam_unix) check pass; user unknown
Dec 24 08:27:29 bubba sshd[6167]: (pam_unix) authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=84.246.69.21
Dec 24 08:27:30 bubba sshd[6165]: error: PAM: User not known to the underlying authentication module for illegal user eric from 84.246.69.21
Dec 24 08:27:30 bubba sshd[6165]: Failed keyboard-interactive/pam for invalid user eric from 84.246.69.21 port 49600 ssh2
Dec 24 08:30:01 bubba CRON[6224]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:30:08 bubba CRON[6224]: (pam_unix) session closed for user root
Dec 24 08:35:01 bubba CRON[6228]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:35:07 bubba CRON[6228]: (pam_unix) session closed for user root
Dec 24 08:35:35 bubba sshd[6232]: Invalid user eric from 190.146.246.36
Dec 24 08:35:35 bubba sshd[6232]: reverse mapping checking getaddrinfo for static-ip-cr19014624636.cable.net.co failed - POSSIBLE BREAK-IN ATTEMPT!
Dec 24 08:35:36 bubba sshd[6234]: (pam_unix) check pass; user unknown
Dec 24 08:35:36 bubba sshd[6234]: (pam_unix) authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=190.146.246.36
Dec 24 08:35:38 bubba sshd[6232]: error: PAM: User not known to the underlying authentication module for illegal user eric from 190.146.246.36
Dec 24 08:35:38 bubba sshd[6232]: Failed keyboard-interactive/pam for invalid user eric from 190.146.246.36 port 19049 ssh2
Dec 24 08:39:01 bubba CRON[6235]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:39:01 bubba CRON[6235]: (pam_unix) session closed for user root
Dec 24 08:40:01 bubba CRON[6248]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:40:08 bubba CRON[6248]: (pam_unix) session closed for user root
Dec 24 08:43:25 bubba sshd[6252]: Invalid user eric from 148.233.140.193
Dec 24 08:43:25 bubba sshd[6254]: (pam_unix) check pass; user unknown
Dec 24 08:43:25 bubba sshd[6254]: (pam_unix) authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=sgd4.accelsa.com.mx
Dec 24 08:43:27 bubba sshd[6252]: error: PAM: User not known to the underlying authentication module for illegal user eric from sgd4.accelsa.com.mx
Dec 24 08:43:27 bubba sshd[6252]: Failed keyboard-interactive/pam for invalid user eric from 148.233.140.193 port 51604 ssh2
Dec 24 08:45:01 bubba CRON[6256]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:45:08 bubba CRON[6256]: (pam_unix) session closed for user root
Dec 24 08:50:01 bubba CRON[6304]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:50:07 bubba CRON[6304]: (pam_unix) session closed for user root
Dec 24 08:55:01 bubba CRON[6314]: (pam_unix) session opened for user root by (uid=0)
Dec 24 08:55:08 bubba CRON[6314]: (pam_unix) session closed for user root
Dec 24 08:59:15 bubba sshd[6330]: Invalid user eric from 220.162.241.11
Dec 24 08:59:16 bubba sshd[6332]: (pam_unix) check pass; user unknown
Dec 24 08:59:16 bubba sshd[6332]: (pam_unix) authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=220.162.241.11
Dec 24 08:59:17 bubba sshd[6330]: error: PAM: User not known to the underlying authentication module for illegal user eric from 220.162.241.11
Dec 24 08:59:17 bubba sshd[6330]: Failed keyboard-interactive/pam for invalid user eric from 220.162.241.11 port 22403 ssh2
Linux is like a wigwam - no windows, no gates, apache inside!
mcg
Posts: 36
Joined: 13 Nov 2009, 22:34

Re: SSH brute force attack prevention

Post by mcg »

Hmm, what I'm seeing in your logs isn't that bad: one failed attempt every 10-15 minutes or so. I wouldn't worry about that. What I'm trying to prevent is several failed attempts *per second*.

That said, a whitelist is certainly doable, as long as the IP addresses you're likely to log in from is fixed. My problem is that I don't have a fixed IP list.
novis
Posts: 30
Joined: 12 Sep 2008, 09:36
Location: Suomi Finland

Re: SSH brute force attack prevention

Post by novis »

Has anyone tried sshblack with Bubba2?
The sshblack script is a real-time security tool for secure shell (ssh). It monitors *nix log files for suspicious activity and reacts appropriately to aggressive attackers by adding them to a "blacklist" created using various firewalling tools -- such as iptables -- available in most modern versions of Unix and Linux. The blacklist is simply a list of source IP addresses that are prohibited from making ssh connections to the protected host. Once a predetermined amount of time has passed, the offending IP address is removed from the blacklist.
mcg
Posts: 36
Joined: 13 Nov 2009, 22:34

Re: SSH brute force attack prevention

Post by mcg »

Hadn't seen that before, but its functionality closely resembles that of fail2ban, which is already in the Debian package list. I preferred to stick with a static iptables-based solution for my application, and that has been working well for me so far.
Post Reply