# Better responses to criminal scan bots than 404?



## obsigna (Oct 14, 2021)

I run a BLog with static pages, i.e. without WordPress and without PHP served by Apache24 on FreeBSD 13. My logs are filled with tons of requests which obviously belong to web scans for finding flaws in WordPress and other CMS installations. I know, we cannot do much against this, other than either avoid these systems (my case) or keep’m updated in a timely manner.

Anyway, I experimented with a few measures (beside the default 404 one) and would like to ask for more ideas.

Redirecting (301) to a Zero-Bomb. Clients which are capable of compression receive a file of 1 MB pre-gzipped zeros, which would expand to 1 GB on the receivers side - factor 1000


Redirecting (301) to a file with zero content


Redirecting (301) to https://127.0.0.1/
The first measure started to become expensive also for my system, and I am even not sure, whether the other side expands the zeros. The second measure seemed to be somewhat effective, because the number of the respective requests were reduced significantly within a few days. Although this reduction might be coincidence, chances are that the criminal bots follow 301 responses, and therefore I switched to the 3rd method now.

So, any more ideas to check for?


----------



## covacat (Oct 14, 2021)

tarpit ?


----------



## jbo (Oct 14, 2021)

I wonder whether you could perform a "reverse" slowloris attack on the bot. Assuming you have a system to identify these bots you could just reduce the bandwidth drastically (eg. to a few bytes per second) which will either eat up their resources (keeps them from doing something else) or they might just give up at one point and remove you from their "list of interests".

I can be completely wrong here. Just came to my mind while reading this.


----------



## obsigna (Oct 14, 2021)

covacat said:


> tarpit ?


By all means, I avoid automatic processing of personal data of 2nd and 3rd parties. That leaves me in the comfortable situation that I am not subject of any respective directives. By definition of the not less criminal privacy industry, IP addresses are personal data. So, tarpit is not an option.


----------



## sko (Oct 14, 2021)

I'm using https://github.com/mariusv/nginx-badbot-blocker on our company webserver and it drastically reduced the amount of bot traffic. We had peaks of 20-30MBit of traffic over several hours during a period of high bot activity and 90+% of traffic was bot-induced. This has come down to <30% and those extremely enduring bot-attacks where hundrets or thousands of bots repeatedly try to access CMS-specific, PhpMySql or various admin-panel related backend stuff that doesn't even exist are gone.

I was also planning on feeding a list of IPs or prefixes of those bots to the firewall, but as this problem has come down to a minimum I didn't see any reason in spending much time on that...


----------



## obsigna (Oct 14, 2021)

jbodenmann said:


> I wonder whether you could perform a "reverse" slowloris attack on the bot. Assuming you have a system to identify these bots you could just reduce the bandwidth drastically (eg. to a few bytes per second) which will either eat up their resources (keeps them from doing something else) or they might just give up at one point and remove you from their "list of interests".
> 
> I can be completely wrong here. Just came to my mind while reading this.


I guess, this would be very nice, if a substantial number of services suffering scan attacks would implement something like this. A single server would perhaps not do much harm.


----------



## jbo (Oct 14, 2021)

obsigna said:


> I guess, this would be very nice, if a substantial number of services suffering scan attacks would implement something like this. A single server would perhaps not do much harm.


I hope that your primary goal is not to cause any harm. It's like with phone scams: Just waste their time.


----------



## obsigna (Oct 14, 2021)

jbodenmann said:


> I hope that your primary goal is not to cause any harm. It's like with phone scams: Just waste their time.


Indeed, the goal is not to cause harm, like the goal of a bee is not to cause harm, but being left alone by simply showing the instruments. Pehaps, I should have said _„... does not hurt enough for being effective“_.


----------



## Jose (Oct 14, 2021)

jbodenmann said:


> I wonder whether you could perform a "reverse" slowloris attack on the bot.


This is exactly what a tarpit is . I use spamd(8) to tarpit known spammers. This increases their costs and reduces the amount of SPAM they can send to legitimate mail servers. I do wish there was something similar for HTTP. It's sometimes also called a "stuttering daemon".


----------



## SirDice (Oct 14, 2021)

Keep in mind that most of these 'attacks' are done from unsuspecting people running a server that got hacked (usually by the same malware that's trying to break into _your_ system). You're never really causing any problems for the 'herders' of those botnets. They don't care if one of their bots is taking ages on your system, they've got 20000 (or a LOT more) others that finish the job for them. Just kill the attacks quickly, and move on. Post a few abuse emails to the ISP of notorious 'attackers' and it'll eventually die down a little bit but it'll flare up a couple months later. These guys have a never ending supply of bots to use, you're never going to make any noticeable impression on them.


----------



## Jose (Oct 14, 2021)

And I'm totally fine with causing problems for these "unsuspecting" people. Maybe they'll become more suspicious and fix their broken stuff already. There's a basic level of politeness that's expected if you're going to run a server on the Internet. Having a poorly maintained box that spews crapware is the equivalent of urinating in the swimming pool. You will be ejected if I'm the lifeguard.


----------



## dd_ff_bb (Oct 14, 2021)

Jose said:


> urinating in the swimming pool



 That thought drives me crazy.

That's why i get by the pool minute its open (after lifeguard cleans the pool, put chlorine etc..), try to finish my laps and leave before anyone gets in.

When i was a kid i used to believe there was a chemical in the pool which turns bright red if somebody urinates. I wish it was true.


----------



## Argentum (Oct 14, 2021)

obsigna said:


> I run a BLog with static pages, i.e. without WordPress and without PHP served by Apache24 on FreeBSD 13. My logs are filled with tons of requests which obviously belong to web scans for finding flaws in WordPress and other CMS installations. I know, we cannot do much against this, other than either avoid these systems (my case) or keep’m updated in a timely manner.


`ipfw add deny ip from  x.y.z.0/24 to any`

ipfw(8)


----------



## obsigna (Oct 14, 2021)

SirDice said:


> Keep in mind that most of these 'attacks' are done from unsuspecting people running a server that got hacked (usually by the same malware that's trying to break into _your_ system). You're never really causing any problems for the 'herders' of those botnets. They don't care if one of their bots is taking ages on your system, they've got 20000 (or a LOT more) others that finish the job for them. Just kill the attacks quickly, and move on. Post a few abuse emails to the ISP of notorious 'attackers' and it'll eventually die down a little bit but it'll flare up a couple months later. These guys have a never ending supply of bots to use, you're never going to make any noticeable impression on them.


Yes, agreed. And actually I try to reduce the impact on my service(s) without spending too much effort at the same time. So, from this point of view, serving an empty file was not that a bad idea. Actually a HTTP 200 response with content-length: 0. Now I think, I improved it a little bit by  returning a HTTP 301 response with content-length: 1.

In the virtual host configuration I have now:

```
...
ErrorDocument 301 "."
RedirectMatch 301 "(.*/index\.php|.*/wp-login\.php|.*/wordpress/|.*/wp/|.*/wp-admin/|.*/wp-includes/|.*/wp-content/|.*/console/|.*/_ignition/|.*/Autodiscover/)" "https://127.0.0.1/"
...
```

`curl -i https://obsigna.com/index.php`

```
HTTP/1.1 301 Moved Permanently
Date: Thu, 14 Oct 2021 18:15:24 GMT
Server: Apache
Location: https://127.0.0.1/
Content-Length: 1
Content-Type: text/html; charset=iso-8859-1

.
```

Argentum, because of the GDPR/EU and the similar one LGBD/BR, I do not log IP addresses for some time now, however, from the past I know that these scan bots rarely come in from recurring IP’s, only a train of scans at the same time comes from the same IP. Therefore, adding these IP’s to the firewall would be an almost fruitless effort.


----------



## Tieks (Oct 14, 2021)

obsigna said:


> I know that these scan bots rarely come in from recurring IP’s


That is my experience too. The few ones that show up on a regular basis may be eligible for a friendly e-mail to their ISP (find out with `whois <ip>`). In terms of efficiency, 404 is likely the best answer in most cases.

However, I think I identified another group that is mostly active during school holidays. Especially on rainy days. If I see such a hacker looking for a file that doesn't exist, I might create it containing a redirect to, say,  www.fbi.gov. Or run an nmap scan on all their lower ports. And yes, I keep this limited to ip's from my own country.


----------



## kjpetrie (Oct 17, 2021)

I've thought about this too. I also have sites hackers assume will be WordPress or the like and try to log into, and it is a real nuisance having logs spammed by such useless requests. However, there really isn't much to do about it other than a 404 or similar response.

The problem is, the machines running these bots will mostly not even have sys admins or even users with a clue about security, and in future more and more of them will be smart TVs, light switches, fridges, washing machines, etc, all sold with IoT connections to phone apps allowing remote control, and opening them up to becoming bots in someone's collection. The world is entering a very frightening phase of consumer devices capable of all sorts of misuse with no one giving a thought to the security implications until a country is shut down by some malicious actor using such bots to sabotage its essential systems.


----------



## obsigna (Oct 18, 2021)

kjpetrie said:


> I've thought about this too. I also have sites hackers assume will be WordPress or the like and try to log into, and it is a real nuisance having logs spammed by such useless requests. However, there really isn't much to do about it other than a 404 or similar response.
> 
> The problem is, the machines running these bots will mostly not even have sys admins or even users with a clue about security, and in future more and more of them will be smart TVs, light switches, fridges, washing machines, etc, all sold with IoT connections to phone apps allowing remote control, and opening them up to becoming bots in someone's collection. The world is entering a very frightening phase of consumer devices capable of all sorts of misuse with no one giving a thought to the security implications until a country is shut down by some malicious actor using such bots to sabotage its essential systems.


This is all clear, and again, 301 to https://127.0.0.1/ seems to be beneficial but at least it is not worse than the simple 404.
Apparently 301 is honoured by the bots, this was a result from me experimenting with 301 to a zero length file. The bots fetched the file as was clearly visible in the logs. In addition this would be a reasonable behaviour for a bot, because there are many real WordPress and other CMS installations in the web which had been moved to a different location and the purpose of these bots is to find them all.

If nothing else, 301 to https://127.0.0.1/ keeps my logs clean, and this does not impose more load to the server than a 404.


----------

