Our server suddenly got big traffic and this traffic detected on Google Analytics.
This spike traffic increase 1000 % – 2000 %. Our server resource very high, and CPU usage about 80 – 90%, usually server load average 10-20%.
On Google Analytics, this high traffic come from Hong Kong and China, even our site is targeted in other specific country.
We found on access log traffic from this user agent:
"GET /sensored HTTP/1.1" 403 177 "-" "Mozilla/5.0 (Linux; Android 7.0; FRD-AL00 Build/HUAWEIFRD-AL00; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/53.0.2785.49 Mobile MQQBrowser/6.2 TBS/043602 Safari/537.36 MicroMessenger/6.5.16.1120 NetType/WIFI Language/zh_CN"
"GET /sensored HTTP/1.1" 200 31 "-" "Mozilla/5.0(Linux;Android 5.1.1;OPPO A33 Build/LMY47V;wv) AppleWebKit/537.36(KHTML,link Gecko) Version/4.0 Chrome/43.0.2357.121 Mobile Safari/537.36 LieBaoFast/4.51.3"
"GET /sensored HTTP/1.1" 403 177 "-" "Mozilla/5.0(Linux;Android 5.1.1;OPPO A33 Build/LMY47V;wv) AppleWebKit/537.36(KHTML,link Gecko) Version/4.0 Chrome/42.0.2311.138 Mobile Safari/537.36 Mb2345Browser/9.0"The request is very high and massive, so our server like under DDOS attack.
We don’t think that our site will have visitor from this country, because our site is specific products and targeted to country in South America.
Block LieBaoFast, MQQBrowser and Mb2345Browser using Modsecurity
Our Nginx has module modsecurity, and we try to add LieBaoFast, MQQBrowser and Mb2345Browser on blocklist user agent.
Then we test it with:
curl -A "Mozilla/5.0 (Linux; Android 7.0; FRD-AL00 Build/HUAWEIFRD-AL00; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/53.0.2785.49 Mobile MQQBrowser/6.2 TBS/043602 Safari/537.36 MicroMessenger/6.5.16.1120 NetType/WIFI Language/zh_CN" -I "https://serverdiary.com"The result is:
HTTP/2 403
server: nginx
date: Fri, 24 Jan 2020 13:21:18 GMT
content-type: text/html
content-length: 548
vary: Accept-EncodingBut when we check Nginx access log, they still can has access and still got 200. Request still exists on Google Analytics real time.
We don’t know why they can by pass modsecurity, even if we try the result is 403 or forbidden.
Block LieBaoFast, MQQBrowser and Mb2345Browser using Nginx configuration
Second option is block those user agent using Nginx configuration.
Nginx if is Evil, read more on Nginx website that they does not recommended using If.
But if you want to use it, below is Nginx configuration to block user agent LieBaoFast, MQQBrowser and Mb2345Browser, put this on server block.
if ($http_user_agent ~* (liebaofast|mqqbrowser|mb2345browser|zh-cn|zh_cn|micromessenger) ) {
return 403;
}
Or you can use Nginx map to block bad bots
## Write your bad User agents want to block
map $http_user_agent $bad_bots {
default 0;
~*scrapyproject 1;
~*nmap 1;
~*sqlmap 1;
~*slowhttptest 1;
~*nikto 1;
~*magpie-crawler 1;
~*python-requests 1;
~*redback 1;
~*liebaofast 1;
~*mqqbrowser 1;
~*micromessenger 1;
~*zh-cn 1;
~*zh_cn 1;
}
if ($bad_bots) {
return 403;
}Above Nginx configuration is not optimal. Our Nginx is still got DDOS request, process it and deny request, and don’t forget that Nginx if is not recommended.
Block LieBaoFast, MQQBrowser and Mb2345Browser using Fail2ban
We think best option is using File2ban. Our nginx does not got request from this Chinese bad bots.
# yum install epel-release
# yum install fail2ban fail2ban-systemdCreate /etc/fail2ban/filter.d/nginx-badbots.conf and paste the following configuration.
[Definition]
mybadbots = Mb2345Browser|LieBaoFast|MQQBrowser|micromessenger|zh_CN|zh-CN|SeznamBot|trendictionbot|magpie-crawler
failregex = ^<HOST> .*(GET|POST|HEAD).*(%(mybadbots)s).*$
ignoreregex =
datepattern = ^[^\[]*\[({DATE})
{^LN-BEG}Copy /etc/fail2ban/jail.conf to /etc/fail2ban/jail.local
# cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.localAdd this code at the bottom of /etc/fail2ban/jail.local
#CUSTOM
#[DEFAULT]
# Ban hosts for one hour:
bantime = 3600
# Override /etc/fail2ban/jail.d/00-firewalld.conf:
banaction = iptables-multiport
[nginx-badbots]
enabled = true
port = http,https
filter = nginx-badbots
logpath = /var/log/nginx/*.access.log
findtime = 43200
maxretry = 1
bantime = 86400
action = iptables-multiport[name=BadBots, port="http,https"]Enable auto start Fail2ban and start service
# systemctl enable fail2ban
# systemctl start fail2banCheck log on /var/log/fail2ban.log
# tail -f /var/log/fail2ban.log
Example output
2020-01-24 15:07:10,869 fail2ban.actions [11121]: NOTICE [nginx-badbots] Ban 27.209.165.186
2020-01-24 15:07:10,936 fail2ban.actions [11121]: NOTICE [nginx-badbots] Ban 60.168.87.136
2020-01-24 15:07:11,002 fail2ban.actions [11121]: NOTICE [nginx-badbots] Ban 27.38.49.133
2020-01-24 15:07:11,213 fail2ban.filter [11121]: INFO [nginx-badbots] Found 122.232.219.148 - 2020-01-24 15:07:11
2020-01-24 15:07:11,250 fail2ban.filter [11121]: INFO [nginx-badbots] Found 223.98.64.36 - 2020-01-24 15:07:11
2020-01-24 15:07:11,264 fail2ban.actions [11121]: NOTICE [nginx-badbots] Ban 122.232.219.148
2020-01-24 15:07:11,327 fail2ban.actions [11121]: NOTICE [nginx-badbots] Ban 223.98.64.36Check IP blocked by Fail2ban with command:
# fail2ban-client status nginx-badbots
example output
Status for the jail: nginx-badbots
|- Filter
| |- Currently failed: 0
| |- Total failed: 28146
