Nginx Rate Limit + Fail2ban

最近遇到L7的攻擊行為 不斷的刷API，做一下筆記

1.使用iptables 限制單一IP連線速率
2.定義API訪問速率
3.使用Fail2ban封鎖

1.使用iptables 限制單一IP連線速率
iptables 限制syn 連線速率

#iptables -A INPUT -p tcp --syn -m connlimit --connlimit-above 35 -j DROP

iptables 限制http 連線速率

#iptables -A INPUT -p tcp --dport 80 -m connlimit --connlimit-above 35 -j DROP

iptables 限制https 連線速率

#iptables -A INPUT -p tcp --dport 443 -m connlimit --connlimit-above 35 -j DROP

2.定義API訪問速率
Nginx Rate Limit 使用了漏桶演算法，當訪問請求速率過快時，這些請求直接丟棄不做處理
以下是針對rate limit的參數做一些說明

參數	說明
limit_req_zone	定義速率限制的參數
Key	定義請求特徵 舉例:$binary_remote_addr (這個變數比remote_addr還省空間)
Zone	定義存儲每個IP地址狀態記憶體及訪問請求限制URL頻率 (官方說明 1M 可以儲存16000 IPs，如果記憶體不夠，則會回應503)
Rate	定義每一個Request最大的速率 舉例:每一秒最多只有能5個request
brust	定義可讓使用者超過rate設定的最大速率
nodelay	定義速率限制請求是否要被延遲

舉例:
每一個來源IP 每秒最多只有20個request，記憶體空間10M
limit_req_zone $binary_remote_addr zone=reqZone1:10m rate=20r/s;
每一個來源IP 每分鐘最多只有10個request，記憶體空間20M
limit_req_zone $binary_remote_addr zone=reqZone2:20m rate=10r/m;

把reqZone1定義在/api URL上

location /api {
    limit_req zone=reqZone1 burst=10 nodelay;
    proxy_pass http://$zuul;
}

把reqZone2定義在/api/v1/result/service/login上

location ^/api/v1/result/service/login {
    limit_req zone=reqZone2 burst=10 nodelay;
    proxy_pass http://$zuul;
}

2.使用Fail2ban封鎖
安裝Fail2ban

#yum -y install epel-release

#yum -y install fail2ban*

設定Filter

#vi /etc/fail2ban/filter.d/nginx-req-limit.conf

#設定reqZone1

# Fail2Ban configuration file
#
# supports: ngx_http_limit_req_module module

[Definition]
failregex = limiting requests, excess:.* by reqZone1.*client: <HOST>
ignoreregex =

#vi /etc/fail2ban/filter.d/nginx-login.conf

#設定reqZone2

# Fail2Ban configuration file
#
# supports: ngx_http_limit_req_module module

[Definition]
failregex = limiting requests, excess:.* by reqZone2.*client: <HOST>
ignoreregex =

設定jail.local

#vi /etc/fail2ban/jail.local

[nginx-req-limit]

#reqZone1
enabled = true
backend = auto
filter = nginx-req-limit
banaction = iptables-multiport[name=nginx-req-limit, port="http,https", protocol=tcp]
logpath = /var/log/nginx/error_nginx.log
findtime = 60
bantime = 1800
maxretry = 5

[nginx-login]

#reqZone2
enabled = true
backend = auto
filter = nginx-login
banaction = iptables-multiport[name=nginx-req-limit, port="http,https", protocol=tcp]
logpath = /var/log/nginx/error_nginx.log
findtime = 180
bantime = 14400
maxretry = 2

可透過 fail2ban-regex 來得知Filter.d內的過濾規則是否正確

#fail2ban-regex /var/log/nginx/error_nginx.log /etc/fail2ban/filter.d/nginx-login.conf

Running tests
=============

Use   failregex filter file : nginx-login, basedir: /etc/fail2ban
Use         log file : /var/log/nginx/error_nginx.log
Use         encoding : UTF-8


Results
=======

Failregex: 2510 total
|-  #) [# of hits] regular expression
|   1) [2510] limiting requests, excess:.* by zone.*reqZone2.*client: 
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [3700] Year(?P<_sep>[-/.])Month(?P=_sep)Day 24hour:Minute:Second(?:,Microseconds)?
`-

Lines: 3724 lines, 0 ignored, 2510 matched, 1214 missed #得知errorlog上有3724行，總共有2510比對到，剩下1214沒有比對到
[processed in 0.45 sec]

Missed line(s): too many to print.  Use --print-all-missed to print all 1214 lines

若沒有問題，則可以啟動服務

#systemctl start fail2ban

如果啟動有問題可以透過debug mode 確認

#/usr/bin/fail2ban-client -v -v start

查看Fail2ban 所有狀態

#fail2ban-client status

Status
|- Number of jail:	2
`- Jail list: nginx-login, nginx-req-limit

針對Fail2ban nginx-login 狀態

#fail2ban-client status nginx-login

Status for the jail: nginx-login
|- Filter
|  |- Currently failed:	0
|  |- Total failed:	29311
|  `- File list:	/var/log/nginx/error_nginx.log
`- Actions
   |- Currently banned:	1170
   |- Total banned:	33811
   `- Banned IP list:	******略過*****

查詢誤判的IP

#grep '223.104.239.220' /var/log/fail2ban.log

2019-04-02 15:56:06,470 fail2ban.filter [10899]: INFO [nginx-req-limit] Found 223.104.239.220
2019-04-02 15:56:06,471 fail2ban.filter [10899]: INFO [nginx-login] Found 223.104.239.220
2019-04-02 15:56:06,674 fail2ban.filter [10899]: INFO [nginx-req-limit] Found 223.104.239.220
2019-04-02 15:56:06,675 fail2ban.filter [10899]: INFO [nginx-login] Found 223.104.239.220
2019-04-02 15:56:07,086 fail2ban.actions [10899]: NOTICE [nginx-login] Ban 223.104.239.220