Goals

1. v7.0 -> v8.2 (auto security enable)

disable default ssl proxy pass by nginx
enable default ssl use default ca created by ‘ElasticSearch auto ‘ crs
enable default ssl generate diffrent ca ( elastchsearch <https> kibana <https> fleetserver ))

2. filebeats -> fleet-server (server<->agent->elasticsearch)

Dashboard

https://kibana.intbird.net/

https://kibana.intbird.net/app/integrations/detail/nginx-1.3.1/assets

https://kibana.intbird.net/app/integrations/detail/system-1.11.0/assets

auto default certificates

1: certs/http_ca.crt

Elastic Auto CA

2: certs/http.p12

current Node Api Used(kibana & other api clients)

elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password

3: certs/transport.p12

elasticsearch-keystore show xpack.security.transport.ssl.keystore.secure_password

elasticsearch-keystore show xpack.security.transport.ssl.truststore.secure_password

custom-certificate

nginx-log:

https://docs.nginx.com/nginx/admin-guide/monitoring/logging/

add https domain name ($servername) to nginx log

log_format compression '$remote_addr - $remote_user [$time_local]'
                    '"$host" "$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent" "$gzip_ratio"';

system-log

prevent policies: iptables + sshd

elasticsearch add custom nginx log field

Stack Management
Ingest Pipelines
Edit pipeline
Manage processor
Grok
Test if match the new nginx access.log
Update the agent-pliocy & nginx-integration

在nginx原始信息的Grok解析中添加后续添加的字段，或者append到数据流发送到 fleetserver

log_format compression  '$remote_addr - $remote_user [$time_local] '
                        '"$request" $status $body_bytes_sent '
                        '"$http_referer" "$http_user_agent" '
			'"$server_name"';

docs:
$ip - - [04/Feb/2023:04:46:57 -0500] "GET /api/v1/xxx HTTP/1.1" 200 9726 "https://$host/$path" "Mozilla/5.0xxx" "$domain"

grok:
"%{DATA:https.domain}"

save to nginx-log-temp

Before

after

iptables: https://wiki.archlinux.org/title/iptables

iptables -A INPUT -p tcp -s 45.0.0.0/8 -j DROP
iptables -D INPUT -p tcp -s 45.0.0.0/8 -j DROP

iptables-docker: https://docs.docker.com/network/iptables/

iptables -I DOCKER-USER -s 45.0.0.0/8 -j DROP
iptables -D DOCKER-USER -s 45.0.0.0/8 -j DROP

fail2ban: https://www.fail2ban.org/wiki/index.php/Main_Page

array=(
*.*.*.*
x.x.x.x
)

for element in ${array[@]}
do
#echo $element
iptables -I INPUT -s $element -j DROP
done


#iptables -D INPUT -s $element -j DROP
iptables -L -n --line-numbers
for index in  num in {x..x}  
do
iptables -D INPUT $index
done

sshd_config: https://linux.die.net/man/8/sshd

Port x
PermitRootLogin prohibit-password 
PasswordAuthentication no
PubkeyAuthentication yes

AllowUsers root@x.x.x.x

default user

default user: elastic

user passwd: ./bin/elasticsearch-reset-password -u elastic -i ( user interactive password)

and more: ls /bin/

default https

https://www.elastic.co/guide/en/kibana/current/docker.html

https://www.elastic.co/guide/en/fleet/master/secure-connections.html

不使用域名的话：自动配置127.0.0.1 和 172.*.0.* ip的自签名证书

version: '2'
services:
  kibana:
    image: docker.elastic.co/kibana/kibana:8.2.0
    #environment:
    #  SERVER_NAME: kibana.example.org
    volumes:
      - ./kibana.yml:/usr/share/kibana/config/kibana.yml

add fleet-agent vs filebeats

https://www.elastic.co/guide/en/fleet/master/secure-connections.html

fleet-agent-server

https://kibana.intbird.net/app/fleet/agents

·我也不太想用443的正式域名，所以自签名一个fleeturl+port的域名·

Quick start : fleet-server-http

sudo ./elastic-agent install  \
  --fleet-server-es=https://publicipOrdomain:9200 \
  --fleet-server-service-token=xxxxxx \
  --fleet-server-es-ca-trusted-fingerprint=xxxxx
  #test: --fleet-server-insecure-http

Production : fleet-server-https

./elastic-agent install --url=https://publicipOrdomain:8220 \
  --fleet-server-es=https://x.x.x.x:9200 \
  --fleet-server-service-token=xxx \
  --fleet-server-policy=fleet-server-policy \
  --fleet-server-es-ca-trusted-fingerprint=xxx \
  --certificate-authorities=/opt/Elastic/ssl/fleet/chain.crt \
  --fleet-server-cert=/opt/Elastic/ssl/fleet/publicipOrdomain_chain.crt  \
  --fleet-server-cert-key=/opt/Elastic/ssl/fleet/publicipOrdomain_key.key

Production: fleet-agent-https:

./elastic-agent install --url=https://publicipOrdomain:8220/ \
--enrollment-token=xxxxx \
--certificate-authorities= /opt/Elastic/ssl/fleet/chain.crt

install success but wait for a long time…

Successfully enrolled the Elastic Agent.
Elastic Agent has been successfully installed.

install package

/usr/bin/elastic-agent uninstall
rm -rf /opt/Elastic/Agent/


curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.2.0-linux-x86_64.tar.gz
tar xzvf elastic-agent-8.2.0-linux-x86_64.tar.gz
cd elastic-agent-8.2.0-linux-x86_64


/opt/Elastic/Agent
/opt/Elastic/install
/opt/Elastic/ssl

agent-panel: https://kibana.intbird.net/app/fleet/policies

add intergration

intergration system:

intergration nginx:

fleet-server:

https://kibana.intbird.net/app/dashboards#/view/nginx-046212a0-a2a1-11e7-928f-5dbe6f6f5519?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))

https://kibana.intbird.net/app/integrations/detail/nginx-1.3.1/overview

filebeats:

https://www.elastic.co/guide/en/beats/filebeat/current/index.html

https://www.elastic.co/guide/en/beats/filebeat/current/running-on-docker.html

https://www.elastic.co/blog/how-to-monitor-nginx-web-servers-with-the-elastic-stack

https://www.nginx.com/blog/monitoring-nginx-plus-statistics-elk/

interation caddy(watting for test…):

log:

output net 127.0.0.1:8220 { dial_timeout <duration> }

Certs Location

https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html

elasticsearch: es:/usr/share/elasticsearch/config/certs

https://www.elastic.co/guide/en/fleet/master/secure-connections.html

fleet-server: /opt/Elastic/Agent/fleet.yml

iptables

-t<表>：指定要操纵的表；
-A：向规则链中添加条目；
-D：从规则链中删除条目；
-i：向规则链中插入条目；
-R：替换规则链中的条目；
-L：显示规则链中已有的条目；
-F：清楚规则链中已有的条目；
-Z：清空规则链中的数据包计算器和字节计数器；
-N：创建新的用户自定义规则链；
-P：定义规则链中的默认目标；
-h：显示帮助信息；
-p：指定要匹配的数据包协议类型；
-s：指定要匹配的数据包源ip地址；
-j<目标>：指定要跳转的目标；
-i<网络接口>：指定数据包进入本机的网络接口；
-o<网络接口>：指定数据包要离开本机所使用的网络接口。

1.表名包括：
raw：高级功能，如：网址过滤。
mangle：数据包修改（QOS），用于实现服务质量。
net：地址转换，用于网关路由器。
filter：包过滤，用于防火墙规则。

2.规则链名包括：
INPUT链：处理输入数据包。
OUTPUT链：处理输出数据包。
PORWARD链：处理转发数据包。
PREROUTING链：用于目标地址转换（DNAT）。
POSTOUTING链：用于源地址转换（SNAT）。

3.动作包括：
accept：接收数据包。
DROP：丢弃数据包。
REDIRECT：重定向、映射、透明代理。
SNAT：源地址转换。
DNAT：目标地址转换。
MASQUERADE：IP伪装（NAT），用于ADSL。
LOG：日志记录。


iptables -A INPUT -s 127.0.0.1 -d 127.0.0.1 -j ACCEPT 
iptables -A INPUT -m state -–state ESTABLISHED,RELATED -j ACCEPT 
iptables -A OUTPUT -j ACCEPT
iptables -A INPUT -p tcp -–dport 22 -j ACCEPT
iptables -A INPUT -p tcp -–dport 80 -j ACCEPT
iptables -A INPUT -p tcp -–dport 443 -j ACCEPT
iptables -A INPUT -j reject
iptables -A FORWARD -j REJECT

Upgrade Elastic & Kibana & FleetServer security

Goals

1. v7.0 -> v8.2 (auto security enable)

2. filebeats -> fleet-server (server<->agent->elasticsearch)

Dashboard

https://kibana.intbird.net/

https://kibana.intbird.net/app/integrations/detail/nginx-1.3.1/assets

https://kibana.intbird.net/app/integrations/detail/system-1.11.0/assets

auto default certificates

custom-certificate

nginx-log:

system-log

elasticsearch add custom nginx log field

save to nginx-log-temp

Before

after

iptables: https://wiki.archlinux.org/title/iptables

sshd_config: https://linux.die.net/man/8/sshd

default user

default https

add fleet-agent vs filebeats

fleet-agent-server

add intergration

intergration system:

intergration nginx:

interation caddy(watting for test…):

Certs Location

iptables

发表回复取消回复

一个有意思的开发问题

NodeJs RWfile

nginx stream : mysql , mongo , redis

Glide load resize by Drawable

Goals

1. v7.0 -> v8.2 (auto security enable)

2. filebeats -> fleet-server (server<->agent->elasticsearch)

Dashboard

https://kibana.intbird.net/

https://kibana.intbird.net/app/integrations/detail/nginx-1.3.1/assets

https://kibana.intbird.net/app/integrations/detail/system-1.11.0/assets

auto default certificates

custom-certificate

nginx-log:

system-log

elasticsearch add custom nginx log field

save to nginx-log-temp

Before

after

iptables: https://wiki.archlinux.org/title/iptables

sshd_config: https://linux.die.net/man/8/sshd

default user

default https

add fleet-agent vs filebeats

fleet-agent-server

add intergration

intergration system:

intergration nginx:

interation caddy(watting for test…):

Certs Location

iptables

发表回复 取消回复

一个有意思的开发问题

NodeJs RWfile

nginx stream : mysql , mongo , redis

Glide load resize by Drawable

发表回复取消回复