Category Archives: els

Setup Packetbeat


title: “Setup Packetbeat”
date: 2024-02-08T14:26:14
slug: setup-packetbeat


Kurzinfo

Dieser Beitrag dokumentiert eine Beispielkonfiguration für Packetbeat. Unten stehen die relevanten Ausschnitte aus packetbeat.yml und ein paar Testkommandos.

Konfiguration (Ausschnitt)

# cat /etc/packetbeat/packetbeat.yml | grep -v '#' | grep -v '^$'
packetbeat.interfaces.device: any
packetbeat.interfaces.poll\\_default\\_route: 1m
packetbeat.interfaces.internal\\_networks:
- private
packetbeat.flows:
timeout: 30s
period: 10s
packetbeat.protocols:
- type: icmp
enabled: false
- type: amqp
- type: cassandra
- type: dhcpv4
- type: dns
ports: [53]
- type: http
ports: [80, 8080, 8000, 5000, 8002]
- type: memcache
ports: [11211]
- type: mysql
ports: [3306, 3307]
- type: pgsql
ports: [5432]
- type: redis
ports: [6379]
- type: thrift
ports: [9090]
- type: mongodb
ports: [27017]
- type: nfs
ports: [2049]
- type: tls
ports:
- 8443
- type: sip
ports: [5060]
setup.template.settings:
index.number\\_of\\_shards: 1
setup.dashboards.enabled: true
setup.kibana:
host: "http://192.168.178.195:5601"
output.elasticsearch:
hosts: ["192.168.178.195:9200"]
preset: balanced
processors:
- add\\_host\\_metadata: ~
- add\\_cloud\\_metadata: ~
- add\\_docker\\_metadata: ~
- detect\\_mime\\_type:
field: http.request.body.content
target: http.request.mime\\_type
- detect\\_mime\\_type:
field: http.response.body.content
target: http.response.mime\\_type

Tests

packetbeat test config
packetbeat test output
packetbeat setup

Elastic Opendistro Certs


title: “Elastic Opendistro Certs”
date: 2021-04-26T20:27:35
slug: elastic-opendistro-certs


/usr/share/elasticsearch/config

root-ca.pem
Common Name: Example Com Inc. Root CA
Organization: Example Com Inc.
Organization Unit: Example Com Inc. Root CA
Valid From: April 21, 2018
Valid To: April 18, 2028
Issuer: Example Com Inc. Root CA, Example Com Inc.
Serial Number: 1 (0x1)

esnode.pem
Common Name: node-0.example.com
Subject Alternative Names: Registered ID:1.2.3.4.5.5, node-0.example.com, localhost, IP Address:127.0.0.1
Organization: node
Organization Unit: node
Locality: test
Valid From: April 21, 2018
Valid To: April 18, 2028
Issuer: Example Com Inc. Root CA, Example Com Inc.
Serial Number: 1524368626614 (0x162eb7353b6)

kirk.pem (Client)
Common Name: kirk
Organization: client
Organization Unit: client
Locality: test
Country: de
Valid From: April 21, 2018
Valid To: April 18, 2028
Issuer: Example Com Inc. Root CA, Example Com Inc.
Serial Number: 1524368626616 (0x162eb7353b8)

Create Client Cert

openssl req -nodes -new -newkey rsa:2048 -sha256 -keyout client.key -out client.csr
openssl x509 -req -in client.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -out client.crt -days 3650 -sha256

curl -vvvk --cert client.crt --key client.key --cacert root-ca.pem https://elastic.apps.thasanger-dev-pop.noris.de/

oc create configmap elasticsearch-tls --from-file=root-ca.pem=root-ca.pem --from-file=root-ca-key.pem=root-ca-key.pem --from-file=elastic.crt=elastic.crt --from-file=elastic.key=elastic.key --from-file=client.crt=client.crt --from-file=client.key=client.key -oyaml --dry-run

Top 10 IP count


title: “Top 10 IP count”
date: 2019-02-13T10:10:02
slug: top-10-ip-count


POST /filebeat-2019.02.12/\_search?size=0
{
 "aggs" : {
 "hits" : {
 "terms" : { "field" : "haproxy.client\_ip"
 }
 }
 }
}

Modify Setting from an index


title: “Modify Setting from an index”
date: 2018-11-12T09:50:57
slug: modify-setting-from-an-index


PUT filebeat-2018.11.02/\_settings
{
 "index": {
 "routing": {
 "allocation": {
 "include": {
 "\_name": ""
 },
 "exclude": {
 "\_name": ""
 },
 "require": {
 "\_name": ""
 }
 }
 }
 }
}

Move a shard to another node


title: “Move a shard to another node”
date: 2018-11-12T09:29:35
slug: move-a-shard-to-another-node


curl -X POST "localhost:9200/\_cluster/reroute" -H 'Content-Type: application/json' -d'
{
 "commands" : [
 {
 "move" : {
 "index" : "filebeat-2018.11.03", "shard" : 0,
 "from\_node" : "elasticsearch01.mgm.muc01.fti.int", "to\_node" : "elasticsearch02.mgm.muc01.fti.int"
 }
 },
 {
 "allocate\_replica" : {
 "index" : "filebeat-2018.11.03", "shard" : 1,
 "node" : "elasticsearch03.mgm.muc01.fti.int"
 }
 }
 ]
}
'

joda-time


title: “joda-time”
date: 2018-10-09T09:34:51
slug: joda-time


The pattern letters are:

Symbol Meaning Presentation Examples
------ ------- ------------ -------
G era text AD
C century of era (>=0) number 20
Y year of era (>=0) year 1996
x weekyear year 1996
w week of weekyear number 27
e day of week number 2
E day of week text Tuesday; Tue
y year year 1996
D day of year number 189
M month of year month July; Jul; 07
d day of month number 10
a halfday of day text PM
K hour of halfday (0~11) number 0
h clockhour of halfday (1~12) number 12
H hour of day (0~23) number 0
k clockhour of day (1~24) number 24
m minute of hour number 30
s second of minute number 55
S fraction of second number 978
z time zone text Pacific Standard Time; PST
Z time zone offset/id zone -0800; -08:00; America/Los\_Angeles
' escape for text delimiter
'' single quote literal '

The count of pattern letters determine the format.

30-elasticsearch-output.conf


title: “30-elasticsearch-output.conf”
date: 2018-10-09T09:28:27
slug: 30-elasticsearch-output-conf


output {
 elasticsearch {
 hosts => ["https://eb843037.qb0x.com:32563/"]
 user => "ec18487808b6908009d3"
 password => "efcec6a1e0"
 index => "apache-%{+YYYY.MM.dd}"
 document\_type => "apache\_logs"
 }
 stdout { codec => rubydebug }
}

02-apache-input.conf


title: “02-apache-input.conf”
date: 2018-10-09T09:27:32
slug: 02-apache-input-conf


input {
 file {
 path => ["/var/log/apache2/access.log"]
 type => "apache\_access"
 }
 file {
 path => ["/var/log/apache2/error.log"]
 type => "apache\_error"
 }
}