beenull/crowdsec
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
crowdsec/docs/getting_started/crowdsec-tour.md
erenJag 4f9d252a15
Add the documentation into crowdsec repo (#3) 
Add the documentation into crowdsec repo
2020-05-15 16:58:24 +02:00

12 KiB
Raw Blame History

Finding configurations

As stated, {{crowdsec.name}} efficiency is dictated by installed parsers and scenarios.

Please take a look at the {{hub.name}} to find the appropriated ones !

List installed configurations

List installed parsers/scenarios/collections/enricher

{{cli.bin}} list

On the machine where you deployed {{crowdsec.name}}, type {{cli.bin}} list to see deployed configurations. This list represents the parsers, scenarios and/or collections that you deployed. They represent what your {{crowdsec.name}} setup can read (logs) and detect (scenarios).

Check {{cli.name}} configuration management for more !

output example ```bash bui@sd:~$ {{cli.bin}} list INFO[0000] Loaded 9 collecs, 14 parsers, 12 scenarios, 1 post-overflow parsers INFO[0000] PARSERS: -------------------------------------------------------------------------------------------------------------------- NAME 📦 STATUS VERSION LOCAL PATH -------------------------------------------------------------------------------------------------------------------- crowdsec/nginx-logs ✔️ enabled 0.3 /etc/crowdsec/config/parsers/s01-parse/nginx-logs.yaml crowdsec/sshd-logs ✔️ enabled 0.3 /etc/crowdsec/config/parsers/s01-parse/sshd-logs.yaml crowdsec/syslog-logs ✔️ enabled 0.4 /etc/crowdsec/config/parsers/s00-raw/syslog-logs.yaml crowdsec/whitelists ✔️ enabled 0.4 /etc/crowdsec/config/parsers/s02-enrich/whitelists.yaml crowdsec/dateparse-enrich ✔️ enabled 0.4 /etc/crowdsec/config/parsers/s02-enrich/dateparse-enrich.yaml crowdsec/iptables-logs ✔️ enabled 0.3 /etc/crowdsec/config/parsers/s01-parse/iptables-logs.yaml crowdsec/naxsi-logs ✔️ enabled 0.4 /etc/crowdsec/config/parsers/s02-enrich/naxsi-logs.yaml crowdsec/http-logs ✔️ enabled 0.4 /etc/crowdsec/config/parsers/s02-enrich/http-logs.yaml crowdsec/geoip-enrich ✔️ enabled 0.4 /etc/crowdsec/config/parsers/s02-enrich/geoip-enrich.yaml -------------------------------------------------------------------------------------------------------------------- INFO[0000] SCENARIOS: ----------------------------------------------------------------------------------------------------------------------------- NAME 📦 STATUS VERSION LOCAL PATH ----------------------------------------------------------------------------------------------------------------------------- crowdsec/http-crawl-non_statics ✔️ enabled 0.4 /etc/crowdsec/config/scenarios/http-crawl-non_statics.yaml crowdsec/iptables-scan-multi_ports ✔️ enabled 0.4 /etc/crowdsec/config/scenarios/iptables-scan-multi_ports.yaml crowdsec/http-scan-uniques_404 ✔️ enabled 0.4 /etc/crowdsec/config/scenarios/http-scan-uniques_404.yaml crowdsec/ssh-bf ✔️ enabled 0.8 /etc/crowdsec/config/scenarios/ssh-bf.yaml ----------------------------------------------------------------------------------------------------------------------------- INFO[0000] COLLECTIONS: ------------------------------------------------------------------------------------------------------------------- NAME 📦 STATUS VERSION LOCAL PATH ------------------------------------------------------------------------------------------------------------------- crowdsec/base-http-scenarios ✔️ enabled 0.1 /etc/crowdsec/config/collections/base-http-scenarios.yaml crowdsec/iptables ✔️ enabled 0.2 /etc/crowdsec/config/collections/iptables.yaml crowdsec/nginx ✔️ enabled 0.2 /etc/crowdsec/config/collections/nginx.yaml crowdsec/sshd ✔️ enabled 0.2 /etc/crowdsec/config/collections/sshd.yaml crowdsec/linux ✔️ enabled 0.2 /etc/crowdsec/config/collections/linux.yaml ------------------------------------------------------------------------------------------------------------------- INFO[0000] POSTOVERFLOWS: -------------------------------------- NAME 📦 STATUS VERSION LOCAL PATH -------------------------------------- -------------------------------------- ```

List existing bans

List current bans

{{cli.bin}} ban list

On the machine where you deployed {{crowdsec.name}}, type {{cli.bin}} ban list to see existing bans. If you just deployed {{crowdsec.name}}, the list might be empty, but don't worry, it simply means you haven't yet been attacked, congrats!

Check {{cli.name}} ban management for more !

output example ```bash bui@sd:~$ {{cli.bin}} ban list 7 local decisions: +--------+----------------+--------------------------------+------+--------+---------+--------------------------------+--------+------------+ | SOURCE | IP | REASON | BANS | ACTION | COUNTRY | AS | EVENTS | EXPIRATION | +--------+----------------+--------------------------------+------+--------+---------+--------------------------------+--------+------------+ | local | 103.218.xxx.xx | crowdsecurity/ssh-bf | 4 | ban | HK | 59077 Shanghai UCloud | 24 | 3h28m24s | | | | | | | | Information Technology Company | | | | | | | | | | Limited | | | | local | 176.174.x.xx | crowdsecurity/ssh-bf | 11 | ban | FR | 5410 Bouygues Telecom SA | 66 | 2h48m6s | | local | 37.49.xxx.xxx | crowdsecurity/ssh-bf | 4 | ban | NL | 0 | 37 | 2h16m35s | | local | 37.49.xxx.xx | crowdsecurity/ssh-bf_user-enum | 5 | ban | NL | 0 | 59 | 2h16m21s | | local | 92.246.xx.xxx | crowdsecurity/ssh-bf_user-enum | 2 | ban | | 0 | 12 | 1h42m2s | | local | 23.237.x.xx | crowdsecurity/ssh-bf | 8 | ban | US | 174 Cogent Communications | 48 | 1h7m48s | | local | 185.153.xxx.xx | crowdsecurity/ssh-bf_user-enum | 59 | ban | MD | 49877 RM Engineering LLC | 449 | 12m54s | +--------+----------------+--------------------------------+------+--------+---------+--------------------------------+--------+------------+ And 64 records from API, 32 distinct AS, 19 distinct countries ```

There are different bans sources:

  • local : bans triggered locally
  • api : bans fetched from the API as part of the global consensus
  • cli : bans added via {{cli.bin}} ban add

Monitor on-going activity (prometheus)

List metrics

{{cli.bin}} metrics

The metrics displayed are extracted from {{crowdsec.name}} prometheus. The indicators are grouped by scope :

  • Buckets : Know which buckets are created and/or overflew (scenario efficiency)
  • Acquisition : Know which file produce logs and if thy are parsed (or end up in bucket)
  • Parser : Know how frequently the individual parsers are triggered and their success rate
output example 
bui@sd:~$ {{cli.bin}}  metrics
INFO[0000] Buckets Metrics:                             
+---------------------------------+-----------+--------------+--------+---------+
|             BUCKET              | OVERFLOWS | INSTANTIATED | POURED | EXPIRED |
+---------------------------------+-----------+--------------+--------+---------+
| crowdsec/http-scan-uniques_404  |        69 |           77 |    424 |       8 |
| crowdsec/ssh-bf                 |         4 |           23 |     53 |      18 |
| crowdsec/ssh-bf_user-enum       | -         |           21 |     23 |      20 |
| crowdsec/http-crawl-non_statics |         9 |           14 |    425 |       5 |
+---------------------------------+-----------+--------------+--------+---------+
INFO[0000] Acquisition Metrics:                         
+------------------------------------------+------------+--------------+----------------+------------------------+
|                  SOURCE                  | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+------------------------------------------+------------+--------------+----------------+------------------------+
| /var/log/nginx/error.log                 |        496 |          496 | -              | -                      |
| /var/log/nginx/http.access.log  |        472 |          465 |              7 |                    847 |
| /var/log/nginx/https.access.log |          1 |            1 | -              |                      2 |
| /var/log/auth.log                        |        357 |           53 |            304 |                     76 |
| /var/log/kern.log                        |       2292 | -            |           2292 | -                      |
| /var/log/syslog                          |       2358 | -            |           2358 | -                      |
+------------------------------------------+------------+--------------+----------------+------------------------+
INFO[0000] Parser Metrics:                              
+---------------------------+------+--------+----------+
|          PARSERS          | HITS | PARSED | UNPARSED |
+---------------------------+------+--------+----------+
| crowdsec/syslog-logs      | 5007 |   5007 |        0 |
| crowdsec/whitelists       | 1015 |   1015 |        0 |
| crowdsec/dateparse-enrich | 1015 |   1015 |        0 |
| crowdsec/geoip-enrich     |  519 |    519 |        0 |
| crowdsec/http-logs        |  962 |    427 |      535 |
| crowdsec/nginx-logs       |  973 |    962 |       11 |
| crowdsec/non-syslog       |  969 |    969 |        0 |
| crowdsec/sshd-logs        |  350 |     53 |      297 |
+---------------------------+------+--------+----------+

Monitor on-going activity (log files)

The {{crowdsec.main_log}} file will tell you what is going on and when an IP is blocked.

Check {{crowdsec.name}} monitoring for more !

output example 
bui@sd:~$ tail -f /var/log/crowdsec-agent.log 
time="14-04-2020 16:06:21" level=warning msg="40 existing LeakyRoutine"
time="14-04-2020 16:14:07" level=warning msg="1.2.3.4 triggered a 4h0m0s ip ban remediation for [crowdsec/ssh-bf]" bucket_id=throbbing-forest event_time="2020-04-14 16:14:07.215101505 +0200 CEST m=+359659.646220115" scenario=crowdsec/ssh-bf source_ip=1.2.3.4
time="14-04-2020 16:15:52" level=info msg="api push signal: token renewed. Pushing signals"
time="14-04-2020 16:15:53" level=info msg="api push signal: pushed 1 signals successfully"
time="14-04-2020 16:21:10" level=warning msg="18 existing LeakyRoutine"
time="14-04-2020 16:30:01" level=info msg="Flushed 1 expired entries from Ban Application"
time="14-04-2020 16:33:23" level=warning msg="33 existing LeakyRoutine"
time="14-04-2020 16:35:58" level=info msg="Flushed 1 expired entries from Ban Application"