Doc improvements (#644)
* fix #640 : add faq about metrics and 'is my setup working' * up documentation go routines, fix #637 * to merge along with https://github.com/crowdsecurity/hub/pull/160 to improve forensic's doc
This commit is contained in:
parent
4565d82f79
commit
da84805f5f
|
@ -53,13 +53,20 @@ SQLite by default as it's suitable for standalone/single-machine setups.
|
|||
|
||||
## How to control granularity of actions ? (whitelists, simulation etc.)
|
||||
|
||||
{{v1X.crowdsec.name}} support both [whitelists]((/Crowdsec/v1/write_configurations/whitelist/) and [simulation](/Crowdsec/v1/references/simulation/) :
|
||||
{{v1X.crowdsec.name}} support both [whitelists](/Crowdsec/v1/write_configurations/whitelist/) and [simulation](/Crowdsec/v1/references/simulation/) :
|
||||
|
||||
- Whitelists allows you to "discard" events or overflows
|
||||
- Simulation allows you to simply cancel the decision that is going to be taken, but keep track of it
|
||||
|
||||
{{v1X.profiles.htmlname}} allows you to control which decision will be applied to which alert.
|
||||
|
||||
## How to know if my setup is working correctly ? Some of my logs are unparsed, is it normal ?
|
||||
|
||||
Yes, crowdsec parsers only parse the logs that are relevant for scenarios :)
|
||||
|
||||
Take a look at `cscli metrics` [and understand what do they mean](/Crowdsec/v1/getting_started/crowdsec-tour/#reading-metrics) to know if your setup is correct.
|
||||
|
||||
|
||||
## How to add whitelists ?
|
||||
|
||||
You can follow this [guide](/Crowdsec/v1/write_configurations/whitelist/)
|
||||
|
|
|
@ -190,6 +190,26 @@ INFO[0000] Local Api Bouncers Metrics:
|
|||
|
||||
</details>
|
||||
|
||||
### Reading metrics
|
||||
|
||||
Those metrics are a great way to know if your configuration is correct :
|
||||
|
||||
The `Acquisition Metrics` is a great way to know if your parsers are setup correctly :
|
||||
|
||||
- If you have 0 **LINES PARSED** for a source : You are probably *missing* a parser, or you have a custom log format that prevents the parser from understanding your logs.
|
||||
- However, it's perfectly OK to have a lot of **LINES UNPARSED** : Crowdsec is not a SIEM, and only parses the logs that are relevant to its scenarios. For example, [ssh parser](https://hub.crowdsec.net/author/crowdsecurity/configurations/sshd-logs), only cares about failed authentication events (at the time of writting).
|
||||
- **LINES POURED TO BUCKET** tell you that your scenarios are matching your log sources : it means that some events from this log source made all their way to an actual scenario
|
||||
|
||||
|
||||
The `Parser Metrics` will let you troubleshoot eventual parser misconfigurations :
|
||||
- **HITS** is how many events where fed to this specific parser
|
||||
- **PARSED** and **UNPARSED** indicate how many events successfully come out of the parser
|
||||
|
||||
For example, if you have a custom log format in nginx that is not supported by the default parser, you will end up seeing a lot of **UNPARSED** for this specific parser, and 0 for **PARSED**.
|
||||
|
||||
For more advanced metrics understanding, [take a look at the dedicated prometheus documentation](/Crowdsec/v1/observability/prometheus/).
|
||||
|
||||
|
||||
## Deploy dashboard
|
||||
|
||||
```bash
|
||||
|
@ -217,3 +237,11 @@ sudo cscli collections install crowdsecurity/nginx
|
|||
Collections are bundles of parsers/scenarios that form a coherent ensemble to analyze/detect attacks for a specific service. It is the most common way to deploy configurations.
|
||||
|
||||
They can be found and browsed on the {{v1X.hub.htmlname}}
|
||||
|
||||
## Scalability
|
||||
|
||||
Crowdsec uses go-routines for parsing and enriching logs, pouring events to buckets and manage outputs.
|
||||
|
||||
By default, one routine of each exists (should be enough to handle ~1K EP/s), and can be changed in `crowdsec_service` of the main configuration file via the [parser_routines](/Crowdsec/v1/references/crowdsec-config/#parser_routines), [buckets_routines](/Crowdsec/v1/references/crowdsec-config/#buckets_routines) and [output_routines](/Crowdsec/v1/references/crowdsec-config/#output_routines) directives.
|
||||
|
||||
Please keep in mind that thanks to the [http API]({{v1X.lapi.swagger}}), the workload of log parsing can be splitted amongst several agents pushing to a single {{v1X.lapi.Htmlname}}.
|
||||
|
|
|
@ -35,6 +35,10 @@ INFO[13-11-2020 13:05:24] (14baeedafc1e44c08b806fc0c1cd92c4/crowdsec) crowdsecur
|
|||
And as these alerts are as well pushed to database, it mean you can view them in metabase, or using cscli !
|
||||
|
||||
|
||||
!!! warning
|
||||
To work in forensic mode, crowdsec-agent relies on [crowdsecurity/dateparse-enrich](https://hub.crowdsec.net/author/crowdsecurity/configurations/dateparse-enrich) to parse date formats. See dedicated hub page for supported formats.
|
||||
|
||||
|
||||
## Injecting alerts into existing database
|
||||
|
||||
If you already have a running crowdsec/Local API running and want to inject events into existing database, you can run crowdsec directly :
|
||||
|
|
Loading…
Reference in a new issue