Commit graph

232 commits

Author SHA1 Message Date
blotus dd03d07355
optimize the flush function by deleting alerts based on their id (#1054) 2021-11-17 10:15:38 +01:00
he2ss 0652e9ed08
feature cscli|crowdsec add additional labels on crowdsec dsn run (#1053)
* feature cscli|crowdsec add additional labels on crowdsec dsn run
2021-11-17 10:08:46 +01:00
Thibault "bui" Koechlin 3c768490ba
fix #873 without breaking backward (#1052) 2021-11-15 14:16:18 +01:00
Kerma Gérald 37c2a10e21
Use math.MaxInt32 instead of math.MaxUint32 (#980)
To fix 32 bits compilation in v1.2.0
https://github.com/crowdsecurity/crowdsec/issues/979

Signed-off-by: Kerma Gérald <gandalf@gk2.net>
2021-11-15 12:14:04 +01:00
Thibault "bui" Koechlin 7362828a3b
add --failures to explain feature : only display failed lines (#1048)
* add --failures to explain feature : only display failed lines

* no error no problem
2021-11-08 18:01:43 +01:00
Thibault "bui" Koechlin 8b0527bf9d
add evt. (#1045) 2021-11-03 15:17:48 +01:00
AlteredCoder fb54388e93
Fix issue 1033 (#1034)
* Fix issue 1033
2021-11-02 12:16:33 +01:00
Thibault "bui" Koechlin d1ce543440
Improve explain (#1039)
* improve explain feature

* nicer display for details, --verbose in favor of --debug for details
2021-11-02 12:06:01 +01:00
Shivam Sandbhor cbada3d435
Allow using cloudwatch using iam role instead of hardcoded tokens (#1035) 2021-11-02 10:25:35 +01:00
mmetc f10187bd6d
typos (#1036) 2021-11-02 09:19:22 +01:00
Thibault "bui" Koechlin 2b2a11fec7
Extra syslog debug (#1030)
* extra logging
2021-11-01 20:55:03 +01:00
AlteredCoder cf57c89177
add name and alias in cscli console enroll (#950)
* add name and alias in cscli console enroll
2021-10-26 15:33:17 +02:00
blotus 25a2d528b0
Alerts flush: Optimization of the flush mechanism (batch and limit to one job) + add cscli alerts flush command (#1024)
- Don't allow running more than one alert flush job at a time to prevent runaway CPU usage in some case. (fix High CPU after Upgrade to 1.2.0 #1022)
 - Add a cscli alerts flush command to manually flush the alerts in the database (fixes Improvement/Manual flush mechanism #1023 ).
 - Enable cascading deletion on alerts as we upgraded ent: Deleting an alert in the database will automatically delete all related decisions, events and meta
 - Add an index on alerts.id to try to improve flush performance with very big sqlite database.
- Flush alert now operates in batch
2021-10-26 13:33:45 +02:00
Thibault "bui" Koechlin 3f99330b3d
Entgo 0.9 (#1018)
* update entgo & sqlite to latest version

* schema update
2021-10-22 16:15:57 +02:00
Shivam Sandbhor a7b1c02bd5
Fix bugs in cloudwatch acq (#991)
* Fix bugs in cloudwatch acq

- Fix concurrent writes to map streamIndexes
- Fix multiple cases of modifying while iterating on slice.
- Fix order of fetching cloudwatch events.
- Remove `startup` hack.

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* Fix cloudwatch tests

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2021-10-22 10:35:05 +02:00
Thibault "bui" Koechlin 3bb2128bf4
fix sort :/ (#1007) 2021-10-12 19:16:24 +02:00
Thibault "bui" Koechlin 1bd6b8f7b9
Multiple fixes (#1006)
* fix #1005 : timestamp in trigger timemachine buckets

* attempt at consistent bucket order for hubtest
2021-10-12 14:09:17 +02:00
Thibault "bui" Koechlin 2961a0ed02
ensure machineID is included early enough into the alert (#1004) 2021-10-11 15:02:16 +02:00
blotus 2bc9f33e12
add ParseUri() expr helper (#994) 2021-10-08 16:50:31 +02:00
AlteredCoder 0ccc69696b
Break on success when alert already has decision (#997) (#999)
* Break on success when alert already has decision (#997)

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2021-10-05 11:30:34 +02:00
Thibault "bui" Koechlin af4bb350c0
hubtests revamp + cscli explain (#988)
* New hubtest CI for scenarios/parsers from the hub
 * New `cscli explain` command to visualize parsers/scenarios pipeline

Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Cristian Nitescu <cristian@crowdsec.net>
2021-10-04 17:14:52 +02:00
Thibault "bui" Koechlin c2fd173d1e
fix node success logic (#993)
* fix node success logic : only fail node on child failure if mother node has no successfull grok
2021-09-28 17:58:07 +02:00
he2ss fb308d5596
fix plugins logging in right level (#990) 2021-09-28 14:44:21 +02:00
he2ss db5ffb0040
Update test env (#987)
* update test_env
2021-09-24 18:06:30 +02:00
blotus f0db3742de
fix usage of regex.Match in cloudwatch module (#986) 2021-09-23 13:52:05 +02:00
Shivam Sandbhor cca76da2d6
Fix crash if plugin config is broken (#964)
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2021-09-10 14:25:34 +02:00
he2ss e651379964
add jsonExtractUnescape Helper (#962)
* add jsonExtractUnescape Helper
2021-09-10 12:43:11 +02:00
AlteredCoder 5ae69aa293
fix stacktrace when mmdb file are not present (#935)
* fix stacktrace when mmdb file are not present
2021-09-09 16:27:30 +02:00
blotus 7a1b955ad1
use our fork of grokky (#953) 2021-09-09 14:46:16 +02:00
Shivam Sandbhor b8e24a1e0b
Make plugin runner configurable and run only registered plugins (#944)
* Make plugin runner configurable and run only registered plugins
2021-09-08 11:36:42 +02:00
Thibault "bui" Koechlin 0ad6165ed2
fix release drafter + readme + remove dead readme for acquis (#933) 2021-09-03 09:07:24 +02:00
Manuel Sabban d7d591ff84
update to use cdn for hub (#920)
* update to use cdn for hub
* add cdn for version
* fix unit tests accodingly with new cdn

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2021-09-02 15:17:37 +02:00
Thibault "bui" Koechlin bed90a832e
fix #919 : display error message (#929)
* fix #919

* fix tests
2021-09-02 12:46:32 +02:00
Thibault "bui" Koechlin 589cb72d41
enforce a bit more parsing for resillience (#928) 2021-09-02 12:34:20 +02:00
Shivam Sandbhor b40fd36607
Add plugin interface code in protobufs package (#921)
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2021-08-31 14:40:17 +02:00
Thibault "bui" Koechlin 68c11dd827
don't try to send/don't notify if plugin chan is nil (#923) 2021-08-31 14:39:32 +02:00
blotus b5d0d56a11
add support for --since in journalctl DSN (#917) 2021-08-31 12:40:22 +02:00
ThinkChaos 448a227079
Minor changes to specific logs (#900)
- Minor changes to specific logs
- Fix LAPI to not push signals to CAPI when disabled #907
2021-08-25 18:30:05 +02:00
Thibault "bui" Koechlin c188d401a3
Improve CAPI pull management (#871)
* prepare for new consensus : thousands of ips

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2021-08-25 11:45:29 +02:00
Thibault "bui" Koechlin 950759f6d6
Output plugins (#878)
* Add plugin system for notifications (#857)
2021-08-25 11:43:29 +02:00
Manuel Sabban 4dbbd4b3c4
Download datafile (#895)
* add the ability to download datafile on cscli hub upgrade on files are missing
* fix stuff + lint
* fix error management

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2021-08-19 09:08:20 +02:00
Shivam Sandbhor f64f20fd53
Document scope parameter for stream API (#897)
Signed-off-by: Shivam Sandbhor <shivam@crowdsec.net>
2021-08-18 16:05:56 +02:00
Nanik b0746fbc4d
fix: add /health endpoint (#881)
* fix: add /health endpoint
2021-08-18 09:06:01 +02:00
Thibault "bui" Koechlin 05ac3ca402
if profile is in debug, log debug even if it matched the profile (#894) 2021-08-17 16:50:16 +02:00
Thibault "bui" Koechlin 25ed1c265d
fix #885 : remove dead dependencies for plugin (#891) 2021-08-17 10:32:15 +02:00
Thibault "bui" Koechlin fc7369c4ea
Fix big serialized entries (#877)
* bump serialized to 8k

* handle oversized serialized entry : progressively strip its size down
2021-08-03 15:46:10 +02:00
Thibault "bui" Koechlin 01028d0a09
Goroutine leak hunt (#874)
* close the writers of gin loggers + kill the tomb of httpServer

* body close defer
2021-07-30 11:41:17 +02:00
blotus cedfca07c2
don't wait for acquis tomb if we have no sources (#868) 2021-07-28 08:58:44 +02:00
Thibault "bui" Koechlin b6ee006078
ensure decisions from CAPI have proper case (#848) 2021-07-02 11:23:46 +02:00
Thibault "bui" Koechlin 033c8e17e8
fix #842 #837 (#845)
* fix #842 and move preflight checks tgth

* handle new container name

Co-authored-by: AlteredCoder <AlteredCoder>
2021-07-01 18:15:22 +02:00