beenull/crowdsec
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
1634 commits 83 branches 177 tags 151 MiB
Commit graph

805 commits

Author SHA1 Message Date
Sebastien Blot d3ce4cbf8e
up 2023-10-04 10:25:32 +02:00
Sebastien Blot d5e0c8a36b
up 2023-10-04 10:25:32 +02:00
Sebastien Blot 7fdd4d04fe
up 2023-10-04 10:25:32 +02:00
Sebastien Blot ca930cce09
wip 2023-10-04 10:25:32 +02:00
Sebastien Blot 502e21bc5b
wip 2023-10-04 10:25:31 +02:00
mmetc 8b5ad6990d
lint: pkg/cwhub (#2510) 
no functional changes
 
 - reformat
 - comments
 - whitespace
 - removed a dot or two in log messages
 - some "var x=y" -> x:=y
 2023-10-03 11:20:56 +02:00
mmetc 6dadfcb2ef
refact: simplify hubtest CopyDir() (#2509) 2023-10-03 11:17:02 +02:00
mmetc bfda483c0a
fix issue #2499 - nil dereference while using capi whitelists (#2501) 2023-10-02 11:42:17 +02:00
Laurence Jones b8e6bd8c9a
[Explain] s02 can cause panic if empty (#2486) 
* Add parsers length check as it can panic is enrich is empty

* Lets get smarter and loop backwards to find last successful stage

* Shorten code

---------

Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
 2023-09-29 12:03:56 +01:00
mmetc 95ed308207
cscli setup: accept stdin; fix proftpd detection test and service unmask (#2496) 2023-09-29 12:58:35 +02:00
Thibault "bui" Koechlin 8f6659a2ec
fix the float comparison by using Abs(a,b) < 1e-6 approach (IEEE 754). Move the initializiation of expr helpers (#2492) 2023-09-28 17:22:00 +02:00
Laurence Jones 37c0c067a8
cscli hubtest whitelist (#2479) 
* Initial tests

* Always print whitelist as we can compare if we mess up the opposite way
 2023-09-20 16:42:19 +01:00
Thibault "bui" Koechlin e4dcdd2572
fix include_capi filter (#2478) 2023-09-20 11:56:00 +02:00
mmetc ac01faf483
strip '=' signs from encoded api keys (#2472) 
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
 2023-09-19 14:00:23 +02:00
Thibault "bui" Koechlin 4c08e1e68c
exclude 'lists' too if we exclude CAPI (#2474) 2023-09-19 13:56:22 +02:00
mmetc d5b6f2974b
Avoid sending nil body with metrics (#2470) 2023-09-19 13:53:50 +02:00
Laurence Jones 64deeab1ec
Fix PO expr whitelist (#2471) 2023-09-19 12:51:03 +01:00
bui 42341222df up 2023-09-19 08:54:31 +02:00
bui a8321b5cc5 up 2023-09-14 09:43:22 +02:00
bui 6a47b9e97d up 2023-09-13 18:03:03 +02:00
bui 7081666199 up 2023-09-13 17:34:53 +02:00
bui 2e60e8021c up wip 2023-09-13 17:12:09 +02:00
blotus 43ef32aa8d
Kafka acquisition: do not create empty events when a read error occurs (#2466) 2023-09-13 13:20:36 +02:00
bui c435447d8e up 2023-09-13 10:57:29 +02:00
bui 6930b1e3e5 up 2023-09-13 10:45:06 +02:00
bui 1286efc74f up 2023-09-12 18:17:58 +02:00
Thibault "bui" Koechlin 0040569fa9
if 'include capi' is false, only exclude capi alerts instead of assuming they necessarily have attached decisions (#2435) 2023-09-12 11:19:36 +02:00
mmetc d45bec4047
minor log message improvements (#2455) 2023-09-12 11:04:56 +02:00
bui 5a0b1b72d3 up 2023-09-12 10:42:28 +02:00
bui 1a5799e058 up 2023-09-12 09:45:14 +02:00
Thibault "bui" Koechlin 4e26e23725
Waap config (#2460) 
* revamp wip
 2023-09-11 10:35:14 +02:00
bui 24d2c264a7 clarify logging if triggering inband or outofband rules 2023-09-05 17:56:02 +02:00
mmetc fd94e2c056
refactor alert/decisions insert/update to avoid database locking in bulk operations (#2446) 2023-09-04 14:21:45 +02:00
Laurence Jones aff80a2863
Add html escape function so it can be invoked from template (#2451) 2023-09-04 09:49:39 +01:00
alteredCoder 0379574b14 support SSL for waf 2023-08-31 11:07:51 +02:00
mmetc 25868f27de
option db_client.decision_bulk_size (#2440) 2023-08-25 17:05:17 +02:00
mmetc c588be0842
golangci-lint: use v1.54, remove unnecessary byte/string conversions (#2438) 2023-08-25 16:22:10 +02:00
alteredCoder e0bd4dc928 fix linter 2023-08-24 12:11:54 +02:00
mmetc 2aa55e9444
move plugins/notifications/* to cmd/notification-* (#2429) 
This ensures keeping all dependencies in sync, and simplifies
packaging under freebsd/gentoo/etc because there is a single
vendor directory.
 2023-08-24 09:46:25 +02:00
mmetc e36df40ba7
pkg/types cleanup (#2398) 
* move function GetLineCountForFile from pkg/types to cscli
* move ParseDuration from pkg/types to pkg/database
* remove unused types.Profile, types.RemediationProfile
 2023-08-24 09:44:46 +02:00
Laurence Jones 86d9384954
Whitelist reason (#2439) 
* Update node.go

Dont update whitelist reason if event is whitelisted

* oops
 2023-08-23 14:51:37 +01:00
bui 4846701ed5 logging 2023-08-21 15:34:18 +02:00
mmetc 6a6501691a
change behavior of flag disable_http_retry_backoff (#2426) 
now it does not attempt any retry, instead of attempting all retries
immediately

example: cannot reach LAPI

Before:

$ CROWDSEC_FEATURE_DISABLE_HTTP_RETRY_BACKOFF=true cscli decisions list
ERRO[27-07-2023 10:44:44] error while performing request: dial tcp [::1]:8080: connect: connection refused; 4 retries left
INFO[27-07-2023 10:44:44] retrying in 0 seconds (attempt 2 of 5)
[...]
ERRO[27-07-2023 10:44:44] error while performing request: dial tcp [::1]:8080: connect: connection refused; 1 retries left
INFO[27-07-2023 10:44:44] retrying in 0 seconds (attempt 5 of 5)
ERRO[27-07-2023 10:44:44] error while performing request: dial tcp [::1]:8080: connect: connection refused; 0 retries left
FATA[27-07-2023 10:44:44] Unable to list decisions : performing request: Get "http://localhost:8080/v1/alerts?has_active_decision=true&include_capi=false&limit=100": could not get jwt token: Post "http://localhost:8080/v1/watchers/login": dial tcp [::1]:8080: connect: connection refused

After:

$ CROWDSEC_FEATURE_DISABLE_HTTP_RETRY_BACKOFF=true ./test/local/bin/cscli decisions list
FATA[11-08-2023 16:49:58] unable to retrieve decisions: performing request: Get "http://127.0.0.1:8080/v1/alerts?has_active_decision=true&include_capi=false&limit=100": could not get jwt token: Post "http://127.0.0.1:8080/v1/watchers/login": dial tcp 127.0.0.1:8080: connect: connection refused
 2023-08-16 21:04:07 +02:00
mmetc afeb541eac
apic: minor refactoring (#2415) 
* apic: minor refactoring

* Add whitelist length check

If user configures the file but fails to define and actual whitelist we should check length to save allocs

* Init with length from file

* extract loop method from ApplyApicWhitelists

* pass pointer

* extract loop method updateBlocklist

---------

Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
 2023-08-10 13:03:47 +02:00
Laurence Jones 93c22f29cf
Unmarshal Json (#2414) 
Log the actual line that caused an error to help debugging
 2023-08-09 09:42:08 +01:00
Manuel Sabban d6361d0a40
conditional overflow doesn't overflow on capacity (#2412) 
* conditional overflow doesn't overflow on capacity

* typo
 2023-08-08 16:12:50 +01:00
mmetc cd9d8f309d
CI: increase test sleep to fix flaky acquisition/file test under win (#2410) 
* CI: increase test sleep to attempt fix for flaky windows acquitition/file test

* wip
 2023-08-08 16:11:32 +02:00
Laurence Jones 0334a9afe8
Add method name to child logger so we can see which function is erroring when in enrichers (#2411) 2023-08-08 13:38:11 +01:00
Sebastien Blot a4ee1e717e
try re2 for @rx operator 2023-08-02 11:47:35 +02:00
Sebastien Blot 59e3d0dfce
distinct: return emtpy slice 2023-08-02 11:43:49 +02:00
alteredCoder 885c283097 remove debug 2023-08-01 10:58:36 +02:00
alteredCoder cbf06c25fb fix outofband evt generation 2023-08-01 10:34:43 +02:00
alteredCoder 353926ec91 add debug 2023-07-31 18:47:54 +02:00
alteredCoder 4332598cd1 add debug 2023-07-31 18:44:32 +02:00
alteredCoder 51295ef577 fix 2023-07-31 18:39:15 +02:00
alteredCoder da37b5566d update 2023-07-31 18:35:35 +02:00
alteredCoder 343d22e7b3 fix rules helpers 2023-07-31 18:29:00 +02:00
Sebastien Blot 711f0474d9
merge from master 2023-07-31 17:05:25 +02:00
Sebastien Blot dd83bdea6b
revert previous bad merge 2023-07-31 17:00:06 +02:00
alteredCoder fc8a0ee9d4 update 2023-07-31 15:06:42 +02:00
bui 4a38cb5bbb logging 2023-07-31 14:47:48 +02:00
bui e4e2bb5504 switch to properly compiled regexp to be able to bail out early 2023-07-31 14:45:21 +02:00
bui a7cd86f725 allow to select what variables shouldd be tracked 2023-07-31 12:15:04 +02:00
Laurence Jones a18df9c3bb
Add bouncers prune command (#2379) 
* Add bouncers prune command

* No point overloading functions

* Add prune to list of commands

* change all short desc to be similar, and made it really really clear when pruning it is not recoverable

* Dont use log. and dont return error on user input to abort
 2023-07-28 15:37:39 +01:00
mmetc ffadd42779
update dependency on go-cs-lib; drop the pkg/ part (#2393) 2023-07-28 16:35:08 +02:00
Laurence Jones 55247cd46a
Add machines prune command (#2011) 
* Add machines prune command

* Fix scope variable for naming scheme

* Add some freshness and add new features

* Fix force and fix duration if less than 60

* Allow duration to be more readable

* Fix description

* Improve func wording and make int machines length

* No point overloading functions

* Add prune to list of commands

* Check if GID is already the group if so no need to chown

* Revert "Check if GID is already the group if so no need to chown"

This reverts commit c7cef1773e.

* change all short desc to be similar, and made it really really clear when pruning it is not recoverable

* Better examples

* Match bouncer like for like

* Fix merge error

* Dont use log. and dont return error on user input to abort
 2023-07-28 15:23:47 +01:00
mmetc ae53c0f1cc
fix "crowdsec-cli/require" log verbosity (#2390) 2023-07-28 09:56:20 +02:00
Thibault "bui" Koechlin 718721b341
fix a confusing debug message (#2386) 
* fix a confusing debug message

* make CTIHelper simply log the error to avoid failing template rendering
 2023-07-28 09:52:21 +02:00
mmetc 5cb7013575
Check cscli preconditions with crowdsec-cli/require package (#2388) 2023-07-27 17:02:20 +02:00
Sebastien Blot dd5e38a2c5
expose internal coraza vars in evt.Waap 2023-07-27 10:01:56 +02:00
Sebastien Blot 2f5a6fbb4f
wip 2023-07-27 09:22:26 +02:00
Sebastien Blot f7e098047f
waf_rules -> waf-rules 2023-07-27 09:22:26 +02:00
Sebastien Blot 792961d757
wip 2023-07-27 09:22:26 +02:00
Sebastien Blot 01ced8fb99
merge 2023-07-27 09:22:26 +02:00
alteredCoder 4993758b36 handle missing headers 2023-07-26 12:47:16 +02:00
mmetc a01ce18b98
replace imports of path with path/filepath (#2330) 2023-07-26 10:29:58 +02:00
alteredCoder c17b103f06 take method from header 2023-07-25 15:24:36 +02:00
Laurence Jones 389ea4293f
Add metabase version override and update (#2370) 
* Add version override and update

* Ooppsie

* Quick fix

* fgs copilot

* Allow user to overwrite image, add warning for exposing metabase and general cleanup

* One ix

* Default image if not found in config, and add a warning to remove and update

* Reorder check system memory checks so it inline with @mmetc best pratices

* No need for err

* Clean up some group code

* Change ipv6 as [] seems to wildcard

* Split loopback warn and disclaimer. Add force yes to start to allow user to accept disclaimer by default

* All cmd commands are RunE clean up

* Update flag name and dont allow a shorthand
 2023-07-25 14:21:25 +01:00
mmetc 395cace69f
fix double push of metrics by properly handling tickers (#2374) 2023-07-25 12:19:26 +02:00
blotus 7106d396dc
expose the FormatAlert function to other packages (#2248) 2023-07-25 09:55:39 +02:00
AlteredCoder b52b4252c1
scenario labels to map string interface (#2201) 
* labels are now map string interface

* restore api url

---------

Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
 2023-07-24 15:19:28 +02:00
mmetc 46fff0b544
Update dependency: docker/docker (#2360) 2023-07-24 11:53:33 +02:00
mmetc b6b6fd026b
typo fix, uppercase 'API', adjusted log level (#2361) 2023-07-21 23:23:24 +02:00
bui a326ffbb1e add distinct 2023-07-20 17:30:58 +02:00
bui b33ba277bf add flatten to manipulate arrays of arrays 2023-07-20 17:10:01 +02:00
bui 54fd2e4e70 fixed 2023-07-20 16:47:07 +02:00
Manuel Sabban 9ac5aeda79
fix the ci by adding the ability to enforce event ordering (#2347) 
* fix the ci by adding the ability to enforce event ordering
 2023-07-20 11:41:30 +02:00
alteredCoder 779ea2e262 fix 2023-07-19 18:19:14 +02:00
alteredCoder 472f40b9d4 fix 2023-07-19 18:18:24 +02:00
alteredCoder ab2c152627 reduce verbosity 2023-07-19 14:39:57 +02:00
alteredCoder 7d8c931d00 add loggers 2023-07-19 14:35:02 +02:00
alteredCoder 8ba692b115 debug 2023-07-19 12:02:38 +02:00
alteredCoder cd5cb55a7e debug 2023-07-19 11:57:14 +02:00
alteredCoder d946286e5c remove spew 2023-07-19 11:50:42 +02:00
alteredCoder e543523ba3 update ban remediation 2023-07-19 10:34:22 +02:00
bui f7eaefa518 up 2023-07-18 18:12:17 +02:00
Sebastien Blot ef4fe8f5d3
merge 2023-07-13 16:22:21 +02:00
blotus 57547c32c9
Aggregate WAF rules into a single event (#2350) 2023-07-13 16:20:04 +02:00
bui a6ba0e869c imp logging 2023-07-11 09:29:17 +02:00
bui 8baeb70998 add metrics 2023-07-10 18:00:19 +02:00
blotus f9ca14f010
add object key in src for S3 acquis (#2342) 2023-07-07 10:09:18 +02:00
blotus 1295de928a
Properly match new files on windows when doing file acquisition (#2329) 2023-07-06 14:45:38 +02:00
mmetc c10bca93df update dependencies on go-plugin and go-hclog (#2341) 
* update dependencies on go-plugin and go-hclog
* bump logrus (panic fix)
* implement HCLogAdapter.Getleve() to satisfy the new interface
 2023-07-06 12:01:07 +02:00
mmetc 9967d60987
errors.Wrap -> fmt.Errorf (#2333) 2023-07-06 10:14:45 +02:00
alteredCoder 84b6570554 Revert "Merge remote-tracking branch 'origin' into coraza_poc_acquis" 
This reverts commit 7098e971c7, reversing
changes made to 13512891e4.
 2023-07-04 18:46:20 +02:00
alteredCoder 7098e971c7 Merge remote-tracking branch 'origin' into coraza_poc_acquis 2023-07-04 17:42:39 +02:00
alteredCoder 13512891e4 add waf_routines 2023-07-04 17:36:56 +02:00
mmetc 17cd792826
CI: update ansible tests for re2 (#2318) 2023-06-29 16:35:19 +02:00
mmetc bd41f855cf
errors.Wrap -> fmt.Errorf (#2317) 2023-06-29 11:34:59 +02:00
blotus e61d5a3034
rename status to state in fire response (#2313) 2023-06-29 11:06:49 +02:00
mmetc 893394ef5f
rename metabase APIClient to avoid confusion (#2305) 2023-06-27 15:07:16 +02:00
mmetc e404e0b608
raise error with invalid 'on_success', 'on_failure' in profile (#2303) 2023-06-27 15:03:07 +02:00
mmetc 85839b0199
support for stdin with "cscli decision import" and raw values (#2291) 
and remove Origin from the struct, which was ignored anyway
 2023-06-27 14:29:42 +02:00
mmetc a910b7beca
non-fatal error if some datasource can't be run (i.e. journalctl but systemd is missing) (#2309) 
This on the other hand, gives a new fatal error when there are no valid datasources.
In the previous version, crowdsec kept running with just a warning if no
acquisition yaml or dir were specified.
 2023-06-27 10:13:13 +02:00
mmetc 507da49b5a
send metrics immediately if agents are added or removed (#2296) 2023-06-23 14:06:04 +02:00
mmetc 9beb5388cb
errors.Wrap -> fmt.Errorf; clean up imports (#2301) 2023-06-23 14:04:58 +02:00
mmetc e42841cd00
Change api_key encoding to base64 to comply with bcrypt max size (#2302) 2023-06-23 13:54:36 +02:00
mmetc 62caffb102
update leakybucket readme (#2298) 2023-06-22 15:35:01 +02:00
mmetc fddf597040
errors.Wrap -> fmt.Errorf; clean up imports (#2297) 2023-06-22 15:01:34 +02:00
mmetc 8bfeb7d90d
Update go dependencies (#2293) 
- update fatih/color (fix windows issue)
- update mongo-driver (fix build issue)
- go.mod: merge two "require" blocks
- update semver dependency (same version as indirect dep), fix test checks in cscli setup
- remove gotest.tools dependency (use testify, cstest)
- update x/ exp, mod, sys dependencies
 2023-06-22 11:31:41 +02:00
Emanuel Seemann 40e6b205bc
Add bayesian bucket type (#2290) 2023-06-21 15:08:27 +02:00
mmetc da6106bd23
spellcheck/style leakybucket readme (#2294) 2023-06-21 11:47:07 +02:00
mmetc f7409d47be
fix error message when failing to parse ip address (#2292) 
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
 2023-06-21 09:22:25 +02:00
Laurence Jones 2c8769adf6
Update jsonextract.go (#2287) 
Return nil instead of empty string as ParseKV does the same
 2023-06-16 18:34:55 +01:00
Sebastien Blot 3fe6e3be14
check for interruption and ignore empty messages 2023-06-16 16:52:01 +02:00
alteredCoder 877d4fc32d update 2023-06-16 14:23:53 +02:00
alteredCoder 07b60233db update waf 2023-06-16 12:19:44 +02:00
Sebastien Blot 9180ac7be9
wip 2023-06-15 22:51:57 +02:00
Sebastien Blot 805752dc62
wip 2023-06-13 17:08:48 +02:00
alteredCoder 40f65de7b9 optim 2023-06-13 16:31:30 +02:00
alteredCoder fa172bed56 up 2023-06-13 15:41:32 +02:00
mmetc b9a3acb03f
light pkg/parser cleanup (#2279) 
* pkg/parser: clean up imports
* remove duplicate import
* simplify boolean expression
* don't check length before range
* if..else if.. -> switch/case
* errors.Wrap -> fmt.Errorf
* typo, lint
* redundant break
 2023-06-13 13:16:13 +02:00
Sebastien Blot a2e6359880
merge 2023-06-09 13:01:58 +02:00
Sebastien Blot c46e2ccdad
up 2023-06-09 13:00:43 +02:00
alteredCoder 61e1cc29d5 update 2023-06-08 17:45:21 +02:00
mmetc 76429f033a
trim pkg/types: move DataSet/GetData to pkg/cwhub, removed unused Clone function (#2271) 2023-06-08 16:49:51 +02:00
mmetc cf747d65e0
fix missing import (#2275) 2023-06-08 15:49:37 +02:00
mmetc 25bb23d8b7
minor refactor to pkg/types, cscli machines (#2270) 
* cleanup: separate ui and logic
* trim some code from pkg/types
 2023-06-08 15:08:51 +02:00
mmetc 6096cb3c9b
Move grok_pattern.go away from pkg/types to trim bouncer dependencies (#2269) 2023-06-08 15:07:30 +02:00
mmetc 8da9d5eefd
don't log notification error if not running under systemd (#2274) 2023-06-08 15:04:48 +02:00
Sebastien Blot 415e2dc68d
merge 2023-06-08 11:22:16 +02:00
bui 739d086325 up 2023-06-07 14:12:42 +02:00
bui 30455a8eb6 progress 2023-06-07 13:45:36 +02:00
mmetc 5b3200173e
don't pre-create log files (not required anymore) (#2267) 
The lumberjack package fixed the issue in natefinch/lumberjack#83 (tested with umask 002) and this code is now redundant since we updated the dependency to v2.2.1.
 2023-06-07 12:58:35 +02:00
bui d123254949 wip 2023-06-06 18:28:06 +02:00
Thibault "bui" Koechlin ee8b31348b
Merge branch 'master' into coraza_poc_acquis 2023-06-06 18:23:59 +02:00
mmetc edd062522d
build against libre2-dev if found (#2255) 2023-06-06 15:46:25 +02:00
mmetc 3cc6b2c0d0
CI: add tests for metrics configuration (#2251) 2023-06-05 23:17:30 +02:00
mmetc 0191faf3a8
update notif threshold test on windows (#2265) 2023-06-05 22:58:13 +02:00
Sebastien Blot 4a7e26af02
wip 2023-06-05 19:33:03 +02:00