beenull/crowdsec
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
1397 commits 83 branches 177 tags 151 MiB
Commit graph

597 commits

Author SHA1 Message Date
mmetc 5b3200173e
don't pre-create log files (not required anymore) (#2267) 
The lumberjack package fixed the issue in natefinch/lumberjack#83 (tested with umask 002) and this code is now redundant since we updated the dependency to v2.2.1.
 2023-06-07 12:58:35 +02:00
mmetc edd062522d
build against libre2-dev if found (#2255) 2023-06-06 15:46:25 +02:00
mmetc 3cc6b2c0d0
CI: add tests for metrics configuration (#2251) 2023-06-05 23:17:30 +02:00
mmetc 0191faf3a8
update notif threshold test on windows (#2265) 2023-06-05 22:58:13 +02:00
mmetc e3cb4ab2c4
do not send more than group_threshold alerts at once to a notification plugin (#2264) 
* do not send more than group_threshold alerts at once to a notification plugin
* Use generic Chunks function, updated tests

---------

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-06-05 12:55:03 +02:00
mmetc a4eee41fd7
log.Warning if a notification is configured twice (#2240) 2023-06-02 14:41:50 +02:00
mmetc 396dcf8e6e
dependencies: replaced function calls to pkg/types, errors.Wrap (#2235) 
we now use a generic pointer function, and slowly remove the deprecated pkg/errors
 2023-06-01 16:31:56 +02:00
mmetc 12c32d507c
CI: refactoring pkg/csplugin tests (#2247) 2023-06-01 10:33:08 +02:00
mmetc 92a9d6c321
types.InSlice() -> slices.Contains() (#2246) 2023-05-31 12:39:22 +02:00
Laurence Jones 4fbc3402fb
Update KV ignore whitespace before and after = (#2236) 
* Update KV ignore whitespace before and after `=`

* Update helpers.go

Don't need whitespace infront of KEY

* Add some tests to ensure edge cases

* Ensure quoted and unquoted values act the same
 2023-05-26 15:35:46 +01:00
blotus 6720d89845
fix lock when dumping the parsing state in explain mode (#2234) 2023-05-26 15:23:50 +01:00
blotus f6924f8c57
generate asserts for evt.Unmarshaled in hubtest (#2214) 2023-05-26 11:44:58 +02:00
mmetc 9167bd107d
decouple bouncer dependencies: use go-cs-lib/pkg/ptr (#2228) 2023-05-25 15:43:39 +02:00
mmetc b2d3520519
decouple bouncer dependencies: use go-cs-lib in test code (#2229) 2023-05-25 15:37:44 +02:00
mmetc 364b833d67
test cleanup: remove /tmp/crowdsec_tests* directories (#2232) 2023-05-25 15:32:32 +02:00
Laurence Jones 0416a41d58
Log info capi whitelists (#2220) 
* add infof command if err was nil

* Fix golint

* Make message more readable and log individual stats

* Missed a d

* Remove '

* simplify if/else logic

---------

Co-authored-by: Marco Mariani <marco@crowdsec.net>
 2023-05-25 10:28:08 +01:00
mmetc 025f14f879
merge system cert pool with own certs (#2226) 2023-05-25 10:10:58 +02:00
mmetc e5fe74ce77
decouple bouncer dependencies: use go-cs-lib/pkg/ptr in apiclient (#2227) 2023-05-25 10:08:52 +02:00
mmetc 534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* (#2216) 
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
 2023-05-23 10:52:47 +02:00
blotus 6e3ca35941
fallback to master for hub index download if it does not exist (#2210) 2023-05-17 11:20:53 +02:00
blotus 412b4c4b0b
fix incorrect version strip (#2206) 2023-05-17 01:13:55 +02:00
Thibault "bui" Koechlin 77f2968267
fix the behavior of json unmarshal to not return the full map (#2199) 2023-05-16 09:10:38 +02:00
Laurence Jones 424215f228
Add ParseKV helper and rework UnmarshalJSON as a proper helper (#2184) 2023-05-12 09:43:01 +02:00
mmetc e1f5ed41df
Implement "cscli config show-yaml" (#2191) 2023-05-11 21:01:13 +02:00
blotus 4ae41a363d
add Hostname helper in expr and templating (#2193) 2023-05-11 14:25:04 +02:00
blotus 71b7a594bd
add indexes on the FK between alerts and {decisions,metas,events} (#2188) 2023-05-11 13:49:01 +02:00
blotus 2701454f23
defaults to inotify to detect changes in file datasource to avoid too many call to stat() (#2181) 2023-05-09 10:03:55 +02:00
blotus e1f4a71357
readd KeyExists expr helper (#2180) 2023-05-04 16:55:34 +02:00
blotus a753ea6981
Add B64decode expr helper (#2183) 2023-05-04 14:15:20 +02:00
Thibault "bui" Koechlin 8f71edaadd
do not error on this filter (#2182) 2023-05-04 13:06:15 +02:00
Thibault "bui" Koechlin 4ff8f498ce
add a LogInfo expr helper (#2179) 2023-05-03 10:07:11 +02:00
AlteredCoder 6bb20fa951
fix issue #2172 (#2177) 2023-04-28 16:32:46 +02:00
AlteredCoder c0e6c1ac78
Fix chooseHubBranch when latest() doesn't work (#2178) 
* Fix chooseHubBranch when latest() doesn't works
 2023-04-28 11:24:04 +02:00
Thibault "bui" Koechlin 3041023ed8
add an optional flag to disable the fetch (#2169) 2023-04-14 11:39:16 +02:00
Thibault "bui" Koechlin 66dfded0cf
significantly increase the max number of scenarios to be sent (#2170) 2023-04-14 11:39:07 +02:00
mmetc 0c5d233563
Minor cleanup and dead code removal (#2166) 2023-04-12 16:57:38 +02:00
Laurence Jones 9a5a937695
Make it more obvious that parser succeeded but was whitelisted (#2167) 
* Make it more obvious that parser succeeded but was whitelisted

* Add more verbose by placing whitelist reason next to why it is ignored
 2023-04-12 10:48:42 +01:00
blotus 0279e549bd
check if the acquis tomb is dying while processing logs in replay mode for file/s3/docker (#2152) 2023-04-04 13:57:06 +02:00
mmetc 3132aa54b7
Properly load k8s audit configuration (#2158) 2023-04-03 21:55:31 +02:00
mmetc 38ab6be7c2
Allow feature.yml to change available subcommands (#2156) 2023-04-03 10:11:56 +02:00
mmetc 3fa555fb25
Rename k8s_audit to k8s-audit (easier to type, consistent with labels) (#2153) 2023-04-03 09:53:38 +02:00
blotus 61bea26486
Add transform configuration option for acquisition (#2144) 2023-03-29 16:04:17 +02:00
blotus 772d5b5c32
Add experimental support for re2 (#2138) 2023-03-28 16:26:47 +02:00
blotus 1095f6c875
use expr.Function for custom functions instead of passing them in the env (#2133) 2023-03-28 10:49:01 +02:00
Thibault "bui" Koechlin 169b844212
fix awkward stacktrace in conditional filter (#2145) 2023-03-27 16:01:42 +02:00
mmetc d769fff1e8
File acquisition: log "file reopen" events instead of writing to stderr (#2139) 2023-03-24 11:24:36 +01:00
mmetc 3884c5f47d
Unit tests: remove leftover files (#2134) 2023-03-22 13:51:37 +01:00
Thibault "bui" Koechlin a3e5f0a3a0
fix dateparse (#2135) 2023-03-22 08:20:21 +01:00
blotus 91eb39cff6
New PAPI commands: reauth + force_pull (#2129) 2023-03-21 14:06:19 +01:00
blotus dc38e5ac00
S3 acquisition datasource (#2130) 2023-03-21 13:54:52 +01:00
Thibault "bui" Koechlin a74e424d53
support ip and cidr based whitelists for capi and 3rd party blocklists (#2132) 
* support ip and cidr based whitelists for capi and 3rd party blocklist
 2023-03-21 11:50:10 +01:00
Thibault "bui" Koechlin d87f088b8f
match expr helper (#2126) 
* match expr helper
 2023-03-21 10:39:17 +01:00
Thibault "bui" Koechlin 618be9ff68
properly update the time structure within event (#2122) 
* properly update the time structure within event to ensure it works in time-machine

* move LIVE and TIMEMACHINE to pkg/types : less code needs to import leakybucket package, and we avoid duplicating constants
 2023-03-16 16:25:50 +01:00
blotus c77fe16943
actually fix expr-debugger to work with the new version (#2124) 2023-03-16 15:20:48 +01:00
blotus 94c7efdb5b
add ToString() helper (#2100) 2023-03-16 15:20:31 +01:00
blotus b1f2063a9a
Only support pgx driver for postgresql (#2118) 2023-03-16 11:02:31 +01:00
Thibault "bui" Koechlin 855f9e6f8d
protect map w/ mutex to avoid concurrent map writes with cscli explain when having many concurrent parser routines (#2113) 2023-03-16 11:01:25 +01:00
Manuel Sabban b451d190b7
try to make reproducible build work (#2119) 
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
 2023-03-13 17:26:33 +01:00
blotus 6aaf3cd50b
Update expr to 1.12.2 (#2110) 2023-03-09 16:56:11 +01:00
mmetc e161507d08
Lint (type inference): remove redundant type declarations (#2111) 2023-03-09 11:56:02 +01:00
Thibault "bui" Koechlin d95b7afe61
Distance support : Impossible travel (#2108) 
* add distance helpers
 2023-03-08 18:29:42 +01:00
Thibault "bui" Koechlin 9d5aaf5ea2
add --origin to cscli decisions delete (#2109) 2023-03-08 18:29:20 +01:00
Thibault "bui" Koechlin 5b0fe4b7f1
support for regexps result cache (#2104) 
* support for regexps result cache : gcache + xxhash

Co-authored-by: Marco Mariani <marco@crowdsec.net>
 2023-03-08 16:07:49 +01:00
blotus 16a3be49e2
do not try to load PAPI is url is not set (#2099) 2023-03-06 15:38:58 +01:00
blotus 85ab9c68a2
Add cscli papi status and cscli papi sync (#2091) 2023-03-03 13:46:28 +01:00
mmetc f6d6c5bb2b
Add tests and typo fixes (#2092) 2023-03-03 11:06:27 +01:00
AlteredCoder 01ea78c10e
Strip version with ~ instead of - (#2076) 2023-02-25 20:05:48 +01:00
Laurence Jones 75d8b821ff
Explain successful parsers only (#2063) 
* Add option to filter down explain to successful parsers useful for me who has every collection installed

* Altered naming conventions so it makes more sense when reading
 2023-02-24 13:49:17 +00:00
Laurence Jones 8acce4637a
Option to disable remote lapi registration (#2010) 
* Allow to disable remote lapi registration

* Extract method and make it extendable as a generic middleware

* Change method name so it make sense to read abort remote if <config>

* golint
 2023-02-24 13:44:21 +00:00
mmetc 20a1bc7d44
chore: simplify pkg/database/alerts (#2062) 2023-02-23 10:25:01 +01:00
mmetc be18fea136
Propagate taints to top collections (fix #2064) (#2066) 2023-02-21 22:12:08 +01:00
mmetc 76ea3a063f
fix message "empty scenario" 2023-02-21 09:59:56 +01:00
blotus 90c38db9f2
Stream decisions from db (#1927) 2023-02-20 15:26:30 +01:00
JDEV 12a4a5fb14
CAPI error code handling tests (#2027) 
* Registration mocked error cases

* Authentication mock error cases

* mini facto

* check that getMEtric still has bouncers/machines keys in output even with empty collections

* fixed defer body close(), no need to defer and fprint arg

* fix fatal call

---------

Co-authored-by: jdv <julien@crowdsec.net>
 2023-02-17 14:57:46 +01:00
blotus 83c3818504
Do not try to refresh JWT token when doing a login request (#2059) 2023-02-16 16:16:26 +01:00
Laurence Jones 5aca11af70
Show s00 stats instead of "first_parser" (#2055) 
* show s00 if verbose is provided

* Clean up code

* Fix failing test
 2023-02-14 14:36:08 +00:00
Cristian Nitescu ecb32d74c6
optimize blocklist fetch (#2039) 2023-02-13 15:06:14 +01:00
Cristian Nitescu f280505eaa
omtimization - remove useless login call (#2036) 2023-02-13 15:05:58 +01:00
blotus 812b87ab48
Add IsIPV4() and IsIP() helpers (#2050) 2023-02-10 14:44:42 +01:00
Thibault "bui" Koechlin 0f5560b62a
more strings helpers (#2040) 
* more strings helpers
 2023-02-09 15:23:21 +01:00
Thibault "bui" Koechlin 1d7d377f8b
changes following BL tests (#2038) 
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-02-08 10:35:21 +01:00
Thibault "bui" Koechlin a0b264047c
allow user to specify stash's cache strategy (#2037) 2023-02-06 15:42:55 +01:00
Cristian Nitescu 987f119c4b
v3 capi and blocklists links support (#2019) 
* v3 model generation

* v3 model generation

* comms

* fixes after master merge

* missing reader close

* use constants defined for types

---------

Co-authored-by: bui <thibault@crowdsec.net>
 2023-02-06 14:06:14 +01:00
mmetc b6be18ca65
cscli setup (#1923) 
Detect running services and generate acquisition configuration
 2023-02-06 07:33:04 +01:00
AlteredCoder 7e871d2278
rename PAPI base URL (#2033) 2023-02-03 12:10:02 +01:00
Thibault "bui" Koechlin e927717fa0
Polling API Integration (#1715) 
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-01-31 14:47:44 +01:00
mmetc e37d09e5b4
use helpers for shorter tests, add a couple of error cases (#2016) 2023-01-26 17:13:31 +01:00
mmetc 3fb3decf49
error if tls.key_file or cert_file are missing (#2020) 2023-01-26 17:12:59 +01:00
mmetc 02be5f3618
allow literal $ in plugin configuration (#2015) 2023-01-23 16:28:43 +01:00
mmetc 47cc60bda9
allow use of literal $ in config.yaml (#2012) 2023-01-23 10:29:29 +01:00
mmetc e5833699c0
cscli config feature-flags (#2006) 2023-01-20 09:32:10 +01:00
Thibault "bui" Koechlin 4f29ce2ee7
CTI API Helpers in expr (#1851) 
* Add CTI API helpers in expr
* Allow profiles to have an `on_error` option to profiles

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-01-19 08:45:50 +01:00
Marco Mariani 0c35d9d43c wip 2023-01-18 15:15:18 +01:00
Marco Mariani 4f25738d6b wip 2023-01-18 15:15:18 +01:00
Marco Mariani 47dbfa770d configure logging earlier 2023-01-18 15:15:18 +01:00
Marco Mariani 91b0f8fee1 load custom configuration paths when agent is disabled 2023-01-18 15:15:18 +01:00
Marco Mariani 2e91a82aa7 load feature.yaml as soon as possible 2023-01-18 15:15:18 +01:00
Thibault "bui" Koechlin f25fdecc3f
normalize scopes for alerts and decisions (#2001) 
* normalize scopes for alerts and decisions
 2023-01-18 14:50:03 +01:00
mmetc 51800132cd
improve feature flag logging (#1986) 
For cscli: it should provide a terse output, not nag users with configuration details. Although it's usually important that cscli and crowdsec have the same enabled features, having it list them every time the command is invoked can be too much.

For crowdsec: when features are set from the environment, it's too early to log where we should. So we can use log.Debug at activation time, and list them again once logging is configured.

 - wrap some functions in csconfig for convenience and DRY
 - for each enabled feature, log.Debug
 - log all enabled features once as Info (crowdsec) or Debug (cscli)
 - file does not exist -> log.Trace
 2023-01-13 13:42:42 +01:00
Cristian Nitescu 73663ff9e7
log the request error even in case of retry (#1988) 2023-01-13 12:58:12 +01:00