Commit graph

91 commits

Author SHA1 Message Date
mmetc fb393f1c57
tests: bump yq, cfssl (#2952) 2024-04-22 17:19:00 +02:00
blotus c6e40191dd
Revert "docker: pre-download all hub items and data, opt-in hub updat… (#2947) 2024-04-18 15:33:51 +02:00
mmetc 0e8a1c681b
docker: pre-download all hub items and data, opt-in hub update/upgrade (#2933)
* docker: pre-download all hub items and data, opt-in hub update/upgrade

* docker/bars: don't purge anything before pre-downloading hub

* Docker: README update
2024-04-08 14:53:12 +02:00
mmetc 6c042f18f0
LAPI: local api unix socket support (#2770) 2024-03-14 10:43:02 +01:00
mmetc 49e0735b53
cscli tests + fix bouncer/machine prune (#2883)
* func tests: "cscli config feature-flags"
* func tests: "cscli bouncers list"
* func tests + fix: "cscli bouncers/machines prune"
* lint
2024-03-11 13:14:01 +01:00
mmetc d8877a71fc
lp metrics: collect datasources and console options (#2870) 2024-03-05 14:56:14 +01:00
mmetc d34fb7e8a8
log processor: share apiclient in output goroutines (#2836) 2024-02-13 14:22:19 +01:00
mmetc 4561eb787b
bats: color formatter in CI (#2838) 2024-02-12 20:15:16 +01:00
mmetc df159b0167
update calls to deprecated x509 methods (#2824) 2024-02-09 13:55:24 +01:00
mmetc fdc525164a
refact "cscli metrics" part 3 (#2807) 2024-02-06 10:07:05 +01:00
mmetc 4160bb8102
refact "cscli decisions" (#2804)
* refact "cscli decisions"
* CI: relax mysql test timing
* lint
2024-02-01 22:36:21 +01:00
mmetc 17db4cb970
refact "cscli machines" (#2777) 2024-02-01 17:22:52 +01:00
Manuel Sabban 3f9e8e81e6
fix some bats tests (#2775) 2024-01-24 19:51:55 +01:00
mmetc 455acf7c90
lapi/papi: when receiving alerts, log and discard invalid addr/range (#2708)
https://github.com/crowdsecurity/crowdsec/issues/2687
2024-01-22 12:24:26 +01:00
mmetc ce32fc019e
func tests improvements (#2759)
* faster reload test
* decode-jwt script
* replace 'netcat' requirement with python script
* fix lapi status test
2024-01-19 13:55:28 +01:00
mmetc 1e0bcedef5
Ignore missing console/context.yaml if not explicitly required by config.yaml (#2726) 2024-01-12 16:29:04 +01:00
mmetc fca8883cd9
cscli capi status -> message for missing credentials (#2730)
* cscli capi status -> message for missing credentials
* lint
2024-01-12 14:41:36 +01:00
mmetc 6960419a2e
Remove redundant file check for capi_whitelists_path (#2728) 2024-01-12 14:17:01 +01:00
mmetc 260f5a7992
pkg/cwhub: improve error messages (#2712)
* pkg/cwhub: improve error messages
* lint
2024-01-11 10:28:58 +01:00
mmetc a504113186
lint (wsl) (#2692) 2024-01-03 10:55:41 +01:00
mmetc 2a2b09b52a
cwhub: install --force repair tainted, non-installed items (#2686) 2024-01-03 10:08:45 +01:00
mmetc ca784b147b
test and log fixes (#2690)
* cscli inspect: suggest --diff if an item is tainted
* appropriate warning, or error if context configuration file is empty
* fix user/group lookup unit test
* fix: allow hub upgrade --force with local items
* fix pkg/parser lookup for 8.8.8.8
* fix func test
* fix hubtests: machines add --force
2024-01-03 09:33:52 +01:00
mmetc 1530d93fc1
Update localstack services + loki (dev and CI) (#2649) 2023-12-19 15:27:04 +01:00
mmetc 44eb4d4a94
Makefile: "make help" target, remove obsolete "notification-email" target (#2282) 2023-12-18 10:13:08 +01:00
mmetc c2c173ac7e
Parallel hubtests (#2667)
* generate hub tests in python
* run hub tests in 3 batches at the same time (hardcoded)
2023-12-15 18:30:20 +01:00
mmetc a79fcaf378
Add "taintedBy" and "--diff" flag to cscli... inspect (#2665)
* "cscli inspect" reports tainted sub-items
* cscli... inspect --diff
* unified diff
* option --diff --rev
* tainted message
* correctly report multiple taint reasons
2023-12-15 15:27:22 +01:00
mmetc a851e14c88
improve deprecation message with file location (#2662)
* better "lapi context" messages
* func tests: include all items in hub_purge_all
* docker + tests: update yq
2023-12-14 16:11:11 +01:00
mmetc 89f704ef18
light pkg/api{client,server} refact (#2659)
* tests: don't run crowdsec if not necessary
* make listen_uri report the random port number when 0 is requested
* move apiserver.getTLSAuthType() -> csconfig.TLSCfg.GetAuthType()
* move apiserver.isEnrolled() -> apiclient.ApiClient.IsEnrolled()
* extract function apiserver.recoverFromPanic()
* simplify and move APIServer.GetTLSConfig() -> TLSCfg.GetTLSConfig()
* moved TLSCfg type to csconfig/tls.go
* APIServer.InitController(): early return / happy path
* extract function apiserver.newGinLogger()
* lapi tests
* update unit test
* lint (testify)
* lint (whitespace, variable names)
* update docker tests
2023-12-14 14:54:11 +01:00
mmetc 67cdf91f94
Short build tag in version number (#2658)
* use short commit hash in version number
* var -> const
* cscli: extract version.go, doc.go
* don't repeat commit hash in version number
2023-12-14 09:16:38 +01:00
mmetc 12d9fba4b3
cscli machines: lint + write output to stdout instead of log (#2657)
* feedback on stdout, not log.Info
* rename parameters to silence warnings from "unusedparams"
* debian postinst: skip duplicate warnings with 'cscli machines add'
* rpm postinst: skip duplicate warnings in 'cscli machines add'
* update func tests
* debian prerm: if dashboard remove fails, explain it's ok
* debian prerm: suppress warnings about wal, capi when attempting to remove the dashboard
* wizard.sh: log format like crowdsec
2023-12-13 15:43:46 +01:00
mmetc 1b4ec66148
fix package tests for 1.5.6-rc2 (#2652)
* fix go deps for test
* fix package tests for 1.5.6-rc2
* fix context func tests
* docker: pin build image version for alpine

---------

Co-authored-by: sabban <github@sabban.eu>
2023-12-11 10:07:09 +01:00
blotus 04f3dc09f9
remove PAPI feature flag (#2601) 2023-12-08 14:55:45 +01:00
mmetc 4acb4f8df3
cwhub: context type (#2631)
* add hub type "context"
* cscli lapi: log.Fatal -> fmt.Errorf; lint
* tests for context.yaml
* load console context from hub
* original & compiled context
* deprecate "cscli lapi context delete"
$ cscli lapi context delete
Command "delete" is deprecated, please manually edit the context file.
* cscli completion: add appsec-rules, appsec-configs, explain, hubtest
2023-12-07 16:20:13 +01:00
Thibault "bui" Koechlin 8cca4346a5
Application Security Engine Support (#2273)
Add a new datasource that:
- Receives HTTP requests from remediation components
- Apply rules on them to determine whether they are malicious or not
- Rules can be evaluated in-band (the remediation component will block the request directly) or out-band (the RC will let the request through, but crowdsec can still process the rule matches with scenarios)

The PR also adds support for 2 new hub items:
- appsec-configs: Configure the Application Security Engine (which rules to load, in which phase)
- appsec-rules: a rule that is added in the Application Security Engine (can use either our own format, or seclang)

---------

Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-12-07 12:21:04 +01:00
mmetc 90d3a21853
CI: use go 1.21.5 (#2640)
* use go 1.21.5
* Simpler go:build directives
2023-12-06 12:38:36 +01:00
mmetc 1ab4487b65
cscli hub list: show only non-empty tables with -o human
* agent config: remove unused LintOnly bool
* Item.IsLocal() -> Item.State.IsLocal(); split method InstallStatus()
* cscli hub list: show only non-empty tables with -o human
2023-12-05 13:38:52 +01:00
mmetc 0f3ae64062
cscli config show: pretty print with package "litter" (#2633) 2023-12-05 10:38:21 +01:00
mmetc 0c4093dcca
Test for acquisition errors in crowdsec -t (#2629) 2023-12-04 23:09:42 +01:00
mmetc a5ab73d458
cscli machines add: don't overwrite existing credential file (#2625)
* cscli machines add: don't overwrite existing credential file
* keep old behavior with --force
Now --force is used both to override the replacement of and existing machine,
and an existing credentials file. To retain the old behavior, the
existence of the file is only checked for the default configuration, not
if explicitly specified.
2023-12-04 22:59:52 +01:00
mmetc 7e5ab344a2
command "cscli hub types" (#2632)
* Command "cscli hub types"; de-duplicate test/bin/preload-hub-items
* don't export Hub.Items -> hub.items
2023-12-01 09:36:38 +01:00
mmetc 6b0bdc5eeb
Refact pkg/cwhub: fix some known issues and reorganize files (#2616)
* bump gopkg.in/yaml.v3
* test: cannot remove local items with cscli
* test dangling links
* test: cannot install local item with cscli
* pkg/cwhub: reorg (move) functions in files
* allow hub upgrade with local items
* data download: honor Last-Modified header
* fatal -> warning when attempting to remove a local item (allows remove --all)
* cscli...inspect -o yaml|human: rename remote_path -> path
* Correct count of removed items
Still no separate counter for the --purge option, but should be clear enough
2023-11-28 23:51:51 +01:00
mmetc ffcab0b2bc
Refactor hub management and cscli commands (#2545) 2023-11-24 15:57:32 +01:00
mmetc 76d4bc7788
cscli bouncers: increase key size, deprecate and ignore --length option (#2531)
the switch to base64 made the keys shorter (24 characters), this PR increases their size to 32 bytes, 42 chars once encoded

Also deprecate the --length option, users can already provide a key
2023-11-24 15:01:13 +01:00
lperdereau 92f923cfa8
Loki integration #2 (#2306)
* Add support for loki datasource

---------

Co-authored-by: Mathieu Lecarme <mathieu@garambrogne.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
2023-11-22 13:31:39 +01:00
mmetc 47eb2e240d
Use go 1.21.4 (#2595) 2023-11-16 11:09:13 +01:00
guangwu ddd6ee8e42
fix: typo (#2582)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-11-08 09:26:34 +01:00
Manuel Sabban 4934fce769
update gantsign.golang name (#2558) 2023-11-07 14:53:14 +01:00
mmetc a254b436c7
use go 1.12.3 (#2535) 2023-10-12 16:28:24 +02:00
mmetc 61d4ccbfdd
use go 1.21.1 (#2418)
* use go 1.21.1, require 1.21
* import "slices" from stdlib
* allow codeql to set version number from tags
* codeql: custom WASM build - the automated one can silently fail
2023-10-04 13:01:57 +02:00
mmetc bfda483c0a
fix issue #2499 - nil dereference while using capi whitelists (#2501) 2023-10-02 11:42:17 +02:00