Commit graph

110 commits

Author SHA1 Message Date
mmetc fca8883cd9
cscli capi status -> message for missing credentials (#2730)
* cscli capi status -> message for missing credentials
* lint
2024-01-12 14:41:36 +01:00
mmetc 6960419a2e
Remove redundant file check for capi_whitelists_path (#2728) 2024-01-12 14:17:01 +01:00
mmetc 89f704ef18
light pkg/api{client,server} refact (#2659)
* tests: don't run crowdsec if not necessary
* make listen_uri report the random port number when 0 is requested
* move apiserver.getTLSAuthType() -> csconfig.TLSCfg.GetAuthType()
* move apiserver.isEnrolled() -> apiclient.ApiClient.IsEnrolled()
* extract function apiserver.recoverFromPanic()
* simplify and move APIServer.GetTLSConfig() -> TLSCfg.GetTLSConfig()
* moved TLSCfg type to csconfig/tls.go
* APIServer.InitController(): early return / happy path
* extract function apiserver.newGinLogger()
* lapi tests
* update unit test
* lint (testify)
* lint (whitespace, variable names)
* update docker tests
2023-12-14 14:54:11 +01:00
blotus 04f3dc09f9
remove PAPI feature flag (#2601) 2023-12-08 14:55:45 +01:00
mmetc 4acb4f8df3
cwhub: context type (#2631)
* add hub type "context"
* cscli lapi: log.Fatal -> fmt.Errorf; lint
* tests for context.yaml
* load console context from hub
* original & compiled context
* deprecate "cscli lapi context delete"
$ cscli lapi context delete
Command "delete" is deprecated, please manually edit the context file.
* cscli completion: add appsec-rules, appsec-configs, explain, hubtest
2023-12-07 16:20:13 +01:00
Thibault "bui" Koechlin 8cca4346a5
Application Security Engine Support (#2273)
Add a new datasource that:
- Receives HTTP requests from remediation components
- Apply rules on them to determine whether they are malicious or not
- Rules can be evaluated in-band (the remediation component will block the request directly) or out-band (the RC will let the request through, but crowdsec can still process the rule matches with scenarios)

The PR also adds support for 2 new hub items:
- appsec-configs: Configure the Application Security Engine (which rules to load, in which phase)
- appsec-rules: a rule that is added in the Application Security Engine (can use either our own format, or seclang)

---------

Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-12-07 12:21:04 +01:00
mmetc 1ab4487b65
cscli hub list: show only non-empty tables with -o human
* agent config: remove unused LintOnly bool
* Item.IsLocal() -> Item.State.IsLocal(); split method InstallStatus()
* cscli hub list: show only non-empty tables with -o human
2023-12-05 13:38:52 +01:00
mmetc ffcab0b2bc
Refactor hub management and cscli commands (#2545) 2023-11-24 15:57:32 +01:00
mmetc 5cd4406f5e
typos/grammar (#2561) 2023-11-07 15:07:36 +01:00
mmetc a6b55f2b5e
cscli config feeature-flags: point user to the right location of feature.yaml (#2539) 2023-10-13 09:52:51 +02:00
mmetc 3b1563a538
Refact cscli hub / pkg/cwhub (part 6) (#2524)
* hub.ConfigDir -> hub.InstallDir; hub.DataDir -> hub.InstallDataDir
* cleanup GetInstalledItemsAsString()
* lint: ReferenceMissingError -> ErrMissingReference
* lint: parent_dir -> parentDir
* link: export Walker type
* lint: return error last
* lint: shadow
* move around and group variable definitions
2023-10-09 21:33:35 +02:00
mmetc 9ae8bd79c5
Refact pkg/csconfig tests (#2526)
* remove unused method
* whitespace, redundant comments
* use test helpers
* move DumpConsoleConfig() from pkg/csconfig to cscli
* package doc header
* var -> const
* rename ./tests -> ./testdata
* shorter tests with more error checks
* lint/formatting
* use helpers; fix tests that didn't actually test
* lint; rename expectedResult -> expected
2023-10-09 11:10:51 +02:00
mmetc 338141f067
Refact cscli hub / pkg/cwhub (part 5) (#2521)
* remove unused yaml tags
* cscli/cwhub: deduplicate, remove dead code
* log.Fatal -> fmt.Errorf
* deflate utils.go by moving functions to respective files
* indexOf() -> slices.Index()
* ItemStatus() + toEmoji() -> Item.status()
* Item.versionStatus()
* move getSHA256() to loader.go
2023-10-06 13:59:51 +02:00
mmetc bfda483c0a
fix issue #2499 - nil dereference while using capi whitelists (#2501) 2023-10-02 11:42:17 +02:00
mmetc fd94e2c056
refactor alert/decisions insert/update to avoid database locking in bulk operations (#2446) 2023-09-04 14:21:45 +02:00
mmetc 25868f27de
option db_client.decision_bulk_size (#2440) 2023-08-25 17:05:17 +02:00
mmetc afeb541eac
apic: minor refactoring (#2415)
* apic: minor refactoring

* Add whitelist length check

If user configures the file but fails to define and actual whitelist we should check length to save allocs

* Init with length from file

* extract loop method from ApplyApicWhitelists

* pass pointer

* extract loop method updateBlocklist

---------

Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
2023-08-10 13:03:47 +02:00
mmetc ffadd42779
update dependency on go-cs-lib; drop the pkg/ part (#2393) 2023-07-28 16:35:08 +02:00
mmetc ae53c0f1cc
fix "crowdsec-cli/require" log verbosity (#2390) 2023-07-28 09:56:20 +02:00
mmetc 5cb7013575
Check cscli preconditions with crowdsec-cli/require package (#2388) 2023-07-27 17:02:20 +02:00
mmetc 9beb5388cb
errors.Wrap -> fmt.Errorf; clean up imports (#2301) 2023-06-23 14:04:58 +02:00
mmetc 3cc6b2c0d0
CI: add tests for metrics configuration (#2251) 2023-06-05 23:17:30 +02:00
mmetc 396dcf8e6e
dependencies: replaced function calls to pkg/types, errors.Wrap (#2235)
we now use a generic pointer function, and slowly remove the deprecated pkg/errors
2023-06-01 16:31:56 +02:00
mmetc b2d3520519
decouple bouncer dependencies: use go-cs-lib in test code (#2229) 2023-05-25 15:37:44 +02:00
Laurence Jones 0416a41d58
Log info capi whitelists (#2220)
* add infof command if err was nil

* Fix golint

* Make message more readable and log individual stats

* Missed a d

* Remove '

* simplify if/else logic

---------

Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-05-25 10:28:08 +01:00
mmetc 025f14f879
merge system cert pool with own certs (#2226) 2023-05-25 10:10:58 +02:00
mmetc 534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* (#2216)
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
2023-05-23 10:52:47 +02:00
mmetc e1f5ed41df
Implement "cscli config show-yaml" (#2191) 2023-05-11 21:01:13 +02:00
mmetc 0c5d233563
Minor cleanup and dead code removal (#2166) 2023-04-12 16:57:38 +02:00
mmetc 38ab6be7c2
Allow feature.yml to change available subcommands (#2156) 2023-04-03 10:11:56 +02:00
Thibault "bui" Koechlin a74e424d53
support ip and cidr based whitelists for capi and 3rd party blocklists (#2132)
* support ip and cidr based whitelists for capi and 3rd party blocklist
2023-03-21 11:50:10 +01:00
blotus b1f2063a9a
Only support pgx driver for postgresql (#2118) 2023-03-16 11:02:31 +01:00
blotus 16a3be49e2
do not try to load PAPI is url is not set (#2099) 2023-03-06 15:38:58 +01:00
Laurence Jones 8acce4637a
Option to disable remote lapi registration (#2010)
* Allow to disable remote lapi registration

* Extract method and make it extendable as a generic middleware

* Change method name so it make sense to read abort remote if <config>

* golint
2023-02-24 13:44:21 +00:00
Thibault "bui" Koechlin e927717fa0
Polling API Integration (#1715)
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-31 14:47:44 +01:00
mmetc e37d09e5b4
use helpers for shorter tests, add a couple of error cases (#2016) 2023-01-26 17:13:31 +01:00
mmetc 47cc60bda9
allow use of literal $ in config.yaml (#2012) 2023-01-23 10:29:29 +01:00
Thibault "bui" Koechlin 4f29ce2ee7
CTI API Helpers in expr (#1851)
* Add CTI API helpers in expr
* Allow profiles to have an `on_error` option to profiles

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-19 08:45:50 +01:00
Marco Mariani 4f25738d6b wip 2023-01-18 15:15:18 +01:00
Marco Mariani 47dbfa770d configure logging earlier 2023-01-18 15:15:18 +01:00
Marco Mariani 91b0f8fee1 load custom configuration paths when agent is disabled 2023-01-18 15:15:18 +01:00
Marco Mariani 2e91a82aa7 load feature.yaml as soon as possible 2023-01-18 15:15:18 +01:00
mmetc 51800132cd
improve feature flag logging (#1986)
For cscli: it should provide a terse output, not nag users with configuration details. Although it's usually important that cscli and crowdsec have the same enabled features, having it list them every time the command is invoked can be too much.

For crowdsec: when features are set from the environment, it's too early to log where we should. So we can use log.Debug at activation time, and list them again once logging is configured.

 - wrap some functions in csconfig for convenience and DRY
 - for each enabled feature, log.Debug
 - log all enabled features once as Info (crowdsec) or Debug (cscli)
 - file does not exist -> log.Trace
2023-01-13 13:42:42 +01:00
mmetc cd4dabde0e
silence yaml.local explicitly in cscli, keep in crowdsec/bouncer logs (#1981) 2023-01-11 09:50:46 +01:00
AlteredCoder 185f9ad541
Alert context (#1895)
Co-authored-by: bui <thibault@crowdsec.net>
2023-01-04 16:50:02 +01:00
mmetc 72c1753fb7
fix tls communication with lapi and user/pw auth (#1956)
allow self-signed TLS encryption with user/pw auth

docker:
 - remove defaults for certificate file locations
 - new envvar INSECURE_SKIP_VERIFY
 - register agent before TLS settings (cscli machine add removes them
   from the credentials file)
2022-12-29 22:00:11 +01:00
Laurence Jones fe23da6e0c
Add postgres socket support, clean some code (#1926) 2022-12-12 16:08:19 +00:00
mmetc cc228f1868
Typos, grammar (#1905) 2022-12-06 15:55:27 +01:00
mmetc fd3e668fe1
add -error flag to crowdsec binary (#1903) 2022-12-03 08:56:11 +01:00
mmetc fa0e590778
removed pid_dir (#1906) 2022-12-02 13:42:43 +01:00