beenull/crowdsec
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
1597 commits 83 branches 177 tags 151 MiB
Commit graph

432 commits

Author SHA1 Message Date
Laurence Jones 05c1825622
Add to dump after postoverflow so we can test within hubtest (#2511) 
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
 2023-11-28 13:18:41 +00:00
Laurence Jones 6a61b919e7
[cscli] notifications test command and slight re write (#2391) 
* Merge main and apply stash

* Rework some of cscli notif stuff and add a generic test which works with non active profiles

* Update wording

* Fix merge

* Final version

* Cleanup
 2023-11-28 13:17:54 +00:00
mmetc ffcab0b2bc
Refactor hub management and cscli commands (#2545) 2023-11-24 15:57:32 +01:00
mmetc 76d4bc7788
cscli bouncers: increase key size, deprecate and ignore --length option (#2531) 
the switch to base64 made the keys shorter (24 characters), this PR increases their size to 32 bytes, 42 chars once encoded

Also deprecate the --length option, users can already provide a key
 2023-11-24 15:01:13 +01:00
mmetc ec199162dc
iso8601: use yyyy-mm-dd in log timestamps instead of dd-mm-yyyy (#2564) 
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
 2023-11-24 14:59:28 +01:00
mmetc 5cd4406f5e
typos/grammar (#2561) 2023-11-07 15:07:36 +01:00
mmetc a6b55f2b5e
cscli config feeature-flags: point user to the right location of feature.yaml (#2539) 2023-10-13 09:52:51 +02:00
mmetc 3b1563a538
Refact cscli hub / pkg/cwhub (part 6) (#2524) 
* hub.ConfigDir -> hub.InstallDir; hub.DataDir -> hub.InstallDataDir
* cleanup GetInstalledItemsAsString()
* lint: ReferenceMissingError -> ErrMissingReference
* lint: parent_dir -> parentDir
* link: export Walker type
* lint: return error last
* lint: shadow
* move around and group variable definitions
 2023-10-09 21:33:35 +02:00
Manuel Sabban 6e228f3f3f
pkg/cwhub: cleanup in argument call (#2527) 
* cleanup in argument call
* update test as well
* cwhub_tests: reduce verbosity and use helpers

---------

Co-authored-by: Marco Mariani <marco@crowdsec.net>
 2023-10-09 13:26:34 +02:00
mmetc 9ae8bd79c5
Refact pkg/csconfig tests (#2526) 
* remove unused method
* whitespace, redundant comments
* use test helpers
* move DumpConsoleConfig() from pkg/csconfig to cscli
* package doc header
* var -> const
* rename ./tests -> ./testdata
* shorter tests with more error checks
* lint/formatting
* use helpers; fix tests that didn't actually test
* lint; rename expectedResult -> expected
 2023-10-09 11:10:51 +02:00
mmetc 338141f067
Refact cscli hub / pkg/cwhub (part 5) (#2521) 
* remove unused yaml tags
* cscli/cwhub: deduplicate, remove dead code
* log.Fatal -> fmt.Errorf
* deflate utils.go by moving functions to respective files
* indexOf() -> slices.Index()
* ItemStatus() + toEmoji() -> Item.status()
* Item.versionStatus()
* move getSHA256() to loader.go
 2023-10-06 13:59:51 +02:00
mmetc 9235f55c47
Refact pkg/cwhub (part 4) (#2518) 
* generalize function: GetInstalledItems, GetInstalledItemsAsString
* extracted function itemKey, happy path
* review comments / remove redundant; rename file to remove build tags
* remove unused fields in Item struct
* unix build tag
 2023-10-05 09:35:03 +02:00
mmetc 61d4ccbfdd
use go 1.21.1 (#2418) 
* use go 1.21.1, require 1.21
* import "slices" from stdlib
* allow codeql to set version number from tags
* codeql: custom WASM build - the automated one can silently fail
 2023-10-04 13:01:57 +02:00
mmetc 5618ba9f46
cscli: refactor hub commands (#2500) 2023-10-04 10:42:47 +02:00
mmetc cba6de024f
cscli: restore config correctly if acquis.d already exists (#2504) 2023-10-02 13:31:04 +02:00
mmetc 3cb9dbdb21
notification-email: configurable timeouts (#2465) 
* configurable timeouts
* parse email timeouts as duration string
* add helo_host to email.yaml
* move html and body tags outside of the loops
* added quotes to href=.., and formatting test
 2023-09-29 16:59:06 +02:00
mmetc 95ed308207
cscli setup: accept stdin; fix proftpd detection test and service unmask (#2496) 2023-09-29 12:58:35 +02:00
Laurence Jones 9dba6db676
add alert alias (#2485) 2023-09-23 19:35:02 +01:00
mmetc d45bec4047
minor log message improvements (#2455) 2023-09-12 11:04:56 +02:00
Laurence Jones 702da0f59a
[enhancement] cscli explain --labels (#2461) 
* Add label support for explain and allow user to provide multiple labels

* Change my mind about empty string

* Add debug and im an idiot 😄
 2023-09-11 14:18:04 +01:00
mmetc fd94e2c056
refactor alert/decisions insert/update to avoid database locking in bulk operations (#2446) 2023-09-04 14:21:45 +02:00
mmetc 22146eb3e4
fix "cscli console disable --all"; cleanup "cscli console" command (#2444) 2023-08-29 11:44:23 +02:00
mmetc b562103024
Make: build with debug symbols in func tests or if DEBUG=1; drop BUILD_VENDOR_FLAGS (#2443) 2023-08-28 15:58:26 +02:00
mmetc 2aa55e9444
move plugins/notifications/* to cmd/notification-* (#2429) 
This ensures keeping all dependencies in sync, and simplifies
packaging under freebsd/gentoo/etc because there is a single
vendor directory.
 2023-08-24 09:46:25 +02:00
mmetc e36df40ba7
pkg/types cleanup (#2398) 
* move function GetLineCountForFile from pkg/types to cscli
* move ParseDuration from pkg/types to pkg/database
* remove unused types.Profile, types.RemediationProfile
 2023-08-24 09:44:46 +02:00
Rasmus b4d9223625
Update main.go (#2431) 2023-08-22 14:09:53 +02:00
Efren 1ec52431b6
Remove duplicate line (#2432) 
Remove duplicate line (60) which is outputting `Configuration Folder` twice in `cscli config show`
 2023-08-22 14:09:19 +02:00
mmetc e8e2ade8f0
remove calls to log.Fatal (#2399) 
* remove log.Fatal from scenarios.go
* remove log.Fatal from collections.go
* remove log.Fatal from parsers.go and postoverflows.go
 2023-08-16 21:04:46 +02:00
mmetc caaed7c515
Timeout on shutdown while waiting for events to be flushed (#2423) 2023-08-16 21:03:15 +02:00
AlteredCoder 31c5727a90
Simplify context add (#2408) 2023-08-04 16:50:35 +02:00
mmetc 644c767019
cscli decisions list -o json => [] instead of null; same for alerts (#2397) 2023-08-03 12:51:50 +02:00
Laurence Jones 6ba682a32f
Update bouncers.go (#2404) 
Fix wrong short
 2023-08-03 11:26:08 +01:00
Laurence Jones a18df9c3bb
Add bouncers prune command (#2379) 
* Add bouncers prune command

* No point overloading functions

* Add prune to list of commands

* change all short desc to be similar, and made it really really clear when pruning it is not recoverable

* Dont use log. and dont return error on user input to abort
 2023-07-28 15:37:39 +01:00
mmetc ffadd42779
update dependency on go-cs-lib; drop the pkg/ part (#2393) 2023-07-28 16:35:08 +02:00
Laurence Jones 55247cd46a
Add machines prune command (#2011) 
* Add machines prune command

* Fix scope variable for naming scheme

* Add some freshness and add new features

* Fix force and fix duration if less than 60

* Allow duration to be more readable

* Fix description

* Improve func wording and make int machines length

* No point overloading functions

* Add prune to list of commands

* Check if GID is already the group if so no need to chown

* Revert "Check if GID is already the group if so no need to chown"

This reverts commit c7cef1773e.

* change all short desc to be similar, and made it really really clear when pruning it is not recoverable

* Better examples

* Match bouncer like for like

* Fix merge error

* Dont use log. and dont return error on user input to abort
 2023-07-28 15:23:47 +01:00
mmetc ae53c0f1cc
fix "crowdsec-cli/require" log verbosity (#2390) 2023-07-28 09:56:20 +02:00
mmetc 5cb7013575
Check cscli preconditions with crowdsec-cli/require package (#2388) 2023-07-27 17:02:20 +02:00
mmetc a01ce18b98
replace imports of path with path/filepath (#2330) 2023-07-26 10:29:58 +02:00
mmetc 1a6f12c88e
Build target for "make tidy" (#2378) 
The make tidy target runs "go mod tidy" in the root directory and all plugins.
 2023-07-26 10:24:37 +02:00
Laurence Jones 389ea4293f
Add metabase version override and update (#2370) 
* Add version override and update

* Ooppsie

* Quick fix

* fgs copilot

* Allow user to overwrite image, add warning for exposing metabase and general cleanup

* One ix

* Default image if not found in config, and add a warning to remove and update

* Reorder check system memory checks so it inline with @mmetc best pratices

* No need for err

* Clean up some group code

* Change ipv6 as [] seems to wildcard

* Split loopback warn and disclaimer. Add force yes to start to allow user to accept disclaimer by default

* All cmd commands are RunE clean up

* Update flag name and dont allow a shorthand
 2023-07-25 14:21:25 +01:00
mmetc 4bc225f26b
change output of "cscli metrics -o [json|raw]" from list of objects to map with table names (#2375) 2023-07-25 13:33:50 +02:00
mmetc b6b6fd026b
typo fix, uppercase 'API', adjusted log level (#2361) 2023-07-21 23:23:24 +02:00
Manuel Sabban 9ac5aeda79
fix the ci by adding the ability to enforce event ordering (#2347) 
* fix the ci by adding the ability to enforce event ordering
 2023-07-20 11:41:30 +02:00
mmetc 3c16139c44
Reduce log verbosity at startup (#2363) 
A configuration syntax test is performed every time the service is
started from systemd. The resulting error, if any, is shown on
journalctl logs.
This PR removes the unnecessary output in crowdsec.log generated by the
configuration test.
 2023-07-19 13:28:52 +02:00
mmetc bb16552aca
Use same levenshtein package for cscli, ent, hcl (#2359) 
remove one dependency, slightly smaller binary
 2023-07-18 11:30:14 +02:00
mmetc 9967d60987
errors.Wrap -> fmt.Errorf (#2333) 2023-07-06 10:14:45 +02:00
mmetc 85839b0199
support for stdin with "cscli decision import" and raw values (#2291) 
and remove Origin from the struct, which was ignored anyway
 2023-06-27 14:29:42 +02:00
mmetc a910b7beca
non-fatal error if some datasource can't be run (i.e. journalctl but systemd is missing) (#2309) 
This on the other hand, gives a new fatal error when there are no valid datasources.
In the previous version, crowdsec kept running with just a warning if no
acquisition yaml or dir were specified.
 2023-06-27 10:13:13 +02:00
mmetc 62caffb102
update leakybucket readme (#2298) 2023-06-22 15:35:01 +02:00
mmetc fddf597040
errors.Wrap -> fmt.Errorf; clean up imports (#2297) 2023-06-22 15:01:34 +02:00
mmetc 25bb23d8b7
minor refactor to pkg/types, cscli machines (#2270) 
* cleanup: separate ui and logic
* trim some code from pkg/types
 2023-06-08 15:08:51 +02:00
mmetc 6096cb3c9b
Move grok_pattern.go away from pkg/types to trim bouncer dependencies (#2269) 2023-06-08 15:07:30 +02:00
mmetc 4e2c9c185b
Implement "crowdsec -fatal" flag; change help message (#2266) 
The -trace...-fatal flags do not change the log destination but only the
verbosity. This change reflects that, and implements "-fatal" which was missing.
 2023-06-08 15:06:06 +02:00
mmetc 8da9d5eefd
don't log notification error if not running under systemd (#2274) 2023-06-08 15:04:48 +02:00
mmetc 3cc6b2c0d0
CI: add tests for metrics configuration (#2251) 2023-06-05 23:17:30 +02:00
mmetc 9ccdddaab1
CI: refactor makefile for plugins and vendor target (#2256) 2023-06-05 23:15:18 +02:00
mmetc 2a8e97d558
show option -winsvc only under windows (#2258) 2023-06-05 13:49:31 +02:00
mmetc 228e4f9acc
cscli: add G (1e9) suffix to metric units (#2254) 2023-06-02 14:38:11 +02:00
mmetc 396dcf8e6e
dependencies: replaced function calls to pkg/types, errors.Wrap (#2235) 
we now use a generic pointer function, and slowly remove the deprecated pkg/errors
 2023-06-01 16:31:56 +02:00
mmetc 92a9d6c321
types.InSlice() -> slices.Contains() (#2246) 2023-05-31 12:39:22 +02:00
mmetc 9167bd107d
decouple bouncer dependencies: use go-cs-lib/pkg/ptr (#2228) 2023-05-25 15:43:39 +02:00
mmetc 1f9f81da70
makefiles: de-duplicate, simplify and remove unused code (#2222) 2023-05-25 10:32:05 +02:00
mmetc 534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* (#2216) 
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
 2023-05-23 10:52:47 +02:00
blotus 6e3ca35941
fallback to master for hub index download if it does not exist (#2210) 2023-05-17 11:20:53 +02:00
mmetc e1f5ed41df
Implement "cscli config show-yaml" (#2191) 2023-05-11 21:01:13 +02:00
Thibault "bui" Koechlin 5ac33aab03
allow batching when importing decisions (#2192) 2023-05-11 14:33:18 +02:00
mmetc 6b744884b0
Update deps to latest stable: go-systemd, tail, cobra, lumberjack, testify (#2164) 2023-04-12 16:58:11 +02:00
mmetc 0c5d233563
Minor cleanup and dead code removal (#2166) 2023-04-12 16:57:38 +02:00
blotus 1e018bdaf8
Wait for both api and agent chans if necessary when daemonize is false or running on windows (#2155) 2023-04-04 15:16:48 +02:00
mmetc 38ab6be7c2
Allow feature.yml to change available subcommands (#2156) 2023-04-03 10:11:56 +02:00
mmetc ea6401ce09
CI: Static builds by default; replace bincover with go -cover from 1.20 (#2150) 
* Makefile: build static binaries only
* Replace bincover with go -cover from 1.20
* CI: Fix timing issue between lapi and agent containers
 2023-03-30 15:05:09 +02:00
blotus 61bea26486
Add transform configuration option for acquisition (#2144) 2023-03-29 16:04:17 +02:00
blotus 1095f6c875
use expr.Function for custom functions instead of passing them in the env (#2133) 2023-03-28 10:49:01 +02:00
blotus 91eb39cff6
New PAPI commands: reauth + force_pull (#2129) 2023-03-21 14:06:19 +01:00
Thibault "bui" Koechlin a74e424d53
support ip and cidr based whitelists for capi and 3rd party blocklists (#2132) 
* support ip and cidr based whitelists for capi and 3rd party blocklist
 2023-03-21 11:50:10 +01:00
AlteredCoder e61a464951
Fix cscli explain when running from testenv (#2114) 
* Fix cscli explain when running from testenv
 2023-03-15 10:26:40 +01:00
mmetc e161507d08
Lint (type inference): remove redundant type declarations (#2111) 2023-03-09 11:56:02 +01:00
mmetc 9faa49c7e8
Load lapi config for config show output (#2097) 
This adds URL and login parameters as it was intended.
Also rewrite configShow and displayOneAlert to use an embedded text/template for shorter code.
 2023-03-08 22:47:25 +01:00
Thibault "bui" Koechlin 9d5aaf5ea2
add --origin to cscli decisions delete (#2109) 2023-03-08 18:29:20 +01:00
Thibault "bui" Koechlin 5b0fe4b7f1
support for regexps result cache (#2104) 
* support for regexps result cache : gcache + xxhash

Co-authored-by: Marco Mariani <marco@crowdsec.net>
 2023-03-08 16:07:49 +01:00
blotus 16a3be49e2
do not try to load PAPI is url is not set (#2099) 2023-03-06 15:38:58 +01:00
blotus e27a0a0e14
display source in alerts list when an alert has multiple decisions (#2098) 2023-03-06 13:51:57 +01:00
blotus b2c2c5ac59
add papi_url in credentials file when enabling console_management, and remove it when disabling console_management (#2095) 2023-03-03 17:03:21 +01:00
blotus 85ab9c68a2
Add cscli papi status and cscli papi sync (#2091) 2023-03-03 13:46:28 +01:00
mmetc f6d6c5bb2b
Add tests and typo fixes (#2092) 2023-03-03 11:06:27 +01:00
mmetc a6bb2cf5e1
Fix log destination in one-shot mode (#2084) 2023-03-01 17:00:04 +01:00
Manuel Sabban 60b3f63851
ugly workaround to fix the tests (#2080) 
* ugly workaround to fix the tests

* add comments

---------

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
 2023-02-28 17:05:11 +01:00
Manuel Sabban 39a4a256fd
fix the way acquisition is stopped (#2069) 
* fix the way acquisition is stopped by draining inputLineChan before terminating it.

---------

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
 2023-02-27 11:21:25 +01:00
Laurence Jones 75d8b821ff
Explain successful parsers only (#2063) 
* Add option to filter down explain to successful parsers useful for me who has every collection installed

* Altered naming conventions so it makes more sense when reading
 2023-02-24 13:49:17 +00:00
mmetc b7d1e2c483
replace log.Fatal -> fmt.Errorf (#2058) 2023-02-20 15:05:42 +01:00
blotus 83c3818504
Do not try to refresh JWT token when doing a login request (#2059) 2023-02-16 16:16:26 +01:00
Thibault "bui" Koechlin 1d7d377f8b
changes following BL tests (#2038) 
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-02-08 10:35:21 +01:00
Cristian Nitescu 987f119c4b
v3 capi and blocklists links support (#2019) 
* v3 model generation

* v3 model generation

* comms

* fixes after master merge

* missing reader close

* use constants defined for types

---------

Co-authored-by: bui <thibault@crowdsec.net>
 2023-02-06 14:06:14 +01:00
mmetc b6be18ca65
cscli setup (#1923) 
Detect running services and generate acquisition configuration
 2023-02-06 07:33:04 +01:00
Thibault "bui" Koechlin e927717fa0
Polling API Integration (#1715) 
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-01-31 14:47:44 +01:00
mmetc d369656b26
agent: fix message when -dsn is provided without -type (#2009) 2023-01-20 16:14:26 +01:00
mmetc e5833699c0
cscli config feature-flags (#2006) 2023-01-20 09:32:10 +01:00
mmetc 4bffc0df21
break in smaller functions cscli hub, hubtest, notifications, parsers, scenarios, simulation (#2004) 2023-01-19 13:29:36 +01:00
mmetc 7bb74b9664
refact cscli decisions (#2003) 2023-01-19 11:02:00 +01:00
Thibault "bui" Koechlin 4f29ce2ee7
CTI API Helpers in expr (#1851) 
* Add CTI API helpers in expr
* Allow profiles to have an `on_error` option to profiles

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-01-19 08:45:50 +01:00