beenull/crowdsec
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
1405 commits 83 branches 177 tags 151 MiB
Commit graph

361 commits

Author SHA1 Message Date
blotus 61bea26486
Add transform configuration option for acquisition (#2144) 2023-03-29 16:04:17 +02:00
blotus 1095f6c875
use expr.Function for custom functions instead of passing them in the env (#2133) 2023-03-28 10:49:01 +02:00
blotus 91eb39cff6
New PAPI commands: reauth + force_pull (#2129) 2023-03-21 14:06:19 +01:00
Thibault "bui" Koechlin a74e424d53
support ip and cidr based whitelists for capi and 3rd party blocklists (#2132) 
* support ip and cidr based whitelists for capi and 3rd party blocklist
 2023-03-21 11:50:10 +01:00
AlteredCoder e61a464951
Fix cscli explain when running from testenv (#2114) 
* Fix cscli explain when running from testenv
 2023-03-15 10:26:40 +01:00
mmetc e161507d08
Lint (type inference): remove redundant type declarations (#2111) 2023-03-09 11:56:02 +01:00
mmetc 9faa49c7e8
Load lapi config for config show output (#2097) 
This adds URL and login parameters as it was intended.
Also rewrite configShow and displayOneAlert to use an embedded text/template for shorter code.
 2023-03-08 22:47:25 +01:00
Thibault "bui" Koechlin 9d5aaf5ea2
add --origin to cscli decisions delete (#2109) 2023-03-08 18:29:20 +01:00
Thibault "bui" Koechlin 5b0fe4b7f1
support for regexps result cache (#2104) 
* support for regexps result cache : gcache + xxhash

Co-authored-by: Marco Mariani <marco@crowdsec.net>
 2023-03-08 16:07:49 +01:00
blotus 16a3be49e2
do not try to load PAPI is url is not set (#2099) 2023-03-06 15:38:58 +01:00
blotus e27a0a0e14
display source in alerts list when an alert has multiple decisions (#2098) 2023-03-06 13:51:57 +01:00
blotus b2c2c5ac59
add papi_url in credentials file when enabling console_management, and remove it when disabling console_management (#2095) 2023-03-03 17:03:21 +01:00
blotus 85ab9c68a2
Add cscli papi status and cscli papi sync (#2091) 2023-03-03 13:46:28 +01:00
mmetc f6d6c5bb2b
Add tests and typo fixes (#2092) 2023-03-03 11:06:27 +01:00
mmetc a6bb2cf5e1
Fix log destination in one-shot mode (#2084) 2023-03-01 17:00:04 +01:00
Manuel Sabban 60b3f63851
ugly workaround to fix the tests (#2080) 
* ugly workaround to fix the tests

* add comments

---------

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
 2023-02-28 17:05:11 +01:00
Manuel Sabban 39a4a256fd
fix the way acquisition is stopped (#2069) 
* fix the way acquisition is stopped by draining inputLineChan before terminating it.

---------

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
 2023-02-27 11:21:25 +01:00
Laurence Jones 75d8b821ff
Explain successful parsers only (#2063) 
* Add option to filter down explain to successful parsers useful for me who has every collection installed

* Altered naming conventions so it makes more sense when reading
 2023-02-24 13:49:17 +00:00
mmetc b7d1e2c483
replace log.Fatal -> fmt.Errorf (#2058) 2023-02-20 15:05:42 +01:00
blotus 83c3818504
Do not try to refresh JWT token when doing a login request (#2059) 2023-02-16 16:16:26 +01:00
Thibault "bui" Koechlin 1d7d377f8b
changes following BL tests (#2038) 
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-02-08 10:35:21 +01:00
Cristian Nitescu 987f119c4b
v3 capi and blocklists links support (#2019) 
* v3 model generation

* v3 model generation

* comms

* fixes after master merge

* missing reader close

* use constants defined for types

---------

Co-authored-by: bui <thibault@crowdsec.net>
 2023-02-06 14:06:14 +01:00
mmetc b6be18ca65
cscli setup (#1923) 
Detect running services and generate acquisition configuration
 2023-02-06 07:33:04 +01:00
Thibault "bui" Koechlin e927717fa0
Polling API Integration (#1715) 
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-01-31 14:47:44 +01:00
mmetc d369656b26
agent: fix message when -dsn is provided without -type (#2009) 2023-01-20 16:14:26 +01:00
mmetc e5833699c0
cscli config feature-flags (#2006) 2023-01-20 09:32:10 +01:00
mmetc 4bffc0df21
break in smaller functions cscli hub, hubtest, notifications, parsers, scenarios, simulation (#2004) 2023-01-19 13:29:36 +01:00
mmetc 7bb74b9664
refact cscli decisions (#2003) 2023-01-19 11:02:00 +01:00
Thibault "bui" Koechlin 4f29ce2ee7
CTI API Helpers in expr (#1851) 
* Add CTI API helpers in expr
* Allow profiles to have an `on_error` option to profiles

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
 2023-01-19 08:45:50 +01:00
Marco Mariani 47dbfa770d configure logging earlier 2023-01-18 15:15:18 +01:00
Marco Mariani 91b0f8fee1 load custom configuration paths when agent is disabled 2023-01-18 15:15:18 +01:00
Marco Mariani 2e91a82aa7 load feature.yaml as soon as possible 2023-01-18 15:15:18 +01:00
Marco Mariani b603bdfccc cscli refact: extracted New.*Cmd from alerts, capi, dashboard; removed (some) globals 2023-01-18 11:09:28 +01:00
mmetc 51800132cd
improve feature flag logging (#1986) 
For cscli: it should provide a terse output, not nag users with configuration details. Although it's usually important that cscli and crowdsec have the same enabled features, having it list them every time the command is invoked can be too much.

For crowdsec: when features are set from the environment, it's too early to log where we should. So we can use log.Debug at activation time, and list them again once logging is configured.

 - wrap some functions in csconfig for convenience and DRY
 - for each enabled feature, log.Debug
 - log all enabled features once as Info (crowdsec) or Debug (cscli)
 - file does not exist -> log.Trace
 2023-01-13 13:42:42 +01:00
mmetc 157589d31e
cscli explain: add crowdsec path option (#1983) 2023-01-12 17:04:28 +01:00
Thibault "bui" Koechlin 6fb962a941
Allow parsers to capture data for future enrichment (#1969) 
* Allow parsers to capture data in a cache, that can be later accessed via expr helpers (fake multi-line support)
 2023-01-11 15:01:02 +01:00
mmetc cd4dabde0e
silence yaml.local explicitly in cscli, keep in crowdsec/bouncer logs (#1981) 2023-01-11 09:50:46 +01:00
mmetc c4deaf0994
cscli: avoid initializing the db configuration twice (#1982) 2023-01-11 09:50:12 +01:00
AlteredCoder 185f9ad541
Alert context (#1895) 
Co-authored-by: bui <thibault@crowdsec.net>
 2023-01-04 16:50:02 +01:00
mmetc 59f6610721
separate cscli cobra constructors: lapi, machines, bouncers, postoverflows (#1945) 2022-12-30 10:13:52 +01:00
mmetc 6efc2688b1
simplify feature flags (#1947) 
Now checking for a feature flag is a one liner,
with no need to control errors.

if fflag.Crowdsec.CscliSetup.IsEnabled() {
   ...
}
 2022-12-26 14:23:41 +01:00
mmetc c022eb1b86
remove ignored flag "-m" in "cscli machines delete" (it takes a positional argument) (#1943) 2022-12-23 17:13:20 +01:00
mmetc ef3a130d54
Cscli config refactoring (#1934) 2022-12-22 12:22:55 +01:00
mmetc 5d2c99bb17
runtime feature flag initialization 2022-12-21 17:19:20 +01:00
mmetc a32aa96752
feature flags (#1933) 
Package fflag provides a simple feature flag system.

 Feature names are lowercase and can only contain letters, numbers, undercores
 and dots.

 good: "foo", "foo_bar", "foo.bar"
 bad: "Foo", "foo-bar"

 A feature flag can be enabled by the user with an environment variable
 or by adding it to {ConfigDir}/feature.yaml

 I.e. CROWDSEC_FEATURE_FOO_BAR=true
 or in feature.yaml:
```
 ---
 - foo_bar
```

 If the variable is set to false, the feature can still be enabled
 in feature.yaml. Features cannot be disabled in the file.

 A feature flag can be deprecated or retired. A deprecated feature flag is
 still accepted but a warning is logged. A retired feature flag is ignored
 and an error is logged.

 A specific deprecation message is used to inform the user of the behavior
 that has been decided when the flag is/was finally retired.
 2022-12-20 16:11:51 +01:00
mmetc 6c19beb937
set cscli log timestamp to 24h (#1917) 2022-12-09 16:48:24 +01:00
blotus fdda940ac0
Add Kubernetes audit acquisition (#1767) 2022-12-06 13:47:29 +01:00
Manuel Sabban 3d72ca731a
Suggest bouncers and machines to delete (#1896) 
* Suggest bouncers to delete

* Autocomplete machines delete cmd

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* Fix lint.

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* fix compilation (git merge errors)

* cleanup go.mod unneeded changes

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Co-authored-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
 2022-12-05 15:39:54 +01:00
mmetc fd3e668fe1
add -error flag to crowdsec binary (#1903) 2022-12-03 08:56:11 +01:00
mmetc 104f5d1fe6
lint: error handling cleanup (#1877) 2022-11-29 09:16:07 +01:00
mmetc 99513f64fd
cscli config show: print host/port/user/dbname when driver=pgx (fix #1866) (#1870) 2022-11-17 11:07:37 +00:00
mmetc 3beb84bcfe
print missing "AS" values as empty strings instead of "0 " (#1867) 2022-11-14 09:55:53 +01:00
mmetc 895691dad1
enabled linters: gocritic, nilerr (#1853) 2022-11-07 10:36:50 +01:00
Thibault "bui" Koechlin 23ffa1e04f
add cscli alerts delete --id (#1843) 
* add cscli alerts delete by id

* test added for cscli delete alert --id

Co-authored-by: Marco Mariani <marco@crowdsec.net>
 2022-10-27 09:07:53 +02:00
Thibault "bui" Koechlin ae6bf39495
support decisions deletion via scenario + alerts delete via ID (#1798) 2022-10-19 14:37:27 +02:00
mmetc 2b7e3ff1e7
warn if no acquisition files are found, acquisition_test refactoring, tests (#1816) 2022-10-17 17:32:08 +02:00
mmetc a96b3e077d
rename pkg/cstest -> pkg/hubtest (#1811) 
keep cstest for generic helper functions
this also avoids circular imports in test files
 2022-10-17 09:24:07 +02:00
mmetc 8fecc2c00b
enable staticcheck linter; fixes (#1806) 
- explicitly ignore returned parameters
 - replace Walk with faster WalkDir
 - log path error during hub dir sync
 - colorize static unit tests
 - removed duplicate import in crowdsec/main.go
 - typos
 - func tests: default datasource in tests/var/log instead of /tmp
 - action setup-go v3
 2022-10-14 16:12:21 +02:00
mmetc 708fa8280a
add test and fix for configuration reload (#1808) 
fix reload issue by returning new configuration to the signal loop

example: run crowdsec, disable agent in the config file, reload config.
Now there is no agent but the signal loop believes there is, so
triggering a reload configuration again will make the process hang
forever.
This commit updates the configuration in the signal loop with the one
returned by the signal handler.
 2022-10-14 15:48:41 +02:00
blotus 7144dca68a
Fix missing metrics cscli (#1809) 2022-10-13 15:49:41 +02:00
mmetc 4b3c9c2806
print cscli usage in color, fix windows terminal detection (#1801) 2022-10-13 12:28:24 +02:00
Shivam Sandbhor 74659a82ab
Fast bulk alert delete (#1791) 2022-10-07 12:40:30 +02:00
mmetc ddd75eae9a
cscli: new tables, --color yes|no|auto option (#1763) 2022-10-07 11:05:35 +02:00
mmetc c920a301e0
make: accept BUILD_VENDOR_FLAGS variable (#1771) 2022-10-04 09:51:35 +02:00
Shivam Sandbhor 52447f6999
Don't suggest an item which user already mentioned. (#1702) 2022-09-28 13:26:47 +02:00
Sean Kelly 568eb1d4e0
Fix misspelling of instantiate participles (#1759) 2022-09-27 17:13:43 +02:00
mmetc 52fbda1a5e
simpler makefiles for static targets (#1744) 2022-09-14 14:22:57 +02:00
Manuel Sabban f3317f78d5
acually postoverflow _before_ reprocessing (#1739) 
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
 2022-09-07 14:16:47 +02:00
mmetc 414282a2c9
golangci-lint 1.49 and related fixes (#1736) 2022-09-06 13:55:03 +02:00
Manuel Sabban 7d0f89df29
Implement reinject command to send notifications of alerts (#1638) 
* implement reinject command to send notifications of alerts using a profile

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
 2022-08-30 15:45:52 +02:00
Manuel Sabban 21255b6391
fix #1724 (#1725) 
* fix #1724

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
 2022-08-30 12:38:28 +02:00
blotus e46ca38cbb
add cscli support dump (#1634) 2022-08-18 11:54:01 +02:00
AlteredCoder fe5f9bfc28
add suggestion on cscli install items (#1686) 2022-08-04 10:09:56 +02:00
blotus dacf6ebc64
Fix cscli notifications list crash (#1697) 2022-08-02 20:44:19 +02:00
AlteredCoder 1002affc16
cscli machines delete: return an error if machines doesn't exist (#1689) 
* cscli machines delete: return an error if machines doesn't exist
 2022-07-28 17:32:12 +02:00
AlteredCoder 16b1ab06a9
Add -a options in cscli alerts list (#1690) 
* Add -a options in cscli alerts list
 2022-07-28 17:31:53 +02:00
blotus 865ff5c88d
windows: log fatal and panic errors to event log (#1667) 2022-07-19 09:48:02 +02:00
AlteredCoder 24b1a99c42
Run cscli hubtest without being root (#1658) 2022-07-13 12:00:26 +02:00
blotus 8decbe7670
Properly handle service shutdown on windows (#1662) 2022-07-13 11:54:12 +02:00
AlteredCoder 5c8e2a8510
Fix 1652 (#1654) 2022-07-13 10:57:07 +02:00
Thibault "bui" Koechlin 920f3d2a7d
fix #1643 : deal with null heartbeat (#1645) 2022-07-07 15:29:30 +02:00
AlteredCoder 5f62d738fc
Add no-capi flag and review some logs (#1628) 
* Add no-capi flag and review some logs
 2022-07-01 16:56:13 +02:00
mmetc 8e7e799304
[wip] serve metrics only after agent and/or lapi are ready; fixed some func tests (#1613) 2022-06-24 15:55:21 +02:00
Laurence Jones 342e7f5272
Cscli Explain Stdin (#1597) 
Allow `cscli explain` to rely on stdin
 2022-06-23 11:51:43 +02:00
mmetc 628d7be1d8
simplify err.Error() to err when used in printf context (#1603) 2022-06-22 15:53:53 +02:00
he2ss 3d6f015211
Add duration expr to add duration formula (#1556) 
* add duration expr to add duration formula
 2022-06-22 11:29:52 +02:00
Thibault "bui" Koechlin a6ed08b239
Add alerts and decisions metrics, LAPI and agent timing prom metrics (#1546) 2022-06-22 11:14:34 +02:00
AlteredCoder 0a39066f9d
Fix #1552 (#1569) 2022-06-22 10:29:02 +02:00
Laurence Jones a1d5a02646
Cscli explain use temp dir (#1598) 
* Write to temp dir instead of CWD
 2022-06-22 10:27:43 +02:00
mmetc d71279f023
added flag crowdsec --warning (#1461) 2022-06-22 09:38:23 +02:00
mmetc c78c833400
CI: colored test output, colored crowdsec and crowdsec-api logs, full final db dump for mysql and sqlite (#1596) 
* github-ci: color unit test output and logs
* new config option: force_color_logs (useful in CI)
* bats: show sqlite/mysql dump at the end
* removed "-v" (print package names) from "go build"
* general workflow cleanup
 2022-06-17 16:12:49 +02:00
mmetc 10585bfecc
enabled linters and fixes for: misspell, predeclared, unconvert, ineffassign, gosimple, govet (#1595) 2022-06-16 14:41:54 +02:00
Thibault "bui" Koechlin 1c0fe09576
Add support for certificate authentication for agents and bouncers (#1428) 2022-06-08 16:05:52 +02:00
blotus 4b311684ab
Add more JSON expr helpers (#1576) 2022-06-08 12:15:29 +02:00
blotus b7f1c5455f
do not rely on /proc/sys/kernel/random/uuid as fallback as it does not exists everywhere (#1575) 2022-06-06 18:20:10 +02:00
mmetc 799cc82bb5
functional tests, minor refactoring and lint/cleanup (#1570) 
* cmd/crowdsec: removed log.Fatal()s, added tests and print error for unrecognized argument
* updated golangci-lint to v1.46
* lint/deadcode: fix existing issues
* tests: cscli config backup/restore
* tests: cscli completion powershell/fish
* err check: pflags MarkHidden()
* empty .dockerignore (and explain the reason)
* tests, errors.Wrap
* test for CS_LAPI_SECRET and minor refactoring
* minor style changes
* log cleanup
 2022-06-06 15:24:48 +02:00
mmetc df7c51f34e
fixed coverage reporting for functional tests; added cscli (#1568) 2022-05-31 10:01:30 +02:00
mmetc 1fc9587919
fix #1283: update and enable error reports from golangci (#1523) 2022-05-25 22:27:50 +02:00
mmetc 1a293a2a27
cwhub: export SetHubBranch (#1559) 2022-05-24 15:46:48 +02:00
Thibault "bui" Koechlin fe09737d80
Add support for machine heartbeat (#1541) 
* add the last_heartbeat field

* add heartbeat controller

* add endpoint of heartbeat

* heartbeat integration

* add last_heartbeat to cscli machines list
 2022-05-19 15:47:27 +02:00