mmetc
ffadd42779
update dependency on go-cs-lib; drop the pkg/ part ( #2393 )
2023-07-28 16:35:08 +02:00
Laurence Jones
55247cd46a
Add machines prune command ( #2011 )
...
* Add machines prune command
* Fix scope variable for naming scheme
* Add some freshness and add new features
* Fix force and fix duration if less than 60
* Allow duration to be more readable
* Fix description
* Improve func wording and make int machines length
* No point overloading functions
* Add prune to list of commands
* Check if GID is already the group if so no need to chown
* Revert "Check if GID is already the group if so no need to chown"
This reverts commit c7cef1773e
.
* change all short desc to be similar, and made it really really clear when pruning it is not recoverable
* Better examples
* Match bouncer like for like
* Fix merge error
* Dont use log. and dont return error on user input to abort
2023-07-28 15:23:47 +01:00
mmetc
ae53c0f1cc
fix "crowdsec-cli/require" log verbosity ( #2390 )
2023-07-28 09:56:20 +02:00
mmetc
5cb7013575
Check cscli preconditions with crowdsec-cli/require package ( #2388 )
2023-07-27 17:02:20 +02:00
mmetc
a01ce18b98
replace imports of path with path/filepath ( #2330 )
2023-07-26 10:29:58 +02:00
mmetc
1a6f12c88e
Build target for "make tidy" ( #2378 )
...
The make tidy target runs "go mod tidy" in the root directory and all plugins.
2023-07-26 10:24:37 +02:00
Laurence Jones
389ea4293f
Add metabase version override and update ( #2370 )
...
* Add version override and update
* Ooppsie
* Quick fix
* fgs copilot
* Allow user to overwrite image, add warning for exposing metabase and general cleanup
* One ix
* Default image if not found in config, and add a warning to remove and update
* Reorder check system memory checks so it inline with @mmetc best pratices
* No need for err
* Clean up some group code
* Change ipv6 as [] seems to wildcard
* Split loopback warn and disclaimer. Add force yes to start to allow user to accept disclaimer by default
* All cmd commands are RunE clean up
* Update flag name and dont allow a shorthand
2023-07-25 14:21:25 +01:00
mmetc
4bc225f26b
change output of "cscli metrics -o [json|raw]" from list of objects to map with table names ( #2375 )
2023-07-25 13:33:50 +02:00
mmetc
b6b6fd026b
typo fix, uppercase 'API', adjusted log level ( #2361 )
2023-07-21 23:23:24 +02:00
Manuel Sabban
9ac5aeda79
fix the ci by adding the ability to enforce event ordering ( #2347 )
...
* fix the ci by adding the ability to enforce event ordering
2023-07-20 11:41:30 +02:00
mmetc
3c16139c44
Reduce log verbosity at startup ( #2363 )
...
A configuration syntax test is performed every time the service is
started from systemd. The resulting error, if any, is shown on
journalctl logs.
This PR removes the unnecessary output in crowdsec.log generated by the
configuration test.
2023-07-19 13:28:52 +02:00
mmetc
bb16552aca
Use same levenshtein package for cscli, ent, hcl ( #2359 )
...
remove one dependency, slightly smaller binary
2023-07-18 11:30:14 +02:00
mmetc
9967d60987
errors.Wrap -> fmt.Errorf ( #2333 )
2023-07-06 10:14:45 +02:00
mmetc
85839b0199
support for stdin with "cscli decision import" and raw values ( #2291 )
...
and remove Origin from the struct, which was ignored anyway
2023-06-27 14:29:42 +02:00
mmetc
a910b7beca
non-fatal error if some datasource can't be run (i.e. journalctl but systemd is missing) ( #2309 )
...
This on the other hand, gives a new fatal error when there are no valid datasources.
In the previous version, crowdsec kept running with just a warning if no
acquisition yaml or dir were specified.
2023-06-27 10:13:13 +02:00
mmetc
62caffb102
update leakybucket readme ( #2298 )
2023-06-22 15:35:01 +02:00
mmetc
fddf597040
errors.Wrap -> fmt.Errorf; clean up imports ( #2297 )
2023-06-22 15:01:34 +02:00
mmetc
25bb23d8b7
minor refactor to pkg/types, cscli machines ( #2270 )
...
* cleanup: separate ui and logic
* trim some code from pkg/types
2023-06-08 15:08:51 +02:00
mmetc
6096cb3c9b
Move grok_pattern.go away from pkg/types to trim bouncer dependencies ( #2269 )
2023-06-08 15:07:30 +02:00
mmetc
4e2c9c185b
Implement "crowdsec -fatal" flag; change help message ( #2266 )
...
The -trace...-fatal flags do not change the log destination but only the
verbosity. This change reflects that, and implements "-fatal" which was missing.
2023-06-08 15:06:06 +02:00
mmetc
8da9d5eefd
don't log notification error if not running under systemd ( #2274 )
2023-06-08 15:04:48 +02:00
mmetc
3cc6b2c0d0
CI: add tests for metrics configuration ( #2251 )
2023-06-05 23:17:30 +02:00
mmetc
9ccdddaab1
CI: refactor makefile for plugins and vendor target ( #2256 )
2023-06-05 23:15:18 +02:00
mmetc
2a8e97d558
show option -winsvc only under windows ( #2258 )
2023-06-05 13:49:31 +02:00
mmetc
228e4f9acc
cscli: add G (1e9) suffix to metric units ( #2254 )
2023-06-02 14:38:11 +02:00
mmetc
396dcf8e6e
dependencies: replaced function calls to pkg/types, errors.Wrap ( #2235 )
...
we now use a generic pointer function, and slowly remove the deprecated pkg/errors
2023-06-01 16:31:56 +02:00
mmetc
92a9d6c321
types.InSlice() -> slices.Contains() ( #2246 )
2023-05-31 12:39:22 +02:00
mmetc
9167bd107d
decouple bouncer dependencies: use go-cs-lib/pkg/ptr ( #2228 )
2023-05-25 15:43:39 +02:00
mmetc
1f9f81da70
makefiles: de-duplicate, simplify and remove unused code ( #2222 )
2023-05-25 10:32:05 +02:00
mmetc
534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* ( #2216 )
...
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
2023-05-23 10:52:47 +02:00
blotus
6e3ca35941
fallback to master for hub index download if it does not exist ( #2210 )
2023-05-17 11:20:53 +02:00
mmetc
e1f5ed41df
Implement "cscli config show-yaml" ( #2191 )
2023-05-11 21:01:13 +02:00
Thibault "bui" Koechlin
5ac33aab03
allow batching when importing decisions ( #2192 )
2023-05-11 14:33:18 +02:00
mmetc
6b744884b0
Update deps to latest stable: go-systemd, tail, cobra, lumberjack, testify ( #2164 )
2023-04-12 16:58:11 +02:00
mmetc
0c5d233563
Minor cleanup and dead code removal ( #2166 )
2023-04-12 16:57:38 +02:00
blotus
1e018bdaf8
Wait for both api and agent chans if necessary when daemonize is false or running on windows ( #2155 )
2023-04-04 15:16:48 +02:00
mmetc
38ab6be7c2
Allow feature.yml to change available subcommands ( #2156 )
2023-04-03 10:11:56 +02:00
mmetc
ea6401ce09
CI: Static builds by default; replace bincover with go -cover from 1.20 ( #2150 )
...
* Makefile: build static binaries only
* Replace bincover with go -cover from 1.20
* CI: Fix timing issue between lapi and agent containers
2023-03-30 15:05:09 +02:00
blotus
61bea26486
Add transform
configuration option for acquisition ( #2144 )
2023-03-29 16:04:17 +02:00
blotus
1095f6c875
use expr.Function for custom functions instead of passing them in the env ( #2133 )
2023-03-28 10:49:01 +02:00
blotus
91eb39cff6
New PAPI commands: reauth + force_pull ( #2129 )
2023-03-21 14:06:19 +01:00
Thibault "bui" Koechlin
a74e424d53
support ip and cidr based whitelists for capi and 3rd party blocklists ( #2132 )
...
* support ip and cidr based whitelists for capi and 3rd party blocklist
2023-03-21 11:50:10 +01:00
AlteredCoder
e61a464951
Fix cscli explain when running from testenv ( #2114 )
...
* Fix cscli explain when running from testenv
2023-03-15 10:26:40 +01:00
mmetc
e161507d08
Lint (type inference): remove redundant type declarations ( #2111 )
2023-03-09 11:56:02 +01:00
mmetc
9faa49c7e8
Load lapi config for config show output
( #2097 )
...
This adds URL and login parameters as it was intended.
Also rewrite configShow and displayOneAlert to use an embedded text/template for shorter code.
2023-03-08 22:47:25 +01:00
Thibault "bui" Koechlin
9d5aaf5ea2
add --origin to cscli decisions delete ( #2109 )
2023-03-08 18:29:20 +01:00
Thibault "bui" Koechlin
5b0fe4b7f1
support for regexps result cache ( #2104 )
...
* support for regexps result cache : gcache + xxhash
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-03-08 16:07:49 +01:00
blotus
16a3be49e2
do not try to load PAPI is url is not set ( #2099 )
2023-03-06 15:38:58 +01:00
blotus
e27a0a0e14
display source in alerts list when an alert has multiple decisions ( #2098 )
2023-03-06 13:51:57 +01:00
blotus
b2c2c5ac59
add papi_url in credentials file when enabling console_management, and remove it when disabling console_management ( #2095 )
2023-03-03 17:03:21 +01:00
blotus
85ab9c68a2
Add cscli papi status
and cscli papi sync
( #2091 )
2023-03-03 13:46:28 +01:00
mmetc
f6d6c5bb2b
Add tests and typo fixes ( #2092 )
2023-03-03 11:06:27 +01:00
mmetc
a6bb2cf5e1
Fix log destination in one-shot mode ( #2084 )
2023-03-01 17:00:04 +01:00
Manuel Sabban
60b3f63851
ugly workaround to fix the tests ( #2080 )
...
* ugly workaround to fix the tests
* add comments
---------
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2023-02-28 17:05:11 +01:00
Manuel Sabban
39a4a256fd
fix the way acquisition is stopped ( #2069 )
...
* fix the way acquisition is stopped by draining inputLineChan before terminating it.
---------
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2023-02-27 11:21:25 +01:00
Laurence Jones
75d8b821ff
Explain successful parsers only ( #2063 )
...
* Add option to filter down explain to successful parsers useful for me who has every collection installed
* Altered naming conventions so it makes more sense when reading
2023-02-24 13:49:17 +00:00
mmetc
b7d1e2c483
replace log.Fatal -> fmt.Errorf ( #2058 )
2023-02-20 15:05:42 +01:00
blotus
83c3818504
Do not try to refresh JWT token when doing a login request ( #2059 )
2023-02-16 16:16:26 +01:00
Thibault "bui" Koechlin
1d7d377f8b
changes following BL tests ( #2038 )
...
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-02-08 10:35:21 +01:00
Cristian Nitescu
987f119c4b
v3 capi and blocklists links support ( #2019 )
...
* v3 model generation
* v3 model generation
* comms
* fixes after master merge
* missing reader close
* use constants defined for types
---------
Co-authored-by: bui <thibault@crowdsec.net>
2023-02-06 14:06:14 +01:00
mmetc
b6be18ca65
cscli setup ( #1923 )
...
Detect running services and generate acquisition configuration
2023-02-06 07:33:04 +01:00
Thibault "bui" Koechlin
e927717fa0
Polling API Integration ( #1715 )
...
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-31 14:47:44 +01:00
mmetc
d369656b26
agent: fix message when -dsn is provided without -type ( #2009 )
2023-01-20 16:14:26 +01:00
mmetc
e5833699c0
cscli config feature-flags ( #2006 )
2023-01-20 09:32:10 +01:00
mmetc
4bffc0df21
break in smaller functions cscli hub, hubtest, notifications, parsers, scenarios, simulation ( #2004 )
2023-01-19 13:29:36 +01:00
mmetc
7bb74b9664
refact cscli decisions ( #2003 )
2023-01-19 11:02:00 +01:00
Thibault "bui" Koechlin
4f29ce2ee7
CTI API Helpers in expr ( #1851 )
...
* Add CTI API helpers in expr
* Allow profiles to have an `on_error` option to profiles
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-19 08:45:50 +01:00
Marco Mariani
47dbfa770d
configure logging earlier
2023-01-18 15:15:18 +01:00
Marco Mariani
91b0f8fee1
load custom configuration paths when agent is disabled
2023-01-18 15:15:18 +01:00
Marco Mariani
2e91a82aa7
load feature.yaml as soon as possible
2023-01-18 15:15:18 +01:00
Marco Mariani
b603bdfccc
cscli refact: extracted New.*Cmd from alerts, capi, dashboard; removed (some) globals
2023-01-18 11:09:28 +01:00
mmetc
51800132cd
improve feature flag logging ( #1986 )
...
For cscli: it should provide a terse output, not nag users with configuration details. Although it's usually important that cscli and crowdsec have the same enabled features, having it list them every time the command is invoked can be too much.
For crowdsec: when features are set from the environment, it's too early to log where we should. So we can use log.Debug at activation time, and list them again once logging is configured.
- wrap some functions in csconfig for convenience and DRY
- for each enabled feature, log.Debug
- log all enabled features once as Info (crowdsec) or Debug (cscli)
- file does not exist -> log.Trace
2023-01-13 13:42:42 +01:00
mmetc
157589d31e
cscli explain: add crowdsec path option ( #1983 )
2023-01-12 17:04:28 +01:00
Thibault "bui" Koechlin
6fb962a941
Allow parsers to capture data for future enrichment ( #1969 )
...
* Allow parsers to capture data in a cache, that can be later accessed via expr helpers (fake multi-line support)
2023-01-11 15:01:02 +01:00
mmetc
cd4dabde0e
silence yaml.local explicitly in cscli, keep in crowdsec/bouncer logs ( #1981 )
2023-01-11 09:50:46 +01:00
mmetc
c4deaf0994
cscli: avoid initializing the db configuration twice ( #1982 )
2023-01-11 09:50:12 +01:00
AlteredCoder
185f9ad541
Alert context ( #1895 )
...
Co-authored-by: bui <thibault@crowdsec.net>
2023-01-04 16:50:02 +01:00
mmetc
59f6610721
separate cscli cobra constructors: lapi, machines, bouncers, postoverflows ( #1945 )
2022-12-30 10:13:52 +01:00
mmetc
6efc2688b1
simplify feature flags ( #1947 )
...
Now checking for a feature flag is a one liner,
with no need to control errors.
if fflag.Crowdsec.CscliSetup.IsEnabled() {
...
}
2022-12-26 14:23:41 +01:00
mmetc
c022eb1b86
remove ignored flag "-m" in "cscli machines delete" (it takes a positional argument) ( #1943 )
2022-12-23 17:13:20 +01:00
mmetc
ef3a130d54
Cscli config refactoring ( #1934 )
2022-12-22 12:22:55 +01:00
mmetc
5d2c99bb17
runtime feature flag initialization
2022-12-21 17:19:20 +01:00
mmetc
a32aa96752
feature flags ( #1933 )
...
Package fflag provides a simple feature flag system.
Feature names are lowercase and can only contain letters, numbers, undercores
and dots.
good: "foo", "foo_bar", "foo.bar"
bad: "Foo", "foo-bar"
A feature flag can be enabled by the user with an environment variable
or by adding it to {ConfigDir}/feature.yaml
I.e. CROWDSEC_FEATURE_FOO_BAR=true
or in feature.yaml:
```
---
- foo_bar
```
If the variable is set to false, the feature can still be enabled
in feature.yaml. Features cannot be disabled in the file.
A feature flag can be deprecated or retired. A deprecated feature flag is
still accepted but a warning is logged. A retired feature flag is ignored
and an error is logged.
A specific deprecation message is used to inform the user of the behavior
that has been decided when the flag is/was finally retired.
2022-12-20 16:11:51 +01:00
mmetc
6c19beb937
set cscli log timestamp to 24h ( #1917 )
2022-12-09 16:48:24 +01:00
blotus
fdda940ac0
Add Kubernetes audit acquisition ( #1767 )
2022-12-06 13:47:29 +01:00
Manuel Sabban
3d72ca731a
Suggest bouncers and machines to delete ( #1896 )
...
* Suggest bouncers to delete
* Autocomplete machines delete cmd
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
* Fix lint.
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
* fix compilation (git merge errors)
* cleanup go.mod unneeded changes
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Co-authored-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-12-05 15:39:54 +01:00
mmetc
fd3e668fe1
add -error flag to crowdsec binary ( #1903 )
2022-12-03 08:56:11 +01:00
mmetc
104f5d1fe6
lint: error handling cleanup ( #1877 )
2022-11-29 09:16:07 +01:00
mmetc
99513f64fd
cscli config show: print host/port/user/dbname when driver=pgx ( fix #1866 ) ( #1870 )
2022-11-17 11:07:37 +00:00
mmetc
3beb84bcfe
print missing "AS" values as empty strings instead of "0 " ( #1867 )
2022-11-14 09:55:53 +01:00
mmetc
895691dad1
enabled linters: gocritic, nilerr ( #1853 )
2022-11-07 10:36:50 +01:00
Thibault "bui" Koechlin
23ffa1e04f
add cscli alerts delete --id
( #1843 )
...
* add cscli alerts delete by id
* test added for cscli delete alert --id
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2022-10-27 09:07:53 +02:00
Thibault "bui" Koechlin
ae6bf39495
support decisions deletion via scenario + alerts delete via ID ( #1798 )
2022-10-19 14:37:27 +02:00
mmetc
2b7e3ff1e7
warn if no acquisition files are found, acquisition_test refactoring, tests ( #1816 )
2022-10-17 17:32:08 +02:00
mmetc
a96b3e077d
rename pkg/cstest -> pkg/hubtest ( #1811 )
...
keep cstest for generic helper functions
this also avoids circular imports in test files
2022-10-17 09:24:07 +02:00
mmetc
8fecc2c00b
enable staticcheck linter; fixes ( #1806 )
...
- explicitly ignore returned parameters
- replace Walk with faster WalkDir
- log path error during hub dir sync
- colorize static unit tests
- removed duplicate import in crowdsec/main.go
- typos
- func tests: default datasource in tests/var/log instead of /tmp
- action setup-go v3
2022-10-14 16:12:21 +02:00
mmetc
708fa8280a
add test and fix for configuration reload ( #1808 )
...
fix reload issue by returning new configuration to the signal loop
example: run crowdsec, disable agent in the config file, reload config.
Now there is no agent but the signal loop believes there is, so
triggering a reload configuration again will make the process hang
forever.
This commit updates the configuration in the signal loop with the one
returned by the signal handler.
2022-10-14 15:48:41 +02:00
blotus
7144dca68a
Fix missing metrics cscli ( #1809 )
2022-10-13 15:49:41 +02:00
mmetc
4b3c9c2806
print cscli usage in color, fix windows terminal detection ( #1801 )
2022-10-13 12:28:24 +02:00
Shivam Sandbhor
74659a82ab
Fast bulk alert delete ( #1791 )
2022-10-07 12:40:30 +02:00