Commit graph

94 commits

Author SHA1 Message Date
mmetc 0f942a95f1
pkg/cwhub - rename methods for clarity (#2961)
* pkg/cwhub - rename methods for clarity

* lint
2024-04-24 11:09:37 +02:00
mmetc 2682f801df
windows: fix data file update (remove before rename) (#2930) 2024-04-05 14:57:33 +02:00
mmetc 2e1ddec107
cscli: Add user-agent to all hub requests (#2915)
* cscli: Add user-agent to all hub requests

* fix unit test and avoid httpmock

* fix windows test
2024-03-25 10:40:41 +01:00
mmetc 7779c7ff0c
hub update: reload crowdsec if only data files have changed (#2912) 2024-03-20 15:46:14 +01:00
mmetc 6c5e8afde9
pkg/cwhub: download data assets to temporary files to avoid partial fetch (#2879) 2024-03-08 10:55:30 +01:00
mmetc a23fe06d68
remove dependencies on enescakir/emoji, gotest.tools (#2837)
* wrap emoji package in pkg/emoji
* remove dependency on enescakir/emoji
* remove dependency on gotest.tools
* lint (whitespace)
2024-02-23 16:05:01 +01:00
mmetc 260f5a7992
pkg/cwhub: improve error messages (#2712)
* pkg/cwhub: improve error messages
* lint
2024-01-11 10:28:58 +01:00
mmetc 2a2b09b52a
cwhub: install --force repair tainted, non-installed items (#2686) 2024-01-03 10:08:45 +01:00
mmetc ca784b147b
test and log fixes (#2690)
* cscli inspect: suggest --diff if an item is tainted
* appropriate warning, or error if context configuration file is empty
* fix user/group lookup unit test
* fix: allow hub upgrade --force with local items
* fix pkg/parser lookup for 8.8.8.8
* fix func test
* fix hubtests: machines add --force
2024-01-03 09:33:52 +01:00
mmetc 6e34d609b7
cscli: silence cwhub logger for non-hub related commands (#2675) 2023-12-19 17:20:09 +01:00
mmetc a79fcaf378
Add "taintedBy" and "--diff" flag to cscli... inspect (#2665)
* "cscli inspect" reports tainted sub-items
* cscli... inspect --diff
* unified diff
* option --diff --rev
* tainted message
* correctly report multiple taint reasons
2023-12-15 15:27:22 +01:00
mmetc 4acb4f8df3
cwhub: context type (#2631)
* add hub type "context"
* cscli lapi: log.Fatal -> fmt.Errorf; lint
* tests for context.yaml
* load console context from hub
* original & compiled context
* deprecate "cscli lapi context delete"
$ cscli lapi context delete
Command "delete" is deprecated, please manually edit the context file.
* cscli completion: add appsec-rules, appsec-configs, explain, hubtest
2023-12-07 16:20:13 +01:00
Thibault "bui" Koechlin 8cca4346a5
Application Security Engine Support (#2273)
Add a new datasource that:
- Receives HTTP requests from remediation components
- Apply rules on them to determine whether they are malicious or not
- Rules can be evaluated in-band (the remediation component will block the request directly) or out-band (the RC will let the request through, but crowdsec can still process the rule matches with scenarios)

The PR also adds support for 2 new hub items:
- appsec-configs: Configure the Application Security Engine (which rules to load, in which phase)
- appsec-rules: a rule that is added in the Application Security Engine (can use either our own format, or seclang)

---------

Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-12-07 12:21:04 +01:00
mmetc 1ab4487b65
cscli hub list: show only non-empty tables with -o human
* agent config: remove unused LintOnly bool
* Item.IsLocal() -> Item.State.IsLocal(); split method InstallStatus()
* cscli hub list: show only non-empty tables with -o human
2023-12-05 13:38:52 +01:00
mmetc 7e5ab344a2
command "cscli hub types" (#2632)
* Command "cscli hub types"; de-duplicate test/bin/preload-hub-items
* don't export Hub.Items -> hub.items
2023-12-01 09:36:38 +01:00
mmetc 6b0bdc5eeb
Refact pkg/cwhub: fix some known issues and reorganize files (#2616)
* bump gopkg.in/yaml.v3
* test: cannot remove local items with cscli
* test dangling links
* test: cannot install local item with cscli
* pkg/cwhub: reorg (move) functions in files
* allow hub upgrade with local items
* data download: honor Last-Modified header
* fatal -> warning when attempting to remove a local item (allows remove --all)
* cscli...inspect -o yaml|human: rename remote_path -> path
* Correct count of removed items
Still no separate counter for the --purge option, but should be clear enough
2023-11-28 23:51:51 +01:00
mmetc ffcab0b2bc
Refactor hub management and cscli commands (#2545) 2023-11-24 15:57:32 +01:00
mmetc 3b1563a538
Refact cscli hub / pkg/cwhub (part 6) (#2524)
* hub.ConfigDir -> hub.InstallDir; hub.DataDir -> hub.InstallDataDir
* cleanup GetInstalledItemsAsString()
* lint: ReferenceMissingError -> ErrMissingReference
* lint: parent_dir -> parentDir
* link: export Walker type
* lint: return error last
* lint: shadow
* move around and group variable definitions
2023-10-09 21:33:35 +02:00
Manuel Sabban 6e228f3f3f
pkg/cwhub: cleanup in argument call (#2527)
* cleanup in argument call
* update test as well
* cwhub_tests: reduce verbosity and use helpers

---------

Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-10-09 13:26:34 +02:00
mmetc 338141f067
Refact cscli hub / pkg/cwhub (part 5) (#2521)
* remove unused yaml tags
* cscli/cwhub: deduplicate, remove dead code
* log.Fatal -> fmt.Errorf
* deflate utils.go by moving functions to respective files
* indexOf() -> slices.Index()
* ItemStatus() + toEmoji() -> Item.status()
* Item.versionStatus()
* move getSHA256() to loader.go
2023-10-06 13:59:51 +02:00
mmetc 9235f55c47
Refact pkg/cwhub (part 4) (#2518)
* generalize function: GetInstalledItems, GetInstalledItemsAsString
* extracted function itemKey, happy path
* review comments / remove redundant; rename file to remove build tags
* remove unused fields in Item struct
* unix build tag
2023-10-05 09:35:03 +02:00
mmetc 89028f17cf
Refact pkg/cwhub (part 3) (#2516)
* removed unused error; comment
* rename loop variables
* happy path
* rename loop variables
* extract function, method
* log.Printf -> log.Infof
* tests -> testdata

from "go help test":

The go tool will ignore a directory named "testdata", making it available
to hold ancillary data needed by the tests.

* align tags
* extract function toEmoji
2023-10-04 12:54:21 +02:00
mmetc 3253b16f0f
Refact pkg/cwhub (part 2) (#2513)
* remove globals for walker callback
* extract method getItemInfo()
* code dedup, if/else -> switch
* dedent: happy path
* remove target variable
2023-10-04 11:17:35 +02:00
mmetc d39131d154
Refact pkg/cwhub (part 1) (#2512)
* wrap errors, whitespace
* remove named return
* reverse CheckSuffix logic, rename function
* drop redundant if/else, happy path
* log.Fatal -> fmt.Errorf
* simplify GetItemMap, AddItem
* var -> const
* removed short-lived vars
* de-duplicate function and reverse logic
2023-10-04 10:34:10 +02:00
mmetc 8b5ad6990d
lint: pkg/cwhub (#2510)
no functional changes
 
 - reformat
 - comments
 - whitespace
 - removed a dot or two in log messages
 - some "var x=y" -> x:=y
2023-10-03 11:20:56 +02:00
mmetc c588be0842
golangci-lint: use v1.54, remove unnecessary byte/string conversions (#2438) 2023-08-25 16:22:10 +02:00
mmetc a01ce18b98
replace imports of path with path/filepath (#2330) 2023-07-26 10:29:58 +02:00
mmetc 9967d60987
errors.Wrap -> fmt.Errorf (#2333) 2023-07-06 10:14:45 +02:00
mmetc 17cd792826
CI: update ansible tests for re2 (#2318) 2023-06-29 16:35:19 +02:00
mmetc bd41f855cf
errors.Wrap -> fmt.Errorf (#2317) 2023-06-29 11:34:59 +02:00
mmetc 9beb5388cb
errors.Wrap -> fmt.Errorf; clean up imports (#2301) 2023-06-23 14:04:58 +02:00
mmetc 76429f033a
trim pkg/types: move DataSet/GetData to pkg/cwhub, removed unused Clone function (#2271) 2023-06-08 16:49:51 +02:00
blotus 6e3ca35941
fallback to master for hub index download if it does not exist (#2210) 2023-05-17 11:20:53 +02:00
AlteredCoder c0e6c1ac78
Fix chooseHubBranch when latest() doesn't work (#2178)
* Fix chooseHubBranch when latest() doesn't works
2023-04-28 11:24:04 +02:00
mmetc 3884c5f47d
Unit tests: remove leftover files (#2134) 2023-03-22 13:51:37 +01:00
mmetc be18fea136
Propagate taints to top collections (fix #2064) (#2066) 2023-02-21 22:12:08 +01:00
mmetc 104f5d1fe6
lint: error handling cleanup (#1877) 2022-11-29 09:16:07 +01:00
mmetc 895691dad1
enabled linters: gocritic, nilerr (#1853) 2022-11-07 10:36:50 +01:00
mmetc df88f4e1e9
randomize pull, push and metric intervals; reload crowdsec only when hub changed (#1846) 2022-10-28 13:55:59 +02:00
mmetc e545933923
fix(cscli): correct and test the behavior of "cscli collections delete" (#1824) 2022-10-25 14:10:51 +02:00
mmetc 6b0097a24b
change warning to debug when directories are missing in hub sync (#1819) 2022-10-18 10:32:54 +02:00
mmetc 8fecc2c00b
enable staticcheck linter; fixes (#1806)
- explicitly ignore returned parameters
 - replace Walk with faster WalkDir
 - log path error during hub dir sync
 - colorize static unit tests
 - removed duplicate import in crowdsec/main.go
 - typos
 - func tests: default datasource in tests/var/log instead of /tmp
 - action setup-go v3
2022-10-14 16:12:21 +02:00
mmetc 414282a2c9
golangci-lint 1.49 and related fixes (#1736) 2022-09-06 13:55:03 +02:00
mmetc eea07b7a1a
golangci-lint v1.48 and fixes for "usestdlibvars" (#1711) 2022-08-16 09:46:10 +02:00
mmetc 628d7be1d8
simplify err.Error() to err when used in printf context (#1603) 2022-06-22 15:53:53 +02:00
mmetc d71279f023
added flag crowdsec --warning (#1461) 2022-06-22 09:38:23 +02:00
mmetc 10585bfecc
enabled linters and fixes for: misspell, predeclared, unconvert, ineffassign, gosimple, govet (#1595) 2022-06-16 14:41:54 +02:00
mmetc 799cc82bb5
functional tests, minor refactoring and lint/cleanup (#1570)
* cmd/crowdsec: removed log.Fatal()s, added tests and print error for unrecognized argument
* updated golangci-lint to v1.46
* lint/deadcode: fix existing issues
* tests: cscli config backup/restore
* tests: cscli completion powershell/fish
* err check: pflags MarkHidden()
* empty .dockerignore (and explain the reason)
* tests, errors.Wrap
* test for CS_LAPI_SECRET and minor refactoring
* minor style changes
* log cleanup
2022-06-06 15:24:48 +02:00
mmetc 1a293a2a27
cwhub: export SetHubBranch (#1559) 2022-05-24 15:46:48 +02:00
blotus 0449ec1868
Windows Support (#1159) 2022-05-17 12:14:59 +02:00