Commit graph

101 commits

Author SHA1 Message Date
Marco Mariani eb1bea26cd load console context from hub 2023-12-07 09:47:00 +01:00
Marco Mariani c1a04ead79 tests for context.yaml 2023-12-07 09:45:46 +01:00
Marco Mariani b21fa99902 cscli lapi: log.Fatal -> fmt.Errorf; lint 2023-12-07 09:45:14 +01:00
Marco Mariani 18b53128a5 add hub type "context" 2023-12-07 09:44:54 +01:00
mmetc f7c5726a0a
minor reverts and tweaks (#2639) 2023-12-05 17:06:25 +01:00
Sebastien Blot 17384368ae
merge master 2023-12-05 14:01:28 +01:00
Sebastien Blot bd2c59b054
fix some tests 2023-12-05 13:55:49 +01:00
mmetc 1ab4487b65
cscli hub list: show only non-empty tables with -o human
* agent config: remove unused LintOnly bool
* Item.IsLocal() -> Item.State.IsLocal(); split method InstallStatus()
* cscli hub list: show only non-empty tables with -o human
2023-12-05 13:38:52 +01:00
mmetc 0f3ae64062
cscli config show: pretty print with package "litter" (#2633) 2023-12-05 10:38:21 +01:00
mmetc 0c4093dcca
Test for acquisition errors in crowdsec -t (#2629) 2023-12-04 23:09:42 +01:00
mmetc a5ab73d458
cscli machines add: don't overwrite existing credential file (#2625)
* cscli machines add: don't overwrite existing credential file
* keep old behavior with --force
Now --force is used both to override the replacement of and existing machine,
and an existing credentials file. To retain the old behavior, the
existence of the file is only checked for the default configuration, not
if explicitly specified.
2023-12-04 22:59:52 +01:00
blotus 872e218b31
Merge branch 'master' into coraza_poc_acquis 2023-12-04 10:00:10 +01:00
mmetc 7e5ab344a2
command "cscli hub types" (#2632)
* Command "cscli hub types"; de-duplicate test/bin/preload-hub-items
* don't export Hub.Items -> hub.items
2023-12-01 09:36:38 +01:00
Sebastien Blot 4b7b138be7
Merge branch 'master' into coraza_poc_acquis 2023-11-29 22:02:51 +01:00
mmetc 6b0bdc5eeb
Refact pkg/cwhub: fix some known issues and reorganize files (#2616)
* bump gopkg.in/yaml.v3
* test: cannot remove local items with cscli
* test dangling links
* test: cannot install local item with cscli
* pkg/cwhub: reorg (move) functions in files
* allow hub upgrade with local items
* data download: honor Last-Modified header
* fatal -> warning when attempting to remove a local item (allows remove --all)
* cscli...inspect -o yaml|human: rename remote_path -> path
* Correct count of removed items
Still no separate counter for the --purge option, but should be clear enough
2023-11-28 23:51:51 +01:00
Sebastien Blot dc39866250
merge from master 2023-11-27 13:34:22 +01:00
mmetc ffcab0b2bc
Refactor hub management and cscli commands (#2545) 2023-11-24 15:57:32 +01:00
mmetc 76d4bc7788
cscli bouncers: increase key size, deprecate and ignore --length option (#2531)
the switch to base64 made the keys shorter (24 characters), this PR increases their size to 32 bytes, 42 chars once encoded

Also deprecate the --length option, users can already provide a key
2023-11-24 15:01:13 +01:00
lperdereau 92f923cfa8
Loki integration #2 (#2306)
* Add support for loki datasource

---------

Co-authored-by: Mathieu Lecarme <mathieu@garambrogne.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
2023-11-22 13:31:39 +01:00
Sebastien Blot 5abc8e0e14
merge hub-1.5.6 2023-11-21 17:46:54 +01:00
mmetc 2c652ef92f
pkg/cwhub documentation (#2607)
* pkg/cwhub: package documentation

* Don't repeat local state in "cscli... inspect"

* lint

* use proper name of the hub item instead of the filename for local items

* hub update: avoid reporting local items as tainted
2023-11-21 17:43:10 +01:00
Sebastien Blot 9580f8e14d
merge hub-1.5.6 2023-11-21 17:28:10 +01:00
mmetc 1509c2d97c
pkg/cwhub refact (#2606)
* Separate Item and ItemState; fill BelongsToCollections with all ancestors and for uninstalled items too
* fix "installed parents" check when removing an item
* keep BelongsToCollections in order (case insensitive)
2023-11-21 11:06:59 +01:00
mmetc 6b317f0723
Refact cwhub: simplify tree scan and dependency checks (#2600)
* method rename: GetInstalledItemsAsString() -> GetInstalledItemNames()
* use path package
* Comments and method names
* Extract method Item.setVersionState() from Hub.itemVisit()
* refact localSync(), itemVisit() etc.
* fix check for cyclic dependencies, with test
2023-11-20 11:41:31 +01:00
Sebastien Blot db40ba7b3b
Merge branch 'hub-1.5.6' into coraza_poc_acquis 2023-11-16 17:12:23 +01:00
mmetc 56ad2bbf98
Refact cwhub: item removal with shared dependencies (#2598)
* Iterate over sub-items in Remove(), not in disable() -- fix shared dependency issue
* Increase hub download timeout to 2 minutes
2023-11-16 17:00:51 +01:00
mmetc d9b0d440bf
Refact cwhub (#2596)
* unused param
* (slightly) simpler ListItems() -> listItems()
* listItems(): always showHeader, deduce showType
ref. https://github.com/crowdsecurity/crowdsec/issues/1068
* simplify Item.disable()
also, .tainted and .installed do not need a default since they are always in the json output now
* Drop unused parameters
2023-11-16 11:09:49 +01:00
mmetc 47eb2e240d
Use go 1.21.4 (#2595) 2023-11-16 11:09:13 +01:00
mmetc 79d019f9a2
Refact cwhub / sort cscli output, case insensitive (#2593)
* dead code: unknown localVersion now defaults to "?"
* skip type declaration; whitespace
* sync: next item if invalid cpath
* func tests for install --force and --ignore
* shorter test names
* sort cscli <itemtype> output, with tests
* cscli: refact hub sort code
2023-11-15 16:59:30 +01:00
mmetc 4a6fd338e0
replace 'timeout' helper with async python script; allow hub preload in func tests; improve item removal (#2591)
* replace 'timeout' helper with async python script; allow hub preload in func tests; improve item removal
* func tests: cscli hub update/upgrade
* docker test update
* Update docker entrypoint to disable items with --force

The --force flag was not transmitted to cscli, but is required after the hub refact
to disable items inside installed collections
2023-11-14 17:36:07 +01:00
Marco Mariani 120f7cf578 Merge branch 'master' into hub-1.5.6 2023-11-14 15:04:50 +01:00
mmetc 042d316fab
Refact cwhub: remove global hub, func test improvements (#2588)
* csConfig.Cscli is always loaded now, configuration paths too
* Remove global/singleton hub instance
* read {index_path} from config instead of assuming {hub_dir}/.index.json
* fix segfault with cscli explain when no parser is installed
* cscli: help text
* hub download timeout 20 sec
* reduce log verbosity
* allow func tests with empty hub or pre-download
* cscli <itemtype> remove --all --purge
2023-11-14 14:58:36 +01:00
Sebastien Blot d6f9bbc0c3
merge hub-1.5.6 branch 2023-11-10 17:36:17 +01:00
mmetc 9d7ed12950
Refact cwhub (#2586)
* Inspect item: always show tainted, installed, etc. when false
* cleanup, comments, unused stuff
* download collection content after downloading dependencies, avoid duplicate call
* Return instances from Item.SubItems()
* shorter i/o code
* inline / simplify getData()
* Handle timeout connections when downloading from hub or data
2023-11-10 10:25:29 +01:00
mmetc ab8de19506
Refact cwhub: move methods from hub to item (#2585)
* Add back pointer Item.hub
* Hub.enableItem() -> Item.enable()
* Rename variable i -> idx (i is used for item instances)
* Move Hub.purgeItem() -> Item.purge()
* Move Hub.disableItem() -> Item.disable()
* Move Hub.downloadItem() -> Item.download()
* Move Hub.downloadLatest() -> Item.downloadLatest()
* Move Hub.DownloadDataIfNeeded() -> Item.DownloadDataIfNeeded()
* Move Hub.InstallItem() -> Item.Install()
* Move Hub.RemoveItem() -> Item.Remove()
* Move Hub.UpgradeItem() -> Item.Upgrade()
* store hub items as pointers
* No need to re-add items to the hub if we use pointers
* Fix parameter calling order + regression test
2023-11-09 15:19:38 +01:00
Sebastien Blot 694028f769
merge hub branch 2023-11-08 20:25:42 +01:00
mmetc f4b5bcb865
Refact cwhub: version comparison and branch selection (#2581)
* simplify GetItemByPath
* hub: sort version numbers by semver
* replace golang.org/x/mod/semver with github.com/Masterminds/semver/v3 (would not compare correctly)
* fix nil dereference with tainted items
* update tests for collections, postoverflows
* fix nil deref
* don't fallback to master if hub is not found, improve message
* explicit message for unknown version / tainted collections
2023-11-08 13:21:59 +01:00
guangwu ddd6ee8e42
fix: typo (#2582)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-11-08 09:26:34 +01:00
Manuel Sabban 4934fce769
update gantsign.golang name (#2558) 2023-11-07 14:53:14 +01:00
mmetc bfd94ceda7
make ParserIndex(), DownloadIndex() private methods (#2579)
* unnecessary pointer type
* ParseIndex() as hub method, don't collect missing items since they are never used
* don't export hub.parseIndex(), hub.downloadIndex()
2023-11-07 10:27:33 +01:00
mmetc 41d19de092
Refact cwhub (#2578)
* Fix suggest functional tests
* comments
* non-empty SubItems() implies collections type
* use "slices" from stdlib
* No need to repeat author field in the index -- take it from the item key
2023-11-06 17:35:33 +01:00
Sebastien Blot 26c876dc38
merge hub-1.6 branch 2023-11-06 15:02:11 +01:00
mmetc 590a19b768
Refact pkg/cwhub: constructor, cscli output
* Single constructor: NewHub() to replace InitHub(), InitHubUpdate()
* sort cscli hub list output
* log.Fatal -> fmt.Errorf
2023-10-31 12:47:39 +01:00
mmetc 17662e59a9
Refact pkg/cwhub, cscli: hub upgrades (#2568)
* fix bats test for "upgrade all items"
* refact UpgradeConfig() -> UpgradeItem(): one item only
* refact RemoveMany() -> RemoveItem()
* Computed value: Item.Local -> Item.IsLocal()
* refact url/branch configuration with LocalHubCfg/RemoteHubCfg
2023-10-30 17:23:50 +01:00
mmetc ac98256602
Refact pkg/cwhub, cmd/crowdsec-cli (#2557)
- pkg/cwhub: change file layout, rename functions
 - method Item.SubItems
 - cmd/crowdsec-cli: generic code for hub items
 - cscli: removing any type of items in a collection now requires --force
 - tests
2023-10-20 14:32:35 +02:00
Sebastien Blot 350e8979b1
merge hub-1.5.6 branch 2023-10-19 12:18:16 +02:00
Sebastien Blot ecbdf2f0e1
merge master branch 2023-10-19 10:51:54 +02:00
mmetc be6555e46c
Refact pkg/csconfig, HubCfg (#2552)
- rename csconfig.Hub -> HubCfg
 - move some Load*() functions to NewConfig()
 - config.yaml: optional common section
 - remove unused working_dir
2023-10-18 09:38:33 +02:00
mmetc 325003bb69 Refact cscli item listing, tests (#2547)
* hub diet; taint tests
* cmd/crowdsec-cli: split utils.go, moved cwhub.GetHubStatusForItemType()
* cscli: refactor hub list commands, fix edge cases
2023-10-17 16:12:41 +02:00
mmetc f496bd1692 bats: more cscli hub tests (#2541)
- updated logs and user messages
- added func tests for all the items: install, remove, upgrade, list
- rewritten taint tests for collections
- removed redundant csconfig.LoadPrometheus()
2023-10-17 16:12:41 +02:00