Commit graph

122 commits

Author SHA1 Message Date
blotus 772d5b5c32
Add experimental support for re2 (#2138) 2023-03-28 16:26:47 +02:00
blotus 1095f6c875
use expr.Function for custom functions instead of passing them in the env (#2133) 2023-03-28 10:49:01 +02:00
blotus dc38e5ac00
S3 acquisition datasource (#2130) 2023-03-21 13:54:52 +01:00
blotus b1f2063a9a
Only support pgx driver for postgresql (#2118) 2023-03-16 11:02:31 +01:00
blotus 6aaf3cd50b
Update expr to 1.12.2 (#2110) 2023-03-09 16:56:11 +01:00
Thibault "bui" Koechlin d95b7afe61
Distance support : Impossible travel (#2108)
* add distance helpers
2023-03-08 18:29:42 +01:00
dependabot[bot] 6888fa2133
Bump golang.org/x/net from 0.0.0-20220722155237-a158d28d115b to 0.7.0 (#2089)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.0.0-20220722155237-a158d28d115b to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/commits/v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-02 14:46:58 +01:00
dependabot[bot] 9ae99964e6
Bump golang.org/x/text from 0.3.7 to 0.3.8 (#2072)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.3.7 to 0.3.8.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.3.7...v0.3.8)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-01 09:30:50 +01:00
dependabot[bot] 19a01d20dd
Bump github.com/containerd/containerd from 1.6.12 to 1.6.18 (#2060) 2023-02-20 11:01:27 +01:00
dependabot[bot] 65fa2bf8c3
Bump github.com/docker/distribution (#1996) 2023-02-20 10:34:52 +01:00
mmetc b6be18ca65
cscli setup (#1923)
Detect running services and generate acquisition configuration
2023-02-06 07:33:04 +01:00
Thibault "bui" Koechlin e927717fa0
Polling API Integration (#1715)
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-31 14:47:44 +01:00
Thibault "bui" Koechlin 4f29ce2ee7
CTI API Helpers in expr (#1851)
* Add CTI API helpers in expr
* Allow profiles to have an `on_error` option to profiles

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-19 08:45:50 +01:00
dependabot[bot] 942aed1219
Bump github.com/containerd/containerd from 1.6.2 to 1.6.12 (#1978)
Bumps [github.com/containerd/containerd](https://github.com/containerd/containerd) from 1.6.2 to 1.6.12.
- [Release notes](https://github.com/containerd/containerd/releases)
- [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md)
- [Commits](https://github.com/containerd/containerd/compare/v1.6.2...v1.6.12)

---
updated-dependencies:
- dependency-name: github.com/containerd/containerd
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: blotus <sebastien@crowdsec.net>
2023-01-12 17:25:53 +01:00
Thibault "bui" Koechlin 6fb962a941
Allow parsers to capture data for future enrichment (#1969)
* Allow parsers to capture data in a cache, that can be later accessed via expr helpers (fake multi-line support)
2023-01-11 15:01:02 +01:00
mmetc a32aa96752
feature flags (#1933)
Package fflag provides a simple feature flag system.

 Feature names are lowercase and can only contain letters, numbers, undercores
 and dots.

 good: "foo", "foo_bar", "foo.bar"
 bad: "Foo", "foo-bar"

 A feature flag can be enabled by the user with an environment variable
 or by adding it to {ConfigDir}/feature.yaml

 I.e. CROWDSEC_FEATURE_FOO_BAR=true
 or in feature.yaml:
```
 ---
 - foo_bar
```

 If the variable is set to false, the feature can still be enabled
 in feature.yaml. Features cannot be disabled in the file.

 A feature flag can be deprecated or retired. A deprecated feature flag is
 still accepted but a warning is logged. A retired feature flag is ignored
 and an error is logged.

 A specific deprecation message is used to inform the user of the behavior
 that has been decided when the flag is/was finally retired.
2022-12-20 16:11:51 +01:00
blotus fdda940ac0
Add Kubernetes audit acquisition (#1767) 2022-12-06 13:47:29 +01:00
Manuel Sabban 3d72ca731a
Suggest bouncers and machines to delete (#1896)
* Suggest bouncers to delete

* Autocomplete machines delete cmd

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* Fix lint.

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* fix compilation (git merge errors)

* cleanup go.mod unneeded changes

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Co-authored-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-12-05 15:39:54 +01:00
mmetc 487bf4e74a
require go 1.19 for plugins; require crowdsec 1.4.1; go mod tidy (#1823) 2022-10-18 17:01:36 +02:00
mmetc 4b3c9c2806
print cscli usage in color, fix windows terminal detection (#1801) 2022-10-13 12:28:24 +02:00
mmetc ddd75eae9a
cscli: new tables, --color yes|no|auto option (#1763) 2022-10-07 11:05:35 +02:00
AlteredCoder b95a67751e
Update ent and grokky package (#1772)
* Update ent and grokky package
2022-10-06 14:55:42 +02:00
Manuel Sabban 83841d801c
fork dlog to ease debian packaging on official repos (#1790)
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-10-06 13:40:31 +02:00
mmetc d4a7288826
spf13/cobra v1.5.0; antonmedv/expr v1.9.0 (#1756) 2022-09-27 16:28:07 +02:00
Laurence Jones e674537d0b
Update sprig to v3 (#1722)
* Update sprig to v3
2022-09-05 09:05:50 +02:00
he2ss ea40ffd655
Datasource/kafka (#1698)
* add Kafka datasource
2022-08-30 17:03:45 +02:00
blotus e46ca38cbb
add cscli support dump (#1634) 2022-08-18 11:54:01 +02:00
AlteredCoder fe5f9bfc28
add suggestion on cscli install items (#1686) 2022-08-04 10:09:56 +02:00
Thibault "bui" Koechlin fe09737d80
Add support for machine heartbeat (#1541)
* add the last_heartbeat field

* add heartbeat controller

* add endpoint of heartbeat

* heartbeat integration

* add last_heartbeat to cscli machines list
2022-05-19 15:47:27 +02:00
Laurence Jones 6d6d82b3af
Memory check for cscli dashboard setup (#1513)
* Add 1gb recmem variable and use memory module

Since checking the RAM is not required to get the container up and running we can change this to a warn level
2022-05-18 11:05:01 +02:00
blotus 635e633520
update machineid to 1.0.2 (#1533) 2022-05-17 18:59:53 +02:00
blotus 0449ec1868
Windows Support (#1159) 2022-05-17 12:14:59 +02:00
blotus 64369b5c2b
add expr XML helpers (#1493) 2022-04-29 13:52:23 +02:00
Manuel Sabban 2e37d5ce97
update machineid lib (#1489)
* update machineid lib

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-04-28 12:18:16 +02:00
Thibault "bui" Koechlin ef20183ecb
go mod update for 1.3.3 (#1462) 2022-04-20 12:57:05 +02:00
mmetc dad22a6aba
instrument main() for tests (#1399) 2022-04-01 11:17:45 +02:00
Thibault "bui" Koechlin d8dc01cd94
Revamp unit tests (#1368)
* Revamp unit tests
* Increase coverage
* Use go-acc to get cross packages coverage

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2022-03-29 14:20:26 +02:00
Shivam Sandbhor c5566e92f3
Fix 1262 pgsql conflict resolve (#1363)
* Fix api for all dbs (#1310)
* DB agnostic lapi sanitize

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* Update ent

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* Fix go dep mess.

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2022-03-17 14:12:13 +01:00
Shivam Sandbhor bb30a3f966
Don't omit fields of bouncer in json (#1354)
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2022-03-16 09:40:34 +01:00
mmetc 10ce45c054
allow notification plugins to work on freebsd and non-root functional tests (#1253)
* random uuid for all platforms
* check group writable and setgid; don't check group ownership
* allow user to run plugins without changing desired user/group (set them to "")
2022-03-09 12:09:50 +01:00
Thibault "bui" Koechlin b66366c28c
Revert "Handle decisions with varying expiry for same IP (#1262)" (#1308)
This reverts commit e4f6cdfc14.
2022-03-04 10:17:31 +01:00
Shivam Sandbhor e4f6cdfc14
Handle decisions with varying expiry for same IP (#1262)
* Upgrade ent and add sql/modifier in codegen

* update db wrappers to sanitize LAPI

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2022-02-16 15:19:14 +01:00
AlteredCoder 5a0843852a
add IpToRange helpers and allows to have an expression with scope Range (#1260)
* add IpToRange helpers and allows to have an expression with scope Range
2022-02-14 16:50:52 +01:00
AlteredCoder b93b8d9a2e
Support PGX (#1186)
* Support PGX

* support sslmode
2022-01-20 11:17:21 +01:00
Thibault "bui" Koechlin 3b04bd3b5b
upgrade grokky following https://github.com/crowdsecurity/grokky/pull/2 (#1187) 2022-01-20 10:51:29 +01:00
Thibault "bui" Koechlin 40ed810c0b
Gin upgrade (#1174)
* upgrade gin / gin-jwt, and add a new 'trusted_proxies' option to provide trusted CIDRs
2022-01-17 17:18:12 +01:00
Thibault "bui" Koechlin 6e92da76ad
lapi to capi : allow push of tainted/custom/manual decisions (#1154)
* add console command to control signal sharing
* modify metrics endpoint to add lastpush

Co-authored-by: alteredCoder <kevin@crowdsec.net>
2022-01-13 16:46:16 +01:00
blotus cc72800f50
Update LAPI swagger (#1155) 2022-01-11 16:45:34 +01:00
blotus 4a11060930
Kinesis datasource (#1147) 2022-01-11 14:19:43 +01:00
blotus ec53fbfdab
require go 1.17 (#1104) 2021-12-16 14:39:58 +01:00
AlteredCoder 88d06260d7
add cscli decisions import (#1038)
* add cscli decisions import

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: bui <thibault@crowdsec.net>
2021-12-15 11:39:37 +01:00
AlteredCoder 4917aa23c9
Docker datasource (#1064)
* add docker datasource
2021-12-02 15:55:50 +01:00
Thibault "bui" Koechlin d1ce543440
Improve explain (#1039)
* improve explain feature

* nicer display for details, --verbose in favor of --debug for details
2021-11-02 12:06:01 +01:00
blotus 25a2d528b0
Alerts flush: Optimization of the flush mechanism (batch and limit to one job) + add cscli alerts flush command (#1024)
- Don't allow running more than one alert flush job at a time to prevent runaway CPU usage in some case. (fix High CPU after Upgrade to 1.2.0 #1022)
 - Add a cscli alerts flush command to manually flush the alerts in the database (fixes Improvement/Manual flush mechanism #1023 ).
 - Enable cascading deletion on alerts as we upgraded ent: Deleting an alert in the database will automatically delete all related decisions, events and meta
 - Add an index on alerts.id to try to improve flush performance with very big sqlite database.
- Flush alert now operates in batch
2021-10-26 13:33:45 +02:00
Thibault "bui" Koechlin 3f99330b3d
Entgo 0.9 (#1018)
* update entgo & sqlite to latest version

* schema update
2021-10-22 16:15:57 +02:00
blotus bd5c119f85
update golang.org/x/sys dep (#983) 2021-09-21 17:06:40 +02:00
blotus 7a1b955ad1
use our fork of grokky (#953) 2021-09-09 14:46:16 +02:00
Shivam Sandbhor 899b2abae7
Avoid code duplication for protobuf in plugins (#918)
* Avoid code duplication for protobuf in plugins

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2021-09-03 12:24:59 +02:00
Thibault "bui" Koechlin 950759f6d6
Output plugins (#878)
* Add plugin system for notifications (#857)
2021-08-25 11:43:29 +02:00
Nanik b0746fbc4d
fix: add /health endpoint (#881)
* fix: add /health endpoint
2021-08-18 09:06:01 +02:00
Thibault "bui" Koechlin ce6a61df1c
Refactor Acquisition Interface (#773)
* Add new acquisition interface + new modules (cloudwatch, syslog)

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2021-06-11 09:53:53 +02:00
Thibault "bui" Koechlin 28446b6d29
Ent update : 0.7.0 (#692)
* up regenerate new schema

* new ent

* update documentation for min required versions

* update documentation
2021-03-15 18:46:52 +01:00
registergoofy 5b7ac4a473
[Rebased] fix races (#633)
* get rid of dead code
* have LeakRoutined started in a tomb
* fix race and multiple small issues in the way we handle tombs
* yet another race fix
* another race
* get rid of leaky.KillSwitch for proper tomb use
* fix deadlock
* empty overflow before exiting
* fix an obvious typo
* proper use of waitgroup
* have a smart signalisation for allowing LeakRoutine being killed
* ugly workaround
* fix lint error
* fix compilation
* fix panic
* shorten lock
* up lock both copy
* wait for crowdsec to die
* fix coding style and lint issue
* go mod tidy

Co-authored-by: bui <thibault@crowdsec.net>
2021-02-25 11:26:46 +01:00
AlteredCoder 5544000d38
lapi: fix ipv6 operations (#567) 2021-01-14 16:27:45 +01:00
Thibault "bui" Koechlin b15fc96ef8
go mod tidy (#566) 2021-01-13 10:07:59 +01:00
AlteredCoder c2517e8eb4
fix docker container creation for metabase (#563) 2021-01-08 14:32:29 +01:00
Thibault "bui" Koechlin f25acab313
remove tachymeter (#561) 2021-01-07 14:07:05 +01:00
Thibault "bui" Koechlin dbb420f79e
local api (#482)
Co-authored-by: AlteredCoder
Co-authored-by: erenJag
2020-11-30 10:37:17 +01:00
registergoofy c6aab9893a
add randomness to machine-id when registering. (#261)
* add randomness to machine-id when registering.

* add some regexp check for machine_id

* typo fix

* fix cwapi unit tests
2020-09-29 13:17:33 +02:00
AlteredCoder 7f45c43eb1 change github workflow to get latest release 2020-08-28 16:37:35 +02:00
Thibault bui Koechlin 32f32b41c7 add json support via expr helpers 2020-05-22 18:12:33 +02:00
Thibault bui Koechlin 2016167654 initial import 2020-05-15 11:39:16 +02:00