Cristian Nitescu
7284c0a47a
retry with backoff requests to CAPI ( #1957 )
...
* backoff on refresh token error
* fix tls communication with lapi and user/pw auth (#1956 )
allow self-signed TLS encryption with user/pw auth
docker:
- remove defaults for certificate file locations
- new envvar INSECURE_SKIP_VERIFY
- register agent before TLS settings (cscli machine add removes them
from the credentials file)
* separate cscli cobra constructors: lapi, machines, bouncers, postoverflows (#1945 )
* use feature toggling to improve testability with http retry backoff
* Add parse unix to dateparse enricher (#1958 )
Add parse unix is we do have a strTime but wasnt parsed using convential golang time
* func tests: redirect stderr to filter extra logs (#1961 )
* backoff on refresh token error
* use feature toggling to improve testability with http retry backoff
* refactor feature backoff toggle for tests
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
2023-01-09 14:49:21 +01:00
blotus
a84e4b6b15
Add conditional bucket ( #1962 )
2023-01-06 09:26:16 +01:00
AlteredCoder
185f9ad541
Alert context ( #1895 )
...
Co-authored-by: bui <thibault@crowdsec.net>
2023-01-04 16:50:02 +01:00
mmetc
033082a31e
ParseUnix() test fix: force UTC ( #1970 )
2023-01-04 16:22:17 +01:00
mmetc
2d81e751a1
fix parser test 2k23 ( #1971 )
2023-01-04 15:46:16 +01:00
Laurence Jones
fd1c38811e
Add parse unix to dateparse enricher ( #1958 )
...
Add parse unix is we do have a strTime but wasnt parsed using convential golang time
2022-12-30 12:47:14 +00:00
mmetc
72c1753fb7
fix tls communication with lapi and user/pw auth ( #1956 )
...
allow self-signed TLS encryption with user/pw auth
docker:
- remove defaults for certificate file locations
- new envvar INSECURE_SKIP_VERIFY
- register agent before TLS settings (cscli machine add removes them
from the credentials file)
2022-12-29 22:00:11 +01:00
Laurence Jones
401739b036
Add unix expr helper ( #1952 )
...
* Add unix expr helper
* Add original value not parsed error
* return early if cannot parse
* Add tests
* Fix negative value
2022-12-29 14:53:06 +00:00
Thibault "bui" Koechlin
e4463c412b
Improve warnings around lack of evt.StrTime
field ( #1954 )
...
* fix #1951 : improve error messages
* make hubtest warn you if you're missing evt.StrTime in your logs
2022-12-29 15:03:32 +01:00
mmetc
6efc2688b1
simplify feature flags ( #1947 )
...
Now checking for a feature flag is a one liner,
with no need to control errors.
if fflag.Crowdsec.CscliSetup.IsEnabled() {
...
}
2022-12-26 14:23:41 +01:00
mmetc
5d2c99bb17
runtime feature flag initialization
2022-12-21 17:19:20 +01:00
mmetc
ff88faf402
updated localstack dependencies, added build cache
2022-12-21 12:20:01 +01:00
mmetc
a32aa96752
feature flags ( #1933 )
...
Package fflag provides a simple feature flag system.
Feature names are lowercase and can only contain letters, numbers, undercores
and dots.
good: "foo", "foo_bar", "foo.bar"
bad: "Foo", "foo-bar"
A feature flag can be enabled by the user with an environment variable
or by adding it to {ConfigDir}/feature.yaml
I.e. CROWDSEC_FEATURE_FOO_BAR=true
or in feature.yaml:
```
---
- foo_bar
```
If the variable is set to false, the feature can still be enabled
in feature.yaml. Features cannot be disabled in the file.
A feature flag can be deprecated or retired. A deprecated feature flag is
still accepted but a warning is logged. A retired feature flag is ignored
and an error is logged.
A specific deprecation message is used to inform the user of the behavior
that has been decided when the flag is/was finally retired.
2022-12-20 16:11:51 +01:00
he2ss
579cecde04
apiclient: fix http roundtrip (clone body also) ( #1758 )
...
* apiclient: fix http roundtrip (clone body also)
2022-12-14 16:42:46 +01:00
Laurence Jones
fe23da6e0c
Add postgres socket support, clean some code ( #1926 )
2022-12-12 16:08:19 +00:00
Laurence Jones
11965f08db
Add socket support to mysql ( #1911 )
2022-12-08 09:33:08 +00:00
mmetc
cc228f1868
Typos, grammar ( #1905 )
2022-12-06 15:55:27 +01:00
blotus
fdda940ac0
Add Kubernetes audit acquisition ( #1767 )
2022-12-06 13:47:29 +01:00
mmetc
fd3e668fe1
add -error flag to crowdsec binary ( #1903 )
2022-12-03 08:56:11 +01:00
mmetc
fa0e590778
removed pid_dir ( #1906 )
2022-12-02 13:42:43 +01:00
mmetc
4a6a9c4355
acquisition: validate datasources before configuration (static checks) ( #1841 )
...
* acquisition: validate datasources before configuration (allow static configuration checks)
* remove comment
* import reviser, format
* error wrap
2022-11-30 17:36:56 +01:00
blotus
60f1228030
use a copy of bucket processors in LeakRoutine ( #1902 )
2022-11-30 10:59:47 +01:00
mmetc
104f5d1fe6
lint: error handling cleanup ( #1877 )
2022-11-29 09:16:07 +01:00
mmetc
66543493b5
fix nil dereference: check that httpServer is set before shutting down ( #1893 )
2022-11-28 11:55:08 +01:00
mmetc
fde9640364
Docker refactoring, tls setup ( #1869 )
2022-11-28 10:35:12 +01:00
blotus
c5079ac15e
invalidate agent token on 403 as well ( #1888 )
2022-11-25 14:35:50 +01:00
mmetc
5bdd3bbfcb
require at least go 1.18 to build ( #1884 )
2022-11-24 11:29:54 +01:00
Laurence Jones
4ac01ed880
Update perms for group read ( #1876 )
2022-11-21 09:49:56 +00:00
mmetc
3beb84bcfe
print missing "AS" values as empty strings instead of "0 " ( #1867 )
2022-11-14 09:55:53 +01:00
Thibault "bui" Koechlin
523343b174
notify when community-blocklist starts pull ( #1845 )
...
* minor change to notify blocklist pull update, will make eventual troubleshooting easier
2022-11-08 10:44:25 +01:00
Thibault "bui" Koechlin
3b4da7e637
fix #1860 : Only repeat the WAL warning once ( #1863 )
...
* fix #1860
2022-11-07 16:36:39 +01:00
mmetc
895691dad1
enabled linters: gocritic, nilerr ( #1853 )
2022-11-07 10:36:50 +01:00
Manuel Sabban
8aca00326d
fix ticker ( #1858 )
...
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-11-04 13:56:43 +01:00
Laurence Jones
668627f890
Add error checking to lookup host ( #1847 )
2022-10-31 18:38:01 +00:00
mmetc
344b1dc559
fixed package tests w/wal, gitignore/typos ( #1849 )
2022-10-31 10:02:51 +01:00
mmetc
df88f4e1e9
randomize pull, push and metric intervals; reload crowdsec only when hub changed ( #1846 )
2022-10-28 13:55:59 +02:00
mmetc
02d2eab18c
update golangci-lint to 1.50 and fixes ( #1828 )
2022-10-26 15:11:37 +02:00
ThinkChaos
22479a289d
Add LookupHost expr lib func ( #1775 )
2022-10-26 10:17:48 +01:00
mmetc
2088bb1f91
fix for #1839 ( #1840 )
2022-10-26 11:02:12 +02:00
blotus
b7c4bfd4e3
Use explicit transaction when inserting community blocklist ( #1835 )
2022-10-26 10:48:17 +02:00
mmetc
e545933923
fix(cscli): correct and test the behavior of "cscli collections delete" ( #1824 )
2022-10-25 14:10:51 +02:00
blotus
bb2f0e938f
Blocklist: Do not duplicate decisions when pulling ( #1796 )
2022-10-19 15:51:40 +02:00
Thibault "bui" Koechlin
ae6bf39495
support decisions deletion via scenario + alerts delete via ID ( #1798 )
2022-10-19 14:37:27 +02:00
mmetc
6b0097a24b
change warning to debug when directories are missing in hub sync ( #1819 )
2022-10-18 10:32:54 +02:00
mmetc
2b7e3ff1e7
warn if no acquisition files are found, acquisition_test refactoring, tests ( #1816 )
2022-10-17 17:32:08 +02:00
mmetc
ec0d2a5ed2
refactor broker_test.go, extract cstest/filenotfound*.go ( #1815 )
2022-10-17 14:17:23 +02:00
mmetc
a96b3e077d
rename pkg/cstest -> pkg/hubtest ( #1811 )
...
keep cstest for generic helper functions
this also avoids circular imports in test files
2022-10-17 09:24:07 +02:00
mmetc
8fecc2c00b
enable staticcheck linter; fixes ( #1806 )
...
- explicitly ignore returned parameters
- replace Walk with faster WalkDir
- log path error during hub dir sync
- colorize static unit tests
- removed duplicate import in crowdsec/main.go
- typos
- func tests: default datasource in tests/var/log instead of /tmp
- action setup-go v3
2022-10-14 16:12:21 +02:00
Manuel Sabban
7359586f1c
fix ticker mix up ( #1807 )
...
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-10-13 14:30:27 +02:00
mmetc
4b3c9c2806
print cscli usage in color, fix windows terminal detection ( #1801 )
2022-10-13 12:28:24 +02:00
mmetc
7674f907c4
replace log.Fatal with t.Fatal ( #1805 )
...
This is required to run deferred teardown functions
2022-10-13 10:42:46 +02:00
mmetc
1d9f861f28
unit tests: always capture testcase variable -> allow parallel testing ( #1797 )
2022-10-10 10:48:26 +02:00
Shivam Sandbhor
74659a82ab
Fast bulk alert delete ( #1791 )
2022-10-07 12:40:30 +02:00
mmetc
ddd75eae9a
cscli: new tables, --color yes|no|auto option ( #1763 )
2022-10-07 11:05:35 +02:00
AlteredCoder
b95a67751e
Update ent and grokky package ( #1772 )
...
* Update ent and grokky package
2022-10-06 14:55:42 +02:00
Manuel Sabban
83841d801c
fork dlog to ease debian packaging on official repos ( #1790 )
...
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-10-06 13:40:31 +02:00
Shivam Sandbhor
65c0b9ebcf
Simplify one shot tests ( #1786 )
2022-10-06 11:57:26 +02:00
blotus
3ba67bad3d
remove a wrong warning when pulling list content from CAPI ( #1789 )
2022-10-06 11:48:06 +02:00
mmetc
9b3be5c2e8
Bulk delete alert optimization ( #1782 )
2022-10-05 17:07:44 +02:00
Shivam Sandbhor
b203b3f444
Fix flakey test in file_tests ( #1783 )
...
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2022-10-05 16:40:09 +02:00
mmetc
6120571421
fix & cleanup cloudwatch_test.go ( #1780 )
2022-10-04 09:48:59 +02:00
mmetc
edced6818a
cleanup + fix flaky tests in file_test.go, apic_test.go ( #1773 )
2022-09-30 16:01:42 +02:00
blotus
bfbe180101
Tighten windows sqlite database permissions ( #1769 )
2022-09-28 16:18:00 +02:00
Sean Kelly
568eb1d4e0
Fix misspelling of instantiate participles ( #1759 )
2022-09-27 17:13:43 +02:00
Laurence Jones
21e5b0d6d0
Improvement: Docker one shot error message ( #1666 )
...
* In one shot, user would only specify one container?
2022-09-27 16:20:30 +02:00
Manuel Sabban
1f06f242cc
fix https://github.com/crowdsecurity/crowdsec/issues/1746 ( #1749 )
...
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-09-14 15:26:26 +02:00
blotus
9b3ff82542
add WAL support for sqlite ( #1752 )
2022-09-14 15:09:54 +02:00
AlteredCoder
7d97729eea
Add config option to enable or not local API and agent ( #1730 )
...
* Add flag to enable or not local API and agent
2022-09-12 14:38:29 +02:00
AlteredCoder
b06167a3fa
Allow plugins to load environment variable ( #1727 )
...
* Allow plugins to load environment variable
2022-09-08 11:41:28 +02:00
Manuel Sabban
b2130b1593
Fix 1737 ( #1738 )
...
* add GetMeta to *types.Event
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-09-07 10:11:39 +02:00
Thibault "bui" Koechlin
9d199fd4a9
fix #1733 : add support for exclusion regexps ( #1735 )
...
* allow to specify a list of regular expressions to skip some specific files
2022-09-06 14:58:37 +02:00
mmetc
414282a2c9
golangci-lint 1.49 and related fixes ( #1736 )
2022-09-06 13:55:03 +02:00
Laurence Jones
e674537d0b
Update sprig to v3 ( #1722 )
...
* Update sprig to v3
2022-09-05 09:05:50 +02:00
he2ss
ea40ffd655
Datasource/kafka ( #1698 )
...
* add Kafka datasource
2022-08-30 17:03:45 +02:00
Manuel Sabban
7d0f89df29
Implement reinject command to send notifications of alerts ( #1638 )
...
* implement reinject command to send notifications of alerts using a profile
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-08-30 15:45:52 +02:00
Thibault "bui" Koechlin
bacea50485
allow user to disable decision deduplication ( #1687 )
...
* allow user to disable decision deduplication
2022-08-26 14:17:46 +02:00
blotus
1f5224b74b
switch to go 1.19 ( #1709 )
2022-08-26 13:31:49 +02:00
mmetc
eea07b7a1a
golangci-lint v1.48 and fixes for "usestdlibvars" ( #1711 )
2022-08-16 09:46:10 +02:00
AlteredCoder
1002affc16
cscli machines delete: return an error if machines doesn't exist ( #1689 )
...
* cscli machines delete: return an error if machines doesn't exist
2022-07-28 17:32:12 +02:00
Thibault "bui" Koechlin
866c200c31
Generic dateparse approach ( #1669 )
...
* Allow any parser to suggest a format string for the date to be parsed.
* allow the enricher functions to get the parser's logger so they can inherit the level
2022-07-28 16:41:41 +02:00
Thibault "bui" Koechlin
0eea20fa7c
revert decision dedup behavior to 1.3.4 ( #1675 )
...
* revert decision dedup behavior to 1.3.4
2022-07-22 11:20:10 +02:00
Thibault "bui" Koechlin
bd91ddaf52
logging consistency for .local files ( #1655 )
2022-07-13 10:56:03 +02:00
blotus
7b8cd63b04
do not set the UDP read buffer size in syslog datasource ( #1657 )
2022-07-13 10:18:03 +02:00
AlteredCoder
39da36361c
Get geoip Country from other objects if not present ( #1659 )
2022-07-12 15:26:34 +02:00
Thibault "bui" Koechlin
73f336363a
bump log level when overloading config file with .local
( #1646 )
...
* bump log level
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2022-07-08 09:29:23 +02:00
blotus
5c1c941851
close response body in heartbeat ( #1637 )
2022-07-06 14:50:54 +02:00
AlteredCoder
5f62d738fc
Add no-capi flag and review some logs ( #1628 )
...
* Add no-capi flag and review some logs
2022-07-01 16:56:13 +02:00
Thibault "bui" Koechlin
ca4cd6d559
attempt to fix ticker leak ( #1620 )
2022-06-30 17:36:01 +02:00
AlteredCoder
02e0f3c095
Fix event.timestamp pointer usage ( #1621 )
...
* Fix event.timestamp pointer usage
* avoid returning an error when creating alerts if something goes wrong during the parsing
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2022-06-30 17:35:42 +02:00
blotus
863facaa33
Properly handle expired decisions with different scopes ( #1616 )
2022-06-29 16:13:04 +02:00
Thibault "bui" Koechlin
15902dcba6
fix #1615 : cleanup based on heartbeat instead ( #1617 )
2022-06-29 13:21:58 +02:00
mmetc
8e7e799304
[wip] serve metrics only after agent and/or lapi are ready; fixed some func tests ( #1613 )
2022-06-24 15:55:21 +02:00
he2ss
7fafb483ad
Pkg/database/fix count decisions since by value ( #1606 )
...
* fix CountDecisionsSinceByValue to have also expired decisions
2022-06-23 12:04:07 +02:00
AlteredCoder
a4f4eabf0a
support yml file ( #1605 )
2022-06-22 17:01:27 +02:00
mmetc
628d7be1d8
simplify err.Error() to err when used in printf context ( #1603 )
2022-06-22 15:53:53 +02:00
he2ss
3d6f015211
Add duration expr to add duration formula ( #1556 )
...
* add duration expr to add duration formula
2022-06-22 11:29:52 +02:00
Thibault "bui" Koechlin
a6ed08b239
Add alerts and decisions metrics, LAPI and agent timing prom metrics ( #1546 )
2022-06-22 11:14:34 +02:00
AlteredCoder
0a39066f9d
Fix #1552 ( #1569 )
2022-06-22 10:29:02 +02:00
mmetc
d71279f023
added flag crowdsec --warning ( #1461 )
2022-06-22 09:38:23 +02:00
mmetc
c78c833400
CI: colored test output, colored crowdsec and crowdsec-api logs, full final db dump for mysql and sqlite ( #1596 )
...
* github-ci: color unit test output and logs
* new config option: force_color_logs (useful in CI)
* bats: show sqlite/mysql dump at the end
* removed "-v" (print package names) from "go build"
* general workflow cleanup
2022-06-17 16:12:49 +02:00