Commit graph

1519 commits

Author SHA1 Message Date
Sebastien Blot 71059ca040
use query_range instead of tail 2023-10-17 14:25:36 +02:00
bui 9341d3fe32 improve bubble up errors and fix a few logging messages 2023-10-17 09:47:57 +02:00
bui 755ca0238b improve bubble up errors and fix a few logging messages 2023-10-17 09:47:41 +02:00
Thibault "bui" Koechlin 1a8c127529
Merge branch 'master' into loki 2023-10-16 18:02:14 +02:00
Laurence Jones 19de3a8a77
Runtime whitelist parsing improvement (#2422)
* Improve whitelist parsing

* Split whitelist check into a function tied to whitelist, also since we check node debug we can make a pointer to node containing whitelist

* No point passing clog as an argument since it is just a pointer to node we already know about

* We should break instead of returning false, false as it may have been whitelisted by ips/cidrs

* reimplement early return if expr errors

* Fix lint and dont need to parse ip back to string just loop over sources

* Log error with node logger as it provides context

* Move getsource to a function cleanup some code

* Change func name

* Split out compile to a function so we can use in tests. Add a bunch of tests

* spell correction

* Use node logger so it has context

* alternative solution

* quick fixes

* Use containswls

* Change whitelist test to use parseipsource and only events

* Make it simpler

* Postoverflow tests, some basic ones to make sure it works

* Use official pkg

* Add @mmetc reco

* Add @mmetc reco

* Change if if to a switch to only evaluate once

* simplify assertions

---------

Co-authored-by: bui <thibault@crowdsec.net>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-10-16 10:08:57 +01:00
Laurence Jones e7ad3d88ae
Clear up some community confusion (#2543) 2023-10-16 10:08:41 +01:00
Thibault "bui" Koechlin 3cd4847093
sort map keys when generating asserts (#2494)
* sort map keys when generating asserts
2023-10-16 09:54:19 +02:00
Louis PERDEREAU a3eff54665 feat: Refactor Tail and add metrics 2023-10-15 21:49:17 +02:00
Louis PERDEREAU e750412a39 feat: Add delay for on tail request 2023-10-15 21:49:17 +02:00
Louis PERDEREAU efc6213c79 fix: Update go.mod and go.sum 2023-10-15 21:49:17 +02:00
Louis PERDEREAU a3e41996d7 fix: Update go-cs-lib/cstest 2023-10-15 21:49:17 +02:00
Louis PERDEREAU fc33f8dfe6 fix: Patch loki acquisition 2023-10-15 21:49:17 +02:00
Louis PERDEREAU 8d05565cc3 fix: Path health command for loki 2023-10-15 21:49:17 +02:00
Louis PERDEREAU 5acf6a8bdc fix: Add loki readiness probe to CI 2023-10-15 21:49:17 +02:00
Louis PERDEREAU 9520670d97 feat: Add to detect test 2023-10-15 21:49:17 +02:00
Louis PERDEREAU 2017e6b113 fix: Add https to dsn 2023-10-15 21:49:17 +02:00
Louis PERDEREAU 810f525531 fix: Update auth yaml config 2023-10-15 21:49:17 +02:00
Louis PERDEREAU 784703ac64 fix: Add uniq id 2023-10-15 21:49:17 +02:00
Louis PERDEREAU 945a381b2e fix: Patch auth and since DSN 2023-10-15 21:49:17 +02:00
Louis PERDEREAU 3dadf6898a fix: Update 2023-10-15 21:49:17 +02:00
Sebastien Blot 8f3a20b7a4 linting 2023-10-15 21:49:17 +02:00
Sebastien Blot 2195509218 properly build URL 2023-10-15 21:49:17 +02:00
Sebastien Blot 3f56a0759f fix potential null deref 2023-10-15 21:49:17 +02:00
Sebastien Blot 47d85e36d2 use a dedicated package for loki queries 2023-10-15 21:49:17 +02:00
Mathieu Lecarme be95db0c02 Fix: don't break the loop, retry. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme e76d58c77a Fix: it's a StreamingAcquisition error, not an ugly copy/paste error from OneShotAcquisition function. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 4edd25fee9 Loki is now registered as an acquisition module. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 4bbe1c7b68 Stop the loki tailing. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 4bfb1385bc Websocket can be closed. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 473b64b376 Better error report, and more logical variable place. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme caa7db0f77 Fix: it's ok to stop the connection. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 1c88584d21 Don't buffer before reading JSON. JSON draining looks far better. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 3471ce066e Close the websocket. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 066d2a1dc5 Forever loop over loki websocket can be stopped. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme ae718e0557 Fix: unreachable return 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 1d7bce7064 Fix: path can be / or empty 2023-10-15 21:49:17 +02:00
Mathieu Lecarme e57150ae1d Fix: New request error handling. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 24f67c0fe5 Default config mode is tail. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 0dad75b91b Better wait. Defer Close. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 195adbadeb Wait for ready is a parameter. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 4f1aae754f Explicit loki error. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme d7a1c6a938 Paginate Loki query. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme a81d52a1ff FeedLoki routine. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 6bf80ee083 OneShot has its test 2023-10-15 21:49:17 +02:00
Mathieu Lecarme ea860b1087 Play this test. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme d8d63a2bf8 Fix: parse JSON loki response. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme bf138b9231 Loky query is mandatory 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 48431f20ed Specific time.Time type for YAML parsing. 2023-10-15 21:49:17 +02:00
Mathieu Lecarme 5edcdb6b57 Date or Duration diff 2023-10-15 21:49:17 +02:00
Mathieu Lecarme c11cbccb2d More interface implementation 2023-10-15 21:49:17 +02:00