Although the current list of queries is only 2 elements ("time" and
"weather"), this allows searches for Whoogle and Searx to be slightly
more random, and potentially avoid any issues with their parent engine
rate limiting them.
* Create nightly update workflow for instances
A nightly GitHub Actions CI workflow has been added to fetch new
instances of supported services within Farside.
Currently only Searx is supported, but obviously others could be added
if there are similarly easy ways to fetch and filter instances
programmatically.
services.json has also been updated with the initial results of the
workflow script.
* Set headers for every HTTPoison request
This serves as a workaround for bot blocking via filtron.
* Expand filtering of searx instances
New filter enforces:
- No Cloudflare
- Good TLS config
- Good HTTP header config
- Vanilla instances or forks
- Instances with 100% search success
Should probably consider automating this at some point, but each instance
seemingly lists their public instances in a different format. Also don't really
want to mess with readme scraping. Idk, I'll think about it.
Whoogle is now eligible for Farside links. I haven't decided how I'd
like this implemented yet though. My idea is actually to refactor part
of Whoogle to work a bit better with this new "instance hopping"
paradigm -- namely, I'd like the config settings to be easily turned
into query params and appended to a farside link when a user's preferred
instance is blocked.
This introduces a number of new changes:
- Services are now inserted into redis with a prefix prepended to the
key name. This allows for easier filtering to get only live instances.
- The home page now uses an eex template for displaying all live
instances for every service, determined by the last update
- A "last_updated" field was added
- farside.ex was added to contain all functionality related to querying
for instances (WIP)
- Other improvements
Once a list of available URLs has been determined for a particular
service, the list is written as "service -> [list of instances]" to a
local redis connection. These can then be used in the greater routing
logic to pick a random instance from the list, or use a fallback
instance if none are determined to be available.
