Wikiless updated their instance json with a couple of changes that broke
Farside's auto update workflow:
- The protocol for each instance is now included by default (no need to prepend
"https://")
- The instances are differentiated between regular, onion, and i2p (no need to
check for ".onion" in regular instance URLs)
Added a new seperate service for only redirecting to SearXNG instances.
Note that plain "searx" redirects will use both SearX and SearXNG
instances for those who don't have a preference between the two.
Closes#23
- teddit (https://teddit.net/about)
A free and open source alternative Reddit front-end focused on
privacy. Inspired by the Nitter project.
- Piped (https://github.com/TeamPiped/Piped)
An alternative privacy-friendly YouTube frontend which is efficient
by design.
- SimplyTranslate (https://simple-web.org/projects/simplytranslate.html)
We aim to provide fast and private translations to the user without
wasting much overhead for extensive styling or JavaScript
___
Also adds SimplyTranslate to the github pipeline, since they provide a
list of the service's public instances.
Closes#4
The steps taken to commit the changes to services.json were overly complicated.
This simplifies the steps to just `exit 0` if there are no changes to commit.
* Create nightly update workflow for instances
A nightly GitHub Actions CI workflow has been added to fetch new
instances of supported services within Farside.
Currently only Searx is supported, but obviously others could be added
if there are similarly easy ways to fetch and filter instances
programmatically.
services.json has also been updated with the initial results of the
workflow script.
* Set headers for every HTTPoison request
This serves as a workaround for bot blocking via filtron.
* Expand filtering of searx instances
New filter enforces:
- No Cloudflare
- Good TLS config
- Good HTTP header config
- Vanilla instances or forks
- Instances with 100% search success
Rather than requiring a traditional crontab install, the app now
leverages quantum-core (link below) to schedule the instance update/sync
task every 5 minutes. Some updates as a result:
- The new job is scheduled at runtime in server.ex.
- The update.exs script was refactored to be compiled along with the
rest of the app as instances.ex.
- Scheduler and Server modules were added for creating and executing
the new update task
- All shell scripts were removed, as they are no longer needed
https://github.com/quantum-elixir/quantum-core
Now allows setting FARSIDE_TEST to skip individually fetching each
instance, and instead just adds all of them to redis instantly. This
allows for an easier time in CI builds, for both the sake of speed and
to prevent a scenario where many simultaneous builds have a noticeable
impact on actual instances.
