Commit graph

233 commits

Author SHA1 Message Date
Vishal 92715b658c Change API parameter 2024-04-03 19:24:12 +05:30
Vishal 2ddf4c897c Rectify if else 2024-04-03 18:25:41 +05:30
Vishal ffefae89a6 Redefine struct 2024-04-03 17:50:53 +05:30
Vishal 39ec761949 fix warnings 2024-04-03 12:57:37 +05:30
Vishal 01f842c445 Rearrange methods 2024-04-03 12:41:18 +05:30
Neeraj Gupta 2fe703df92
[server] Increase embedding fetch limit (#1300)
## Description

Also use different semaphore than existing diff API

## Tests
2024-04-03 12:38:34 +05:30
Vishal 18c48c7e0a Fix typo in comment 2024-04-03 11:14:55 +05:30
Manav Rathi 885308471f
[server] Add a notice that minio is only for getting started
From our Discord, someone commented

> if minio's cautions about Single-Node, Single-Drive are to be taken seriously:
>
> "SNSD deployments use a zero-parity erasure coded backend that provides no
  added reliability or availability beyond what the underlying storage volume
  implements. These deployments are best suited for local testing and
  evaluation, or for small-scale data workloads that do not have availability or
  performance requirements."

MinIO was never meant as a production replacement, it was only to make it easy
for people to get started. So add a notice in the docs re this.
2024-04-03 10:01:48 +05:30
Neeraj Gupta 85778bcdaa [server] Fix handling of redundant auth update 2024-04-03 07:05:03 +05:30
Vishal b8100b1273 rename functions 2024-04-02 17:45:30 +05:30
Vishal a5fcbbf901 add listmonk mailing list control (subscribe/usubscribe) 2024-04-02 17:36:53 +05:30
Neeraj Gupta 25cedd5e2f
[server] Gracefully handle stale collection entry (#1260)
## Description

## Tests
2024-03-30 15:24:52 +05:30
Neeraj Gupta f6dca2dfc9
[server] Drop locationTag table and related code (#1245)
## Description

## Tests
2024-03-29 16:25:39 +05:30
Neeraj Gupta ef250acad9
[server] Add person EntityType (#1246)
## Description

## Tests
2024-03-29 16:12:59 +05:30
Neeraj Gupta 459c4515a0 [server] Drop locationTag table and related code 2024-03-29 16:08:28 +05:30
Manav Rathi 54d93647ef
Add SMTP for sending emails (#1228)
## Tests
Tested for templated mails in user.go, userauth.go, stripe.go,
public_collection.go, playstore.go, admin.go, email_notification.go

Screenshots for all the templates -
[https://drive.google.com/drive/folders/1f7BjM5MHicRUbr5jlCe2-4MGz7s1moLw?usp=sharing](url)
2024-03-29 09:22:40 +05:30
Vishal d5f0334a34 Remove isEnabled from config 2024-03-28 17:44:08 +05:30
Vishal 1f75633c5c Private method to bottom of the file 2024-03-28 15:03:27 +05:30
Manav Rathi 2b379640ca
Allow billing plans to be mounted into the Docker container if needed 2024-03-28 13:07:43 +05:30
Vishal cc3376154f Change name 2024-03-27 22:39:12 +05:30
Vishal a7692b7d2e Update if else 2024-03-27 21:40:40 +05:30
Vishal 9893e3f305 Add Amazon SES for sending mails 2024-03-27 19:58:27 +05:30
Vishnu Mohandas eef33e9c0c
[mobile][web] Redirect to payment portal if subscription is past due (#1222)
## Description

When a customer whose Stripe subscription is past due (within the 30 day
window after expiry time and has not been cancelled) clicks on the
subscription modal, take them to the payment portal to complete the
subscription.

## Tests

- [x] Tested web
- [x] Tested mobile
2024-03-27 16:29:58 +05:30
vishnukvmd 44403c94c2 Cancel SEPA subscriptions when payments fail 2024-03-27 16:20:58 +05:30
Vishnu Mohandas 0c400db649
[server] Fix Stripe related issues (#1152)
## Description

### Stripe
- Settings have been modified to "Cancel" the subscription if all
attempts to charge an invoice fails. The previous configuration was to
set the subscription to "Past Due".

### Museum
- Updated our Stripe workflow to use `PaymentBehaviorAllowIncomplete` in
case of SEPA payments
- Fixed the issue caused by us spawning unnecessary subscriptions when a
customer tried to renew their expired subscription
- Removed the email that was being sent out when a Stripe subscription
expired. We will have to run our own cron to send a notification and
perform necessary clean up. When payments fail, we will still send a
notification to intimate the customer that their account is on hold.

### Mobile
- Fixed the dialogs and error messages on the payment page

## Tests
The following cases were tested for SEPA, Credit Cards, and Credit Cards
with 3DS.
- [x] Purchase
  - [x] Success
  - [x] Failure
- [x] Renewal
  - [x] Success
  - [x] Failure
- [x] Upgrades
  - [x] Success
  - [x] Failure
- [x] Cancellation
- [x] Resumption post recovery

## Note

These changes are likely to break in the event of some corner cases we
haven't thought of. But this will land us in a state that is better than
where we are right now in terms of collecting payments for expired
subscriptions and those via SEPA.
2024-03-27 14:43:36 +05:30
vishnukvmd 25a054287a Prevent duplicate emails for SEPA 2024-03-27 14:22:30 +05:30
Manav Rathi e803b7f053
Document in main README 2024-03-27 13:44:24 +05:30
Manav Rathi 4a4a6ba44a
Give an example of the pull command 2024-03-27 13:04:28 +05:30
Manav Rathi cb45a070be
Tweaks 2024-03-27 12:58:17 +05:30
Manav Rathi 4ba76e4aa2
Tweaks 2024-03-27 12:46:08 +05:30
Manav Rathi 7c0f41166c
Add a first cut of the publish script 2024-03-27 12:42:29 +05:30
Manav Rathi b31332ff30
Outline the plan 2024-03-27 12:42:26 +05:30
vishnukvmd c9618d8038 Update copy 2024-03-26 21:31:29 +05:30
vishnukvmd 888aca7e6c Extract common code 2024-03-26 21:11:57 +05:30
vishnukvmd 31d3319d20 Revert "Depend on subscription-past-due events to trigger the account-on-hold email"
This reverts commit a131c16f4e.
2024-03-26 20:59:02 +05:30
vishnukvmd a131c16f4e Depend on subscription-past-due events to trigger the account-on-hold email 2024-03-26 20:29:55 +05:30
vishnukvmd f2aac262b1 Remove unused flag 2024-03-25 23:27:39 +05:30
vishnukvmd 4afc056a9f Stop listening to subscription deleted events 2024-03-25 23:26:48 +05:30
vishnukvmd 2dcf5daf54 Handle cases where default payment method might not be specified 2024-03-25 23:16:20 +05:30
vishnukvmd 29245b73b9 Cancel subscriptions on SEPA failures immediately 2024-03-25 23:15:43 +05:30
vishnukvmd d003542e42 Simplify how last payment method is fetched 2024-03-25 23:13:56 +05:30
Manav Rathi a4ecc2c344
Update ente.io/faq => help.ente.io where possible
On desktop, now there Help menu will have an "Ente Help" option instead of "FAQ"
(Unrelated: I noticed that user facing strings in the desktop app are not
localized).

The only place remaining after this is the sidebar on mobile
(`support_section_widget.dart`), but I wasn't sure of what string to use (and
how to add it to the localized strings for mobile) so I've left it unchanged.
2024-03-25 19:26:10 +05:30
vishnukvmd a275ab7740 Set subscription to expire 7 days after payment failure 2024-03-23 14:37:29 +05:30
vishnukvmd e634310cb1 Remove unused method 2024-03-21 14:49:37 +05:30
vishnukvmd b95d7c241d Create new subs only if the existing subscription is in a cancelled state 2024-03-21 14:48:29 +05:30
vishnukvmd 3cf680eec1 [WiP] Prevent subscription cancellation 2024-03-21 14:42:38 +05:30
vishnukvmd 1f17f2442a Handle SEPA failures 2024-03-21 14:33:10 +05:30
vishnukvmd 859af52c64 Remove commented code 2024-03-21 13:55:42 +05:30
vishnukvmd 78e5e3bdb1 Minor refactor 2024-03-21 13:31:46 +05:30
vishnukvmd 5635e63cb7 Update handling for subscription updated events 2024-03-21 13:29:42 +05:30
Abhinav 520f7ab64c only use PaymentBehaviorAllowIncomplete for SEPA payment method 2024-03-21 07:18:28 +05:30
Abhinav 4e074770d2 invoice client specific invoice api 2024-03-21 06:48:49 +05:30
vishnukvmd 49893d9732 Update on-hold email subject 2024-03-20 14:31:33 +05:30
vishnukvmd db81f52cf1 Update mail when subscription is on-hold 2024-03-20 14:31:01 +05:30
vishnukvmd 628565cfaf Merge branch 'main' into sepa-fix 2024-03-20 14:08:36 +05:30
Vishnu Mohandas 8101fe1495
Patch few Stripe related issues (#1156) 2024-03-20 14:07:26 +05:30
vishnukvmd 61d5f3189b Update cancellation status on receiving webhook from Stripe 2024-03-20 14:05:09 +05:30
Abhinav 6b749294ce fix sepa issue 2024-03-19 18:40:32 +05:30
Neeraj Gupta 449284a6a1
[server] API to fetch ML embeddings for given fileIDs (#1144)
## Description
- Also started storing the embedding size and version in the table. This
will come handy while comparing overall size when different compression
and serialization formats are used..
- Also, it can be used to smartly auto-download the embeddings or inform
the user about approximate size when they decide to enable indexing or
search on a particular client.

## Tests

 Verified that embedding fetch and store is working fine.
 Verified that embeddings/diff API is working fine.
2024-03-19 12:23:24 +05:30
Neeraj Gupta 685e1b82c9 [server] Fix query for diff API 2024-03-19 12:09:39 +05:30
Neeraj Gupta 2b3494e61c [server] Return embedding version in API response 2024-03-19 11:25:00 +05:30
Neeraj Gupta e8756a8cf7 [server] Remove unused code 2024-03-19 11:06:36 +05:30
Neeraj Gupta 3df49beb07 [server] Remove unused model name 2024-03-19 11:04:25 +05:30
Neeraj Gupta 05b0f1649f [server] store embedding objSize & version 2024-03-19 11:02:22 +05:30
Neeraj Gupta 7634b2c153 [server] DB migration scripts for emedding table changes 2024-03-19 10:56:25 +05:30
Manav Rathi 303b12c709
Fix the dependency on nginx 2024-03-19 09:31:51 +05:30
Neeraj Gupta f0236acf8f [server] Minor bug fixes in embedding/files API 2024-03-19 06:15:55 +05:30
Manav Rathi 19724367df
Support nginx configuration reloads 2024-03-18 21:47:53 +05:30
Manav Rathi fb0e2d2604
Don't restart museum on restarting nginx 2024-03-18 15:54:25 +05:30
Manav Rathi c177efe79c
Add documentation comments 2024-03-18 15:44:00 +05:30
Manav Rathi 27b13765d7
Fix the environment var 2024-03-18 15:23:23 +05:30
Manav Rathi f494832bd7
Limit max connections at the nginx level 2024-03-18 15:21:47 +05:30
Neeraj Gupta c564e50d59 [server] change method type to post 2024-03-18 14:38:11 +05:30
Manav Rathi 53cf5ffc45
Fix some paths in README 2024-03-18 14:35:42 +05:30
Manav Rathi a341f81932
[museum] Support running behind Nginx (#1130)
- Move deployment related stuff into a separate folder
- Document the overall approach better
- Add an Nginx specific environment and service definition
- Remove the restart limiter
2024-03-18 13:56:01 +05:30
Neeraj Gupta b9ec72bbb4 Merge branch 'main' into file_embedding 2024-03-18 13:12:14 +05:30
Manav Rathi 7ded8ad4fe
Fix key name 2024-03-18 13:07:36 +05:30
Manav Rathi 30cf709eb2
Wrap 2024-03-18 12:39:16 +05:30
Manav Rathi d8347c3976
Remove extra environment 2024-03-18 12:36:29 +05:30
Manav Rathi 5ee878c30e
Require nginx before starting museum.nginx 2024-03-18 12:33:27 +05:30
Manav Rathi f695c5e0d3
Remove extra newline 2024-03-18 11:06:54 +05:30
Manav Rathi 303b26b95f
Tweak the nginx setup 2024-03-18 11:04:31 +05:30
Manav Rathi 9ba9b8a984
Add initial nginx conf 2024-03-18 10:39:39 +05:30
Manav Rathi d893efe5d1
Remove the restart limiter
During recent DDoS attacks, these restart limits had to be manually
disabled. For now, disabling them in our reference copy too, will bring them
back if they are needed later.
2024-03-18 10:27:42 +05:30
Manav Rathi a0599f7b33
Outline the nginx approach 2024-03-18 10:25:11 +05:30
Manav Rathi 9beedcf4e0
Move into separate folder 2024-03-18 10:16:55 +05:30
Neeraj Gupta 47747f1d56 Update config 2024-03-18 10:12:02 +05:30
Neeraj Gupta 7de60f1ced [server] Reduce discord message frequency 2024-03-18 10:02:13 +05:30
Neeraj Gupta 4acc8395ba [server] Prefix discord msg with hostname 2024-03-18 09:59:35 +05:30
Neeraj Gupta 12b9ac4db6
[server] Rate limit (#1121)
## Description

## Tests
2024-03-17 09:59:59 +05:30
Neeraj Gupta e927064476 [server] Add endpoint to get embedding for givenFilesIds 2024-03-17 09:37:02 +05:30
Neeraj Gupta 36982c5332 [server] Add util to findMissingElements while comparing two arrays 2024-03-16 02:30:25 +05:30
Manav Rathi 92ae1be40a
Add a new flag to support S3 path style URLs without disabling SSL
The use case for this came up in Discord - someone is running a custom self
hosted instance where they're using MinIO as their "production S3", but it is
not local (so they don't want the clients to connect without SSL). So we need a
way to get MinIO to work, which needs path style URLs, without also disabling
the SSL.
2024-03-15 17:18:48 +05:30
Manav Rathi 87d39ae2dd
Disable the example.org domains in the default self-hosted configuration
People won't be able to get emails, but they can still create accounts on that
domain in the default setup. So just comment it out as a saner default.
2024-03-15 17:09:40 +05:30
Manav Rathi 966cc697e7
[server] Move the server related gitattributes within that folder
I mistakenly assumed that the gitattributes file can only be at the top
level. However, this doesn't seem to be the case - I noticed this in a recent PR
- https://github.com/ente-io/ente/pull/1104/files.

And it is indeed documented:

> When deciding what attributes are assigned to a path, Git .... .gitattributes
> file in the same directory as the path in question. ..
>
> - https://git-scm.com/docs/gitattributes

Ref: https://stackoverflow.com/questions/71784062/can-we-have-multiple-gitattributes-files-in-different-repo-subfolders
2024-03-14 16:35:36 +05:30
Manav Rathi 2905315e00
[server] Improve the architecture diagram
Add a dotted arrow between museum and the hot object storage to indicate that
clients talk directly to the primary S3 storage.

This has confused some users who were trying to setup self-hosting, them not
realizing that the minio buckets need to be accessible outside the container.

The actual flow is - for both GET and PUT, museum conjures up pre-signed URLs by
talking to the S3 bucket, and vends these pre-signed URLs back to the
client. Thereafter though (unless routed via a Cloudflare worker, which is not
the case when self-hosting), the clients directly connect to this presigned URL.

Maybe too much information to convey with a single dotted arrow, but at least
it'll give a hint that there is a direct connection there somewhere.
2024-03-14 16:23:53 +05:30
Manav Rathi c71d06d405
Mention the self-hosting docs in the server README 2024-03-14 11:12:27 +05:30
vishnukvmd 77276d8d6c [server] Add admin API to change email address 2024-03-13 17:31:38 +05:30
Neeraj Gupta c0a2347b80 [server] Refactor /user/plans 2024-03-12 12:17:04 +05:30
Neeraj Gupta e843ea6669 [server] Change param type from uuid to string 2024-03-11 22:45:53 +05:30
Neeraj Gupta bb68b22adb [server] mark queue as deprecated 2024-03-11 15:41:18 +05:30
Neeraj Gupta 5accf4c6a8 [server] Remove dead code related to collecton trashV2 2024-03-11 15:41:18 +05:30
Neeraj Gupta 28335700e3 [server] Remove deprecated API 2024-03-11 15:41:18 +05:30
Neeraj Gupta 9711e0e29e Improve log 2024-03-11 12:34:03 +05:30
Neeraj Gupta d7292dc670 [server] Release locks before initiating any other controller 2024-03-11 12:34:03 +05:30
Neeraj Gupta 6930aaf220 [server]Update freq for metadatacron 2024-03-11 12:34:03 +05:30
Neeraj Gupta 12903a3748 [server] Clean up collecitonDiff logs 2024-03-11 12:34:03 +05:30
Neeraj Gupta 4af3030c81 [server] Speed up RemoveComplianceHolds 2024-03-11 11:43:02 +05:30
Neeraj Gupta c32f0a28f1 [server] Release previous locks taken by host on statup 2024-03-11 11:43:02 +05:30
Neeraj Gupta 89d761a096 Reduce lock duration 2024-03-11 10:39:46 +05:30
Neeraj Gupta e667eef951 Tweak speed for crons 2024-03-11 10:39:46 +05:30
Neeraj Gupta 6e160dca43 Fix minor bugs 2024-03-08 15:15:00 +05:30
Neeraj Gupta 1f7d9dbb86 Rename passKey to passkey 2024-03-08 15:15:00 +05:30
Neeraj Gupta a780598607 Fix sql query 2024-03-08 15:15:00 +05:30
Neeraj Gupta 7f66714d96 Refactor + bug fixes 2024-03-08 15:15:00 +05:30
Neeraj Gupta 980ab6c49c Refactor: extend totp recovery API to recover passkey 2024-03-08 15:15:00 +05:30
Neeraj Gupta fe181fecbe Rename 2024-03-08 15:15:00 +05:30
Neeraj Gupta 50c3a7a8e5 Store resetSecret in encrypted form 2024-03-08 15:15:00 +05:30
Neeraj Gupta f766484b2e Rename account_recovery -> two_factor_recovery 2024-03-08 15:15:00 +05:30
Neeraj Gupta 42e4364fda Add APIs to allow user to skip passkey based two-fa 2024-03-08 15:15:00 +05:30
Neeraj Gupta 09a7d557d2 Add API to get account two recovery status 2024-03-08 15:15:00 +05:30
Neeraj Gupta 13bae268ec Add models passkey reset via recoveryKey 2024-03-08 15:15:00 +05:30
Neeraj Gupta 23fcce245d Add table for account recovery 2024-03-08 15:15:00 +05:30
Neeraj Gupta 8186246690
[server] Update freq for removing compliance hold (#731)
## Description
- Taking lock at host level to avoid redundant downstream request when
multiple host pick same items.

## Tests
Tested locally.
2024-03-07 13:08:04 +05:30
Manav Rathi 795c2baf0f Fix lint 2024-03-05 16:53:15 +05:30
Manav Rathi 3cad79dfa9 Add museum release action 2024-03-05 16:39:11 +05:30
Manav Rathi 0b37f74636 Add museum lint action 2024-03-05 16:33:20 +05:30
Neeraj Gupta 34743171f9
[server] Add API to re-queue item for processing (#679)
## Description

## Tests
2024-03-05 15:15:23 +05:30
Neeraj Gupta 3513d7a297
[server] Remove entry from object_keys before marking del as completed (#677)
## Description

## Tests
2024-03-05 14:56:02 +05:30
Manav Rathi 44eb8353a2 Update server/ to account for new monorepo structure 2024-03-02 18:04:59 +05:30
Neeraj Gupta f796a7f63a
[server] Remove custom metrics for collection diff (#520) 2024-03-02 17:14:35 +05:30
Julien bb32d7684a
docs: Fixed url to RUNNING.md in README 2024-03-02 10:39:26 +01:00
Neeraj Gupta 531bb344fe Import museum 2024-03-01 13:37:01 +05:30