Commit graph

2867 commits

Author SHA1 Message Date
Abhinav 6b6a75df22 disable edits for shared collections 2022-01-21 18:45:08 +05:30
Abhinav 47ca9300dc handle null url 2022-01-21 18:33:11 +05:30
Abhinav c917f8144e reposition report button 2022-01-21 18:31:06 +05:30
Abhinav a358525b36 handle sharing errors 2022-01-21 18:24:57 +05:30
Abhinav 13f8968ee9 handle token exipring 2022-01-21 17:33:20 +05:30
Abhinav 6a3ed25c21 Merge branch 'master' into shareable-url 2022-01-21 17:22:33 +05:30
Abhinav Kumar aa45ddc3a9
Merge pull request #316 from ente-io/404-page
add 404 page
2022-01-21 17:16:49 +05:30
Abhinav 9fa5662880 add 404 page 2022-01-21 16:48:25 +05:30
Abhinav f807b96df8 set collection after fetching 2022-01-21 15:13:35 +05:30
Abhinav a96d115aa2 chnage URL format 2022-01-21 14:56:54 +05:30
Abhinav f4debffff8 build fix 2022-01-21 14:10:17 +05:30
Abhinav f4acef55ef mode copy code to codeblock 2022-01-21 14:05:11 +05:30
Abhinav 005142fe68 fix arrow style 2022-01-21 14:03:57 +05:30
Abhinav 8498ae54c0 update codeblock and move to new file 2022-01-21 13:59:46 +05:30
Abhinav 3ca08bd4f6 add copy to clipboard option 2022-01-21 12:33:23 +05:30
Abhinav 0284830085 refactor tooltip styling 2022-01-21 12:33:23 +05:30
Abhinav a99238b25b update api urls 2022-01-21 12:33:23 +05:30
Abhinav 5eb36c1e15 restyle url sharing 2022-01-21 12:33:10 +05:30
Shailesh Pandit bce97ce579 Reduce clusterSelectionEpsilon 2022-01-20 16:47:32 +05:30
Abhinav dfbea0b6ab rename publicAccessURL to publicURL 2022-01-20 15:50:52 +05:30
Abhinav ffe5062d7a fix video download for public collection 2022-01-20 15:49:40 +05:30
Abhinav 8d81368a34 change Url to URL 2022-01-20 12:13:10 +05:30
Abhinav 1a74eccdd7 change Url to URL 2022-01-20 12:12:27 +05:30
Abhinav 1796fae8a9 add cache control to prevent cache 2022-01-19 21:51:22 +05:30
Abhinav 6303f2d606 fix send report 2022-01-19 17:14:46 +05:30
Abhinav dea7e186b4 again set url ref 2022-01-19 17:04:49 +05:30
Abhinav 072e471f27 update to hadle new URL format 2022-01-19 16:24:43 +05:30
Abhinav 13ebc48f9c add deleteSharableURL api and UI 2022-01-19 16:23:59 +05:30
Abhinav 2d23b4a70c add logic to prevent opera and MSStream undefined errors 2022-01-19 12:55:48 +05:30
Abhinav a648695e44 add go to ente button 2022-01-19 12:55:15 +05:30
Abhinav 894802bc81 add device detection logic 2022-01-19 10:31:36 +05:30
Abhinav f2caf2758b add abuse report logic 2022-01-18 18:38:24 +05:30
Shailesh Pandit ac57f25f76 Fix build 2022-01-18 11:41:27 +05:30
Shailesh Pandit 94a7ec4796 Use new local sync context every batchSize sync
Maintain local sync context in ml service
Sync context to only store counts of files and faces synced
2022-01-18 08:07:18 +05:30
Abhinav 1ce3a9dc6a udpate getPublicCollectionLastSyncTime 2022-01-17 18:41:26 +05:30
Abhinav ad7f24e8df disable report abuse form 2022-01-17 18:16:33 +05:30
Abhinav 292c3c4a1f add report abuse buttona and form 2022-01-17 18:12:53 +05:30
Abhinav 2b5cea72a5 rename sharedCollection to publicCollection 2022-01-17 16:48:13 +05:30
Abhinav a53ebd5ce4 add local caching to sharedURLAlbum gallery
collection info api and ui element
open in ente button
2022-01-17 16:48:13 +05:30
Abhinav 8d81fc0342 add OpenInEnte component 2022-01-17 16:48:13 +05:30
Abhinav 493b85b322 add todo for shared-collection info api 2022-01-17 16:48:13 +05:30
Abhinav ecc2ce7093 add logic and apis for viewing sharedAlbum Thumbnails and files 2022-01-17 16:48:13 +05:30
Abhinav d7832a2e08 encode collectionKey as its not URL 2022-01-17 16:48:13 +05:30
Abhinav 80f3c1bdda small bug fixes 2022-01-17 16:48:13 +05:30
Abhinav 429211f9a8 add create sharable url logic
ui for opening shared collections
2022-01-17 16:48:09 +05:30
Shailesh Pandit eed5497da8 More logging for queue stats and worker
Temporarily add timeout to livesync queue
2022-01-16 20:30:20 +05:30
Abhinav ef1dd9a453 return currentTrash if getTrash fails 2022-01-16 16:22:57 +05:30
Shailesh Pandit 4999f8b4e0 Extract MAX_FACE_DISTANCE_PERCENT as constant 2022-01-16 13:08:49 +05:30
Shailesh Pandit 43c792e46d Fix local env needing refresh to login
by loading tfjs-tflite as dynamic import
2022-01-16 13:01:29 +05:30
Shailesh Pandit 18db6f44dc Fix SimpleJob to set correct state after stopped 2022-01-16 11:47:04 +05:30
Abhinav c9bedcdd6e return empty array if updateTrash fails 2022-01-15 13:16:27 +05:30
Abhinav 05f19c3303 handle plan modal opening error better 2022-01-15 11:01:09 +05:30
Shailesh Pandit d1bab5ec84 Fix build
Revisit todos
2022-01-15 09:39:27 +05:30
Shailesh Pandit 4845c1d605 Convert not compatible local images before sync
Only sync image types locally
2022-01-15 09:05:56 +05:30
Shailesh Pandit f7a05fcb39 Use pqueue addAll instead of Promise.all
Add timeout to downloads during ml sync
2022-01-14 19:45:39 +05:30
Shailesh Pandit f7f40bf5ec Include actual store in increment version tx 2022-01-14 19:10:42 +05:30
Abhinav 01db5a28ce change user-plans api url 2022-01-14 18:57:44 +05:30
Abhinav 8ca2eeb8c7 remove unnecessary api key 2022-01-14 17:43:53 +05:30
Abhinav 22528bb7ca dont animate currentSubscribedPlan 2022-01-14 17:43:40 +05:30
Abhinav 1fe2a97f8c call user-plans api if token present 2022-01-14 17:41:33 +05:30
Shailesh Pandit ec0108dda6 Start and stop sync job from debug page
using ml work manager
2022-01-14 15:51:37 +05:30
Shailesh Pandit c454c50341 Select nearest pass2 detection in blazeface
Remove duplicates after 2nd pass
Updated min face detection probability based on results
2022-01-14 12:28:11 +05:30
Abhinav 13c362f16c fixes unable to select consective item range select 2022-01-13 14:18:50 +05:30
Abhinav 6516246499 fixes https://sentry.ente.io/organizations/ente/issues/2338/
adds rangeStart undefined check
2022-01-13 14:18:00 +05:30
Shailesh Pandit f5e45a8b73 Show filtered faces based on detection probability
Some styling changes of debug page
2022-01-13 13:39:13 +05:30
Shailesh Pandit 255219f778 Fix idb upgrades
Use oldVersion to apply all required upgrades
2022-01-13 12:03:11 +05:30
Abhinav Kumar 44b2907c69
Merge pull request #295 from ente-io/fix-search-count
fix search count by updating it when filteredData changes
2022-01-13 11:14:58 +05:30
Abhinav Kumar 8aa55eb411
Merge pull request #291 from ente-io/first-upload-name
use suggested name if available during first upload
2022-01-13 11:14:45 +05:30
Abhinav 41a23dbc0b fix search count by updating it when filteredData changes 2022-01-13 10:39:14 +05:30
Shailesh Pandit 5f6e617f68 Show upto 200 people in search box
Make people list in search box scrollable
2022-01-13 09:28:01 +05:30
Shailesh Pandit 09c9deda5e Disabling large mst binary tree display on ml debug
Fix wrong assumptions in faceId comment
2022-01-13 07:16:54 +05:30
Shailesh Pandit 4cd781d201 Update hdbscan to remove mst tree from storage 2022-01-12 20:35:04 +05:30
Abhinav 000f1546fe fix getFileOriginalName if name doesn't have edited suffix 2022-01-12 17:00:58 +05:30
Abhinav f37cac0a39 fix casing of ParsedMetadataJSON 2022-01-12 16:59:27 +05:30
Abhinav bed6a086a6 use suggested if available during first upload 2022-01-12 14:42:46 +05:30
Shailesh Pandit c14f8cc90f Display face of person only based on detection probability
Fix comparator of findFirstIfSorted
2022-01-12 14:15:22 +05:30
Shailesh Pandit fbe6e5f8cf Check if old faces are in same order before using
Keep old ml version if present
2022-01-12 12:05:31 +05:30
Shailesh Pandit ef6c564d62 Only set fields related to each ml sync step
Reset error fields on successful sync of file
2022-01-12 10:53:06 +05:30
Shailesh Pandit 7ac252aca8 Handle some non file related errors in ml sync
Handles network error and session expired errors
2022-01-11 19:08:26 +05:30
Shailesh Pandit 864885d586 Update default face clustering parameters 2022-01-11 09:53:19 +05:30
Shailesh Pandit 0c1db302c1 Extract ConfigEditor as component 2022-01-11 09:42:31 +05:30
Shailesh Pandit 2ceca69514 Move ml react components to a folder 2022-01-11 08:44:33 +05:30
Shailesh Pandit bd7be81acf Fix build 2022-01-11 07:08:42 +05:30
Shailesh Pandit aa6b40b724 Make ml sync configs editable on debug page 2022-01-10 22:28:09 +05:30
Shailesh Pandit b90cd62fa8 Store configs in db 2022-01-10 20:20:06 +05:30
Shailesh Pandit 52078b3e5e Ad Dbscan clustering method 2022-01-10 17:38:51 +05:30
Shailesh Pandit cc7d2355f7 Update hdbscan to use cluser selection method and epsilon 2022-01-10 14:56:00 +05:30
Shailesh Pandit 4237b9cfc7 Display clusters and mst in treeview on ml debug
from last clustering run
2022-01-09 22:40:55 +05:30
Shailesh Pandit bb000c0c69 ML Debug refresh for all identified people
Clear people index button
2022-01-09 18:32:25 +05:30
abhinavkgrd 7dc9246519
Merge pull request #284 from ente-io/rename-type-MetadataObject-to-Metadata
Rename type MetadataObject to Metadata
2022-01-09 18:05:13 +05:30
abhinavkgrd 71a97caa03
Merge pull request #281 from ente-io/reuse-file-readers
reuse file readers
2022-01-09 18:00:51 +05:30
abhinavkgrd b504e072f3
Merge pull request #288 from ente-io/fix-get-original-filename-logic
fix getFileOriginalName logic
2022-01-09 18:00:39 +05:30
Shailesh Pandit 8eebdb226e Fix: clear old person id in face before reclustering 2022-01-09 17:41:52 +05:30
Abhinav 4cdc93e345 renamed sentry constants to follow the global convention 2022-01-09 13:55:07 +05:30
Abhinav 48eac037ea fix getFileOriginalName logic 2022-01-09 12:25:40 +05:30
Abhinav d42af9cd1a pass reader correctly instead of passing workers 2022-01-09 11:05:50 +05:30
Shailesh Pandit 90bf35b05d Make minSamples of hdbscan configurable 2022-01-09 10:06:38 +05:30
Shailesh Pandit 0d9d79e979 Display all peope on ml debug 2022-01-09 10:05:13 +05:30
Shailesh Pandit aac2318513 Moving default ml sync configs to constants
Limit sync job max interval to around 15 mins
2022-01-09 07:43:21 +05:30
Shailesh Pandit 2dd48e6ceb MLWorkManager refactor
Extract generic SimpleJob from ML Sync Job
Terminate sync job and live sync workers after work done
2022-01-08 23:27:29 +05:30
Vishnu Mohandas 5d3e284fcd
Merge pull request #280 from ente-io/disable-sentry-release-cli-for-local-build
disable sentry release cli for local build
2022-01-08 18:44:27 +05:30
abhinavkgrd 9d727f23b6
Merge pull request #287 from ente-io/handle-exifr-error
handle exifr parsing failure gracefully
2022-01-08 15:08:05 +05:30
Abhinav 6abf883b3e Merge branch 'reuse-file-readers' into rename-type-MetadataObject-to-Metadata 2022-01-08 13:56:22 +05:30
Abhinav d4eb485829 handle exifr parsing fail 2022-01-08 13:44:51 +05:30
Abhinav 0e7ab5c886 fix if fileReader not present check 2022-01-08 13:34:50 +05:30
Abhinav 585d3acf88 fix multiple folder detection for sibling folder with no common ancestor 2022-01-08 13:09:43 +05:30
Abhinav f788490573 fixc worker passed instead of FileReader to getMimeTypeFromBlob 2022-01-08 12:29:10 +05:30
Abhinav 15a03a87b3 rename type MetadataObject to Metadata 2022-01-08 10:34:40 +05:30
Shailesh Pandit c995729ab9 Use face crop cache to store person display face 2022-01-07 19:20:56 +05:30
Abhinav 8fc784a1f5 replace one missed worker.getUint8ArrayView function 2022-01-07 18:34:42 +05:30
Abhinav bd3d6d84ce use global file reader and reuse them wherever possible 2022-01-07 18:28:22 +05:30
Abhinav 5427fbf232 move isSentryEnabled needed by both config util and constants/sentry to seperate file sentryConfigUtil
as neither of the two files can import each other
2022-01-07 18:06:22 +05:30
Abhinav fbf6476d70 move sentry constants to constants/sentry 2022-01-07 17:52:37 +05:30
Shailesh Pandit 03a3a57055 Use only path of cached face crop in export 2022-01-07 13:56:45 +05:30
Shailesh Pandit 1b0d6e8da4 Catch error while showing blob image to not crash app 2022-01-07 09:18:39 +05:30
Shailesh Pandit 0fbb65cb9a Remove one time face interface migration 2022-01-07 09:07:20 +05:30
Shailesh Pandit 3d81ffd84b Export and Import ml data from ml debug 2022-01-07 09:06:18 +05:30
Abhinav 028bbdd01d Merge branch 'master' into fix-queue-processing-order 2022-01-06 19:25:47 +05:30
abhinavkgrd 6d219275c0
Merge pull request #263 from ente-io/upload-types-refactor
Extract types and constants from services and components
2022-01-06 18:36:23 +05:30
Shailesh Pandit 943fa990cb Migration for new Face interface 2022-01-06 15:05:00 +05:30
abhinavkgrd b297e9fd64
Merge pull request #278 from ente-io/fix-double-counting-selected-items
Fix overlapping range select behaviour
2022-01-06 14:30:46 +05:30
Abhinav 73882919d1 handle case where range select with few selected and few unselected items
select all the items in the union of two ranges
2022-01-06 14:12:41 +05:30
Abhinav 6d76611079 fix counting again 🥲 2022-01-06 13:53:29 +05:30
Abhinav 314f8d44b1 handle range deselection 2022-01-06 13:35:27 +05:30
Abhinav a24ace5946 deselect items if overlap between range selected value 2022-01-06 13:07:52 +05:30
Abhinav e41b3c184f make onSelect required prop 2022-01-06 13:07:18 +05:30
Shailesh Pandit 945139b758 Embedding service interface refactor
Use face imageBitmaps as input
Use Float32Array as output and for storage in db
2022-01-06 12:15:26 +05:30
Abhinav 2044f4f6cf remove unneccsary console log 2022-01-06 12:15:13 +05:30
Abhinav bf6a0ef7cc fix double counting selected items if new and previous range select overlap 2022-01-06 12:14:18 +05:30
Abhinav 14b1e59c90 abstract SmallLoadingSpinner 2022-01-06 10:21:08 +05:30
Abhinav 3c4a0f7b6a add loading spinner during creation udpate api call 2022-01-06 10:11:33 +05:30
Shailesh Pandit 2b5aafa0bc Rename blazeface and mobilefacenet services 2022-01-06 07:22:05 +05:30
Shailesh Pandit 19c3654926 Rename tfjs face detection and embedding service files 2022-01-06 07:20:29 +05:30
Shailesh Pandit 19aae1a75b Refactor interfaces
Use composition to store face analysis results
Add face id to stored face
2022-01-06 06:49:08 +05:30
Abhinav b752b77c26 Merge branch 'master' into upload-types-refactor 2022-01-05 16:36:39 +05:30
Shailesh Pandit 1a41c7c194 Fix getUnidentifiedFaces for empty face crops 2022-01-05 14:43:25 +05:30
Shailesh Pandit b88ccee6ed Update files version after syncLocalFiles 2022-01-05 14:39:55 +05:30
Shailesh Pandit be476995b2 Remove one time migration for face crops 2022-01-05 14:15:49 +05:30
Shailesh Pandit 2f3d06cb4d Migrate face crops from indexdb to cache 2022-01-05 13:54:21 +05:30
Abhinav 9e099d4e67 moved queueProcessor to service directory 2022-01-05 13:28:24 +05:30
Abhinav e4f649c467 refactor upload service constants and types 2022-01-05 13:27:10 +05:30
Abhinav 2faade0d42 move KEK interface to types 2022-01-05 13:05:18 +05:30
Abhinav 344d2b2c0c move ProgressUpdater interface to UIService 2022-01-05 13:00:29 +05:30
Abhinav 7285af6977 keep NULL_LOCATION along with Location type defination 2022-01-05 12:50:10 +05:30
Abhinav 44bc3916a5 move isDataStream type check util function to types/upload 2022-01-05 12:46:57 +05:30
Abhinav caef21f939 merge types/user imports 2022-01-05 12:41:16 +05:30
Abhinav 63960c207b fix User import 2022-01-05 12:40:36 +05:30
Abhinav a5849600b1 move errorUtil from utils/common to utils/error 2022-01-05 12:25:48 +05:30
Abhinav b00d7ea2b0 move file enums to constants/file 2022-01-05 12:18:46 +05:30
Abhinav 9e0fef4743 move export enums to constants/export 2022-01-05 12:14:22 +05:30
Shailesh Pandit 2471a57b3b Store face crops in cache to reduce io
when updating indexeddb
2022-01-05 12:12:24 +05:30
Abhinav ce22425be9 move collection enum to constants/collection 2022-01-05 12:11:32 +05:30
Abhinav 11e990ffe9 move file constants to constants/file 2022-01-04 17:27:19 +05:30
Abhinav 8a96b73b37 moved export constants to constants/export 2022-01-04 17:19:14 +05:30
Abhinav 88ea78ebb3 dont export constants used only in one file 2022-01-04 17:14:34 +05:30
Abhinav 8f0a0d3f18 fix FIX_CREATION_TIME_VISIBLE_TO_USER_IDS import 2022-01-04 17:12:43 +05:30
Abhinav 9b48275144 moved upload constants to constants/upload 2022-01-04 17:12:29 +05:30
Abhinav eddfbed37c moved user constants to constants/user 2022-01-04 16:57:57 +05:30
Abhinav 5a84003388 move collection constants to constants/collection 2022-01-04 16:54:39 +05:30
Abhinav dfdf2897d4 move constants from util to src directory and split into sub folders 2022-01-04 16:50:54 +05:30
Abhinav b10a43b3a8 move gallery related types to types/gallery 2022-01-04 16:27:59 +05:30
Abhinav f30a5103bc renamed types to util/constants 2022-01-04 16:19:58 +05:30
Abhinav 41b8ed55a2 moved user related types to types/user 2022-01-04 16:09:42 +05:30
Abhinav 4011b6fbe8 move trash related types to types/trash 2022-01-04 15:51:35 +05:30
Abhinav 79f6ebc454 moved search related types to types/search 2022-01-04 15:48:33 +05:30
Abhinav b5b31a8350 move non exported constant back to origin file 2022-01-04 15:36:16 +05:30
Abhinav aba964922e moved file related types to types/file 2022-01-04 15:20:14 +05:30
Abhinav f8af3a9a39 moved export types to types/export 2022-01-04 14:21:47 +05:30
Abhinav 2aba5cb058 moved collection related types to types/collection 2022-01-04 14:17:38 +05:30
Abhinav 1a1c4d145c created types for billing and renamed billingUtil to billing/index 2022-01-04 14:05:41 +05:30
Shailesh Pandit 8ab7dc38e1 Ignore empty faces in AllFacesMap 2022-01-04 13:54:07 +05:30
Shailesh Pandit 68a976f628 Revert one time migration 2022-01-04 13:26:22 +05:30
Shailesh Pandit 86b2efbdd6 Fix migration for one time run 2022-01-04 13:07:34 +05:30
abhinavkgrd 1d0aa42630
Merge pull request #273 from ente-io/web-security-clean
harden web security
2022-01-04 13:03:09 +05:30
Abhinav 530f40f5a0 Merge branch 'master' into upload-types-refactor 2022-01-04 12:20:28 +05:30
Shailesh Pandit 4cb552046a Use IndexedDB without forage to use indexes 2022-01-04 11:26:16 +05:30
Abhinav 9acb76795e fix sentry tunnel URL 2022-01-03 15:33:08 +05:30
Shailesh Pandit fa7453323d Remove unnecessary sorting while getting out of sync files
Add timing logs to getPeopleList and getOutOfSyncFiles
2022-01-03 15:02:48 +05:30
Shailesh Pandit fb402e9b69 Sync deleted files, remove ml data for them
Reset sync job backoff on local file updates
2022-01-03 10:06:30 +05:30
Shailesh Pandit 2cb9ebf402 Exponential back off for ml sync job 2022-01-03 08:05:00 +05:30
Shailesh Pandit b9d32cc1e6 Dispose sync context in local file sync flow 2022-01-03 07:13:59 +05:30
Shailesh Pandit fc459171f4 Fix typo 2022-01-02 22:33:40 +05:30
Shailesh Pandit 502a023fa3 Run ml sync concurrently 2022-01-02 22:27:42 +05:30
Shailesh Pandit 86052cc393 Run face clustering using configured method
Save face clustering results
Display noise faces on ml debug page
2021-12-31 12:08:50 +05:30
Shailesh Pandit 30f9088c81 Show face crops and images in rows 2021-12-29 18:45:47 +05:30
Shailesh Pandit 778431d01b Normalize landmark and box thickness in ml debug 2021-12-29 17:11:46 +05:30
Shailesh Pandit 9d3bf5ce6f Detected face transforms to and from face crop 2021-12-29 12:10:11 +05:30
Shailesh Pandit 162d760b80 Use transformation-matrix package for transforms 2021-12-29 10:32:01 +05:30
Shailesh Pandit 385acec1ab Do 2nd pass of blazeface on close ups
for better accuracy
Transform utils
2021-12-28 20:30:52 +05:30
Shailesh Pandit 72b2a6ad8b Handle empty faces in unidentified faces 2021-12-25 14:22:38 +05:30
Shailesh Pandit 8177cd336b Show unidentified faces in info panel
Add local file debugging info to ml debug page
showing aligned face box and landmarks and face crops
2021-12-25 13:07:24 +05:30
Shailesh Pandit 91bd053cf5 Use face crops while calculating embeddings
Simulate mlVersion update requiring only embeddings to be recalculated
without downloading full image
2021-12-24 12:27:29 +05:30
Shailesh Pandit 7b06f5d8f8 imageSource not updated based on actual source 2021-12-23 23:59:49 +05:30
Shailesh Pandit 44f4362f0f Store ml sync error count for a file, limit retry
Download original quality files for photos and live photos only
2021-12-23 23:40:21 +05:30
Shailesh Pandit b4c31c5845 Store face crops, extract aligned faces from face crops
Align faces using center, size and rotation only, using this aligned faces can be extracted without whole image
2021-12-23 18:27:53 +05:30
Shailesh Pandit 4ebcddbb84 Provision to run ml sync on original images
To capture high resolution face images
Use ImageBitmap till tf models as tf manipulations on variable dimention images leads to gpu memory leak - https://github.com/tensorflow/tfjs/issues/604
2021-12-20 22:20:20 +05:30
Abhinav 6e62f312bf update to use ente domain url for workes instead of worker.dev cf domains 2021-12-20 15:51:24 +05:30
Abhinav 40058f0883 fix typo 2021-12-20 13:21:12 +05:30
Abhinav bff8411284 adds download collection option 2021-12-20 13:00:56 +05:30
Abhinav e066ee5ff1 added start and finish loading helper function 2021-12-20 12:59:39 +05:30