beenull/ente
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
2844 commits 18 branches 104 tags 490 MiB
Commit graph

946 commits

Author SHA1 Message Date
Shailesh Pandit f7a05fcb39 Use pqueue addAll instead of Promise.all 
Add timeout to downloads during ml sync
 2022-01-14 19:45:39 +05:30
Shailesh Pandit c454c50341 Select nearest pass2 detection in blazeface 
Remove duplicates after 2nd pass
Updated min face detection probability based on results
 2022-01-14 12:28:11 +05:30
Shailesh Pandit 5f6e617f68 Show upto 200 people in search box 
Make people list in search box scrollable
 2022-01-13 09:28:01 +05:30
Shailesh Pandit c14f8cc90f Display face of person only based on detection probability 
Fix comparator of findFirstIfSorted
 2022-01-12 14:15:22 +05:30
Shailesh Pandit fbe6e5f8cf Check if old faces are in same order before using 
Keep old ml version if present
 2022-01-12 12:05:31 +05:30
Shailesh Pandit ef6c564d62 Only set fields related to each ml sync step 
Reset error fields on successful sync of file
 2022-01-12 10:53:06 +05:30
Shailesh Pandit 7ac252aca8 Handle some non file related errors in ml sync 
Handles network error and session expired errors
 2022-01-11 19:08:26 +05:30
Shailesh Pandit bd7be81acf Fix build 2022-01-11 07:08:42 +05:30
Shailesh Pandit b90cd62fa8 Store configs in db 2022-01-10 20:20:06 +05:30
Shailesh Pandit 52078b3e5e Ad Dbscan clustering method 2022-01-10 17:38:51 +05:30
Shailesh Pandit cc7d2355f7 Update hdbscan to use cluser selection method and epsilon 2022-01-10 14:56:00 +05:30
Shailesh Pandit bb000c0c69 ML Debug refresh for all identified people 
Clear people index button
 2022-01-09 18:32:25 +05:30
Shailesh Pandit 8eebdb226e Fix: clear old person id in face before reclustering 2022-01-09 17:41:52 +05:30
Shailesh Pandit 90bf35b05d Make minSamples of hdbscan configurable 2022-01-09 10:06:38 +05:30
Shailesh Pandit 2dd48e6ceb MLWorkManager refactor 
Extract generic SimpleJob from ML Sync Job
Terminate sync job and live sync workers after work done
 2022-01-08 23:27:29 +05:30
Shailesh Pandit c995729ab9 Use face crop cache to store person display face 2022-01-07 19:20:56 +05:30
Shailesh Pandit 0fbb65cb9a Remove one time face interface migration 2022-01-07 09:07:20 +05:30
Shailesh Pandit 3d81ffd84b Export and Import ml data from ml debug 2022-01-07 09:06:18 +05:30
Shailesh Pandit 943fa990cb Migration for new Face interface 2022-01-06 15:05:00 +05:30
Shailesh Pandit 945139b758 Embedding service interface refactor 
Use face imageBitmaps as input
Use Float32Array as output and for storage in db
 2022-01-06 12:15:26 +05:30
Shailesh Pandit 2b5aafa0bc Rename blazeface and mobilefacenet services 2022-01-06 07:22:05 +05:30
Shailesh Pandit 19c3654926 Rename tfjs face detection and embedding service files 2022-01-06 07:20:29 +05:30
Shailesh Pandit 19aae1a75b Refactor interfaces 
Use composition to store face analysis results
Add face id to stored face
 2022-01-06 06:49:08 +05:30
Shailesh Pandit b88ccee6ed Update files version after syncLocalFiles 2022-01-05 14:39:55 +05:30
Shailesh Pandit be476995b2 Remove one time migration for face crops 2022-01-05 14:15:49 +05:30
Shailesh Pandit 2f3d06cb4d Migrate face crops from indexdb to cache 2022-01-05 13:54:21 +05:30
Shailesh Pandit 2471a57b3b Store face crops in cache to reduce io 
when updating indexeddb
 2022-01-05 12:12:24 +05:30
Shailesh Pandit 68a976f628 Revert one time migration 2022-01-04 13:26:22 +05:30
Shailesh Pandit 86b2efbdd6 Fix migration for one time run 2022-01-04 13:07:34 +05:30
Shailesh Pandit 4cb552046a Use IndexedDB without forage to use indexes 2022-01-04 11:26:16 +05:30
Shailesh Pandit fa7453323d Remove unnecessary sorting while getting out of sync files 
Add timing logs to getPeopleList and getOutOfSyncFiles
 2022-01-03 15:02:48 +05:30
Shailesh Pandit fb402e9b69 Sync deleted files, remove ml data for them 
Reset sync job backoff on local file updates
 2022-01-03 10:06:30 +05:30
Shailesh Pandit 2cb9ebf402 Exponential back off for ml sync job 2022-01-03 08:05:00 +05:30
Shailesh Pandit b9d32cc1e6 Dispose sync context in local file sync flow 2022-01-03 07:13:59 +05:30
Shailesh Pandit fc459171f4 Fix typo 2022-01-02 22:33:40 +05:30
Shailesh Pandit 502a023fa3 Run ml sync concurrently 2022-01-02 22:27:42 +05:30
Shailesh Pandit 86052cc393 Run face clustering using configured method 
Save face clustering results
Display noise faces on ml debug page
 2021-12-31 12:08:50 +05:30
Shailesh Pandit 385acec1ab Do 2nd pass of blazeface on close ups 
for better accuracy
Transform utils
 2021-12-28 20:30:52 +05:30
Shailesh Pandit 8177cd336b Show unidentified faces in info panel 
Add local file debugging info to ml debug page
showing aligned face box and landmarks and face crops
 2021-12-25 13:07:24 +05:30
Shailesh Pandit 91bd053cf5 Use face crops while calculating embeddings 
Simulate mlVersion update requiring only embeddings to be recalculated
without downloading full image
 2021-12-24 12:27:29 +05:30
Shailesh Pandit 7b06f5d8f8 imageSource not updated based on actual source 2021-12-23 23:59:49 +05:30
Shailesh Pandit 44f4362f0f Store ml sync error count for a file, limit retry 
Download original quality files for photos and live photos only
 2021-12-23 23:40:21 +05:30
Shailesh Pandit b4c31c5845 Store face crops, extract aligned faces from face crops 
Align faces using center, size and rotation only, using this aligned faces can be extracted without whole image
 2021-12-23 18:27:53 +05:30
Shailesh Pandit 4ebcddbb84 Provision to run ml sync on original images 
To capture high resolution face images
Use ImageBitmap till tf models as tf manipulations on variable dimention images leads to gpu memory leak - https://github.com/tensorflow/tfjs/issues/604
 2021-12-20 22:20:20 +05:30
Shailesh Pandit 0e366264a0 TF memory cleaned only with tf.dispose and tf.tidy 
Remove usage of startScope and endScope, can be used for debugging
 2021-12-15 17:20:07 +05:30
Shailesh Pandit f5ca124fc9 Only sync updated ml methods for existing files 2021-12-14 19:44:13 +05:30
Shailesh Pandit 2d17dd30e9 For uploaded files use local files for ML Sync 
Extract out ML work manager and sync job from worker
Start and stop ML Sync job on app start, login and logout
 2021-12-12 23:12:18 +05:30
Shailesh Pandit 470f9fb931 Move machine learning types to types folder 2021-12-08 12:43:51 +05:30
Shailesh Pandit 4d89e7df96 Revert comented ffmpeg code 
As not required after common upload types are extracted out
 2021-12-08 12:22:12 +05:30
Shailesh Pandit d65de21be5 Merge branch 'upload-types-refactor' into machine-learning 2021-12-08 11:59:07 +05:30