Shailesh Pandit
d5199acc43
Extract common types in upload service to upload types
...
Required for not including code required for upload service in other parts just requiring common types
e.g. ffmpegService is included in machine learning web worker while including downloadManager service
2021-12-08 11:53:34 +05:30
Shailesh Pandit
28b77bcf0a
Merge branch 'master' into machine-learning
2021-12-07 18:41:29 +05:30
Vishnu Mohandas
79046cc694
Enable batch edit of creation-time for 243
2021-12-03 10:08:54 +05:30
Shailesh Pandit
39386941f5
Sync people index only after new files are synced
...
Maintain files and people latest versions
2021-12-01 19:21:02 +05:30
Shailesh Pandit
498603eb34
Clear tf memory usage after each sync run
2021-12-01 19:00:57 +05:30
Abhinav
a0708ac3f8
log dateTime value on getUNIXTime fail
2021-11-30 11:15:36 +05:30
Abhinav
0e57c8830b
fail safe error handling
2021-11-30 11:14:33 +05:30
Abhinav
9454540175
update name to getCachedOriginalFile
2021-11-29 14:14:28 +05:30
Abhinav
3a0e6b2f98
renamed get preview to getThumbnail
2021-11-29 14:12:24 +05:30
Abhinav
a1672d3c2f
handle saving to cache in getPreview itself
2021-11-29 14:11:41 +05:30
Abhinav
1f3734b577
fix isFileHEIC check
2021-11-29 12:09:21 +05:30
Abhinav
9e3c294775
check in memory cache for thumbnail before hitting browser cache
2021-11-29 12:08:53 +05:30
Abhinav
f997c7457a
use converted suffix instead of forPreview to to avoid duplicate download of unconverted files
2021-11-29 11:18:28 +05:30
Abhinav
f688d620cc
prevent duplicate file download calls
2021-11-29 10:42:46 +05:30
Shailesh Pandit
1d41644ac8
Add people to search, filter photos by person
2021-11-28 21:45:44 +05:30
Shailesh Pandit
af130f803b
Set person face chip to one having max pixels
...
And detection probability score
2021-11-28 11:28:13 +05:30
Shailesh Pandit
e96e1a9ee4
Show people face chips in info panel of image
2021-11-28 11:17:56 +05:30
Shailesh Pandit
52b20c9783
Rename personId to id
2021-11-27 15:40:29 +05:30
Shailesh Pandit
320099df59
Sync people from generated clusters
...
Update personId in faces
2021-11-27 15:31:03 +05:30
Shailesh Pandit
83e21d7aab
Fix build, better naming, disable tsne
2021-11-27 11:34:26 +05:30
Shailesh Pandit
0d72fccd7a
Streamline clustering config and service
...
Fix tsne
MLDebug to have start and stop ml sync
2021-11-27 09:55:39 +05:30
Shailesh Pandit
8d6ec08d08
Run clustering on all faces in db
2021-11-26 20:31:34 +05:30
Shailesh Pandit
b4ad482431
Process new and outdated files in ml sync
2021-11-26 19:59:13 +05:30
Shailesh Pandit
1f0853debd
Run ml-worker batches continuously
...
Persist versioned faces in indexdb
2021-11-25 22:08:28 +05:30
abhinavkgrd
4c30c45724
Merge pull request #249 from ente-io/ffmpeg-load-fail-handling
...
ffmpeg load fail handling
2021-11-25 12:11:53 +05:30
abhinavkgrd
863b814d88
Merge pull request #237 from ente-io/fix-file-type-not-logged
...
missed a place where type info was not logged
2021-11-25 11:55:19 +05:30
Abhinav
7f8d2b84c8
clear failed ffmpeg instance..so new load is trigerred
2021-11-25 11:28:16 +05:30
Abhinav
62a761e5ca
handle missing key or token
2021-11-23 09:45:19 +05:30
Shailesh Pandit
9068b5643a
Limit tsne to 200 faces
...
As it takes exponential time and 200 faces is enough for analysis
2021-11-22 11:49:05 +05:30
Shailesh Pandit
d16837262d
Use arcface alignment for mobilefacenet
...
Use rotation alignment in blazeface to fapi flow
Extract aligned face images before embeddings calculations
2021-11-20 14:56:36 +05:30
Abhinav
e13787df7a
only pass non trashed file user files for search
2021-11-20 12:15:51 +05:30
Abhinav
8929f235e7
improve error handling
2021-11-20 12:06:25 +05:30
Abhinav
b928052165
make fix creation time visible to array of userIds
2021-11-19 00:25:11 +05:30
Abhinav
18c6edab2c
null check
2021-11-19 00:25:11 +05:30
Abhinav
b2162c4398
add options for fix creation time
2021-11-19 00:24:47 +05:30
Abhinav
2b7e5f2074
one more place with same issue
2021-11-18 10:12:19 +05:30
abhinavkgrd
e913615f57
Merge pull request #221 from ente-io/fix-creation-time
...
Fix creation time
2021-11-17 22:26:16 +05:30
Shailesh Pandit
eeec2d1012
Make TFJSFaceEmbeddingService api compatible
2021-11-17 16:42:55 +05:30
Shailesh Pandit
0380fb64e3
Create worker env for face-api without global patch
2021-11-17 16:41:45 +05:30
Abhinav
d62739e259
show fix creation for selected user
2021-11-17 15:10:09 +05:30
Shailesh Pandit
3764e67709
Add required face-api.js files to work with latest tfjs
2021-11-17 14:14:43 +05:30
Shailesh Pandit
4171e8cb3a
Use blazeface for detection, face-api for alignment and embeddings
2021-11-17 14:01:30 +05:30
Abhinav
8ad2d3f23f
skip non image file before downoading itself
2021-11-17 13:51:21 +05:30
Abhinav
0c23ace8ff
remove redundant code
2021-11-17 13:29:46 +05:30
Abhinav
f8de2f21c7
move setProgressTracker to finally block for any case update
2021-11-17 13:16:29 +05:30
Abhinav
80aa50b8fb
only try to parse image exif
2021-11-17 13:11:33 +05:30
Abhinav
80fda73a79
move fix CreationTime option to selectionOption bar
2021-11-16 12:27:23 +05:30
Abhinav
f94322ddef
log the exifparsing inside getExifData itself
2021-11-16 11:40:46 +05:30
Shailesh Pandit
cbaba79db9
Use hdbscan debug info to visualize mst tree
2021-11-14 16:30:23 +05:30
Abhinav
e8aa6141dd
bubble up error to caller from queue processor
2021-11-13 12:50:37 +05:30
Shailesh Pandit
afa8fc2a81
Use hdbscan implemented following scikit-learn package
2021-11-13 10:41:32 +05:30
Neeraj Gupta
ad11422034
Merge pull request #220 from ente-io/fix-file-type-not-logged
...
fix file type not logged on image thumbnail generation fail
2021-11-12 10:50:47 +05:30
Abhinav
88f643192c
fix getExifData breaking for some files
2021-11-10 14:53:23 +05:30
Abhinav
c47d24eb8c
removed unneccessary getExifDataFromURL
2021-11-10 14:52:40 +05:30
Abhinav
3d3213e989
replace exif only if creationTime different than exif
2021-11-10 13:47:21 +05:30
Abhinav
76842b92a6
Merge branch 'master' into fix-creation-time
2021-11-10 12:50:34 +05:30
Vishnu Mohandas
46b7bada93
Merge pull request #219 from ente-io/fix-collection-sort
2021-11-10 12:41:30 +05:30
Abhinav
4b27ed4fd5
updated info object key from type
to fileFormat
, as sentry does not recognise type
as valid property name for some reason
2021-11-10 12:14:56 +05:30
Abhinav
270c43521d
better solution to avoid saving merged metadata to local storage
2021-11-10 10:21:29 +05:30
Shailesh Pandit
4d8c1571df
Use TSNE to visualize faces over 2d plot
2021-11-09 19:16:52 +05:30
Abhinav
3c5e74f16d
update the files array so that returned files from the function is sorted as the caller expect sorted files and use spread operator so that deep copy of file is used to set state to avoid #212 problem
2021-11-09 17:08:27 +05:30
Abhinav
ebe839acf0
Merge branch 'master' into update-file-title
2021-11-09 15:46:12 +05:30
Shailesh Pandit
4bd52d53ee
Use HDBSCAN for clustering and visualize results
2021-11-07 19:09:39 +05:30
Shailesh Pandit
48388aa125
Use centroid of cluster as summary
...
Increase face detection minConfidence
2021-11-07 07:39:28 +05:30
Shailesh Pandit
0ed8df8c17
Make maxFaceDistance configurable
2021-11-06 14:26:12 +05:30
Shailesh Pandit
15fcfbde44
Distribute noise to nearest initial clusters if distance is within limit
2021-11-06 14:03:12 +05:30
Shailesh Pandit
dbb5e99fda
Restrict face distance further
2021-11-06 11:27:07 +05:30
Shailesh Pandit
833f25e07f
Add batch size, return all info to derive face images later
...
Required to reduce worker -> caller payload size later
2021-11-06 10:59:05 +05:30
Shailesh Pandit
4a94ecbbed
Clean tf memory after usage
2021-11-06 09:34:44 +05:30
Shailesh Pandit
0db387cdb1
Tweak default parameters based on initial results
2021-11-05 22:05:10 +05:30
Shailesh Pandit
a5453b8858
Add minFaceSize to limit small faces
2021-11-05 21:44:38 +05:30
Shailesh Pandit
6fb22e429c
Testing with face face-api.js
2021-11-05 20:56:41 +05:30
Shailesh Pandit
41aad5e205
Add Dlib alignment and other clustering methods
2021-11-05 20:35:23 +05:30
Abhinav
b47cf3b1fa
await for fetchUploadURLs
2021-11-05 18:36:25 +05:30
Shailesh Pandit
e5597dc017
Get only unique files for ml sync
2021-11-05 14:03:23 +05:30
Abhinav
91825db617
Merge branch 'master' into update-file-title
2021-11-05 13:26:12 +05:30
Shailesh Pandit
4d1f8d28ac
Undo changes which got removed
2021-11-05 11:24:10 +05:30
Abhinav
80ba92c98c
dont export getCollectionLastSyncTime
2021-11-04 19:25:19 +05:30
Abhinav
7444ad183e
dont set files directly to state, use a copy
2021-11-04 19:01:02 +05:30
Abhinav
d2ee821977
refactored getCollectionLastSyncTime
2021-11-04 18:53:35 +05:30
Abhinav
8d79b443fe
add setLocalFile and remove appendPhotoSwipeProps funcitons
2021-11-04 18:46:04 +05:30
Shailesh Pandit
aeccc14d3c
Make clusterFaceDistance and minClusterSize configurable on ml debug page
2021-11-04 18:23:41 +05:30
Shailesh Pandit
15707f8344
WIP: hacks to ignore dom access from web worker
2021-11-04 17:41:37 +05:30
Shailesh Pandit
955e2e965f
Add dbscan clustering
2021-11-04 17:01:00 +05:30
Shailesh Pandit
6d126e1569
Run face detection and embeddings on all local files
2021-11-04 16:48:10 +05:30
Shailesh Pandit
551e91d278
Add TFJSFaceEmbeddingService and tflite model
...
Set wasm path
2021-11-04 16:35:09 +05:30
Shailesh Pandit
9c70801a16
Add tfjs wasm files, set wasm files path
2021-11-04 16:31:56 +05:30
Shailesh Pandit
2f767ed83f
Add TFJSFaceDetectionService
2021-11-04 16:11:07 +05:30
Shailesh Pandit
377400228a
Machine learning worker
...
Call machine learning service inside a worker
ML Debug page to trigger worker
2021-11-04 15:18:30 +05:30
Vishnu Mohandas
8f77bda4c6
Merge pull request #210 from ente-io/update-change-creation-time-ui
...
Update change creation time UI
2021-11-03 22:02:52 +05:30
Abhinav
e573a7f180
fix file not found chrome error
2021-11-03 21:24:46 +05:30
Abhinav
4c5df92e51
add max and min time limit
2021-11-03 20:06:00 +05:30
Abhinav
e6a3ac35ee
Merge branch 'master' into fix-creation-time
2021-11-03 16:05:32 +05:30
Abhinav
5095bde562
adds updateCreationTimeWithExif script
2021-11-03 16:03:01 +05:30
Vishnu Mohandas
d8eb2231be
Merge pull request #207 from ente-io/migrate-trashed-file-thumbnail
...
migrate trashFiles thumbnail too
2021-11-03 15:53:23 +05:30
Abhinav
801de8f3cc
add log if all large thumbnail are not present locally
2021-11-02 21:05:23 +05:30
Abhinav
ee67921a75
migrate trashFiles thumbnail too
2021-11-02 20:59:31 +05:30
Abhinav
af302f6980
send fileType info with exif data missing logs
2021-11-02 20:50:07 +05:30
Abhinav
99169e407a
correct lower limit edited time
2021-11-02 14:10:03 +05:30
Abhinav
f43e2cf6d6
added file name charcter limit
2021-11-02 13:21:00 +05:30