beenull/ente
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
3893 commits 18 branches 104 tags 490 MiB
Commit graph

1257 commits

Author SHA1 Message Date
Rushikesh Tote ccb56ad313 update on each successful upload 2022-03-24 17:08:01 +05:30
Rushikesh Tote 26f10156c3 fix file name 2022-03-24 11:36:22 +05:30
Rushikesh Tote 1a402b1001 get file type in web app 2022-03-24 10:52:36 +05:30
Rushikesh Tote 35c07784bb refactor according to review 2022-03-24 09:23:57 +05:30
Rushikesh Tote 6e5d38e772 fix errors 2022-03-22 18:34:47 +05:30
Rushikesh Tote 497ea885e1 added deduplication service 2022-03-22 18:31:59 +05:30
Rushikesh Tote 42bfc71de6 address review comments 2022-03-22 11:38:15 +05:30
Rushikesh Tote 02ee68e5a3 refactor 2022-03-20 16:39:49 +05:30
Rushikesh Tote 7a6e14d12f added native file upload for electron 2022-03-18 18:30:01 +05:30
Rushikesh Tote 1082a71359
fix types 2022-03-14 18:40:01 +05:30
Abhinav af9e8d1a61 move timeout logic to heicConverterService 2022-03-14 09:44:47 +05:30
Abhinav abe68ba660 reject on error 2022-03-14 09:11:50 +05:30
Abhinav 3305840c9f add check to only send jwt token header if token present 2022-03-13 15:05:33 +05:30
Rushikesh Tote 579f7eecf6
fix to recover failed imports on the desktop app 2022-03-13 11:57:14 +05:30
Abhinav Kumar 67ee9506b4
Merge pull request #440 from ente-io/improve-exif-parsing 
parseExif by self if exifr return unparsed value
 2022-03-11 20:28:16 +05:30
Abhinav 5e58a9d671 rename to dateTime 2022-03-11 20:20:56 +05:30
Abhinav 74eac71440 refactor code 2022-03-11 20:19:55 +05:30
Abhinav 935e09f2b8 return null on error 2022-03-11 20:11:59 +05:30
Abhinav 0e45731c3a parseExif by self if exifr return unparsed value 2022-03-11 20:00:55 +05:30
Abhinav 603d961eca Merge branch 'master' into settings_button 2022-03-11 19:01:26 +05:30
Abhinav 0c2ef1f9b8 return results 2022-03-11 13:41:50 +05:30
Abhinav 6eec9a8af6 reduce retry to 3 2022-03-10 22:07:30 +05:30
Abhinav 8da1a3d2eb add heic conversion failed logs 2022-03-10 21:41:07 +05:30
Abhinav d55ceb4d8b logError on each failed attempt 2022-03-10 21:38:57 +05:30
Abhinav 6b5e4e5f0b reduce wait and breath time 2022-03-10 21:23:14 +05:30
Abhinav b84e49d1e4 handle revive fail 2022-03-10 19:55:38 +05:30
Abhinav 9bf97ff457 add retry for heic conversion 2022-03-10 13:16:36 +05:30
Abhinav c594364f06 add upload http callls specific retrier 2022-03-10 13:14:42 +05:30
Abhinav c61dbccdb1 add a 100ms breath time after heic conversion 2022-03-10 12:38:28 +05:30
Abhinav 933fd2f367 discard worker and create new one if heic conversion fails 2022-03-10 12:10:18 +05:30
Abhinav 6a4a1e4f42 Merge branch 'master' into fix-heic-convert-fail 2022-03-10 10:12:04 +05:30
Abhinav Kumar 0cbabbc234
Revert "Skip jpeg type detection" 2022-03-10 09:54:03 +05:30
Abhinav 992b8fb1d8 fix variable name 2022-03-10 09:24:57 +05:30
Abhinav cb754b562b move FORMAT_MISSED_BY_FILE_TYPE_LIB to typeDetectionService 2022-03-10 09:23:46 +05:30
Abhinav 6c9385a65d Revert "Merge pull request #435 from ente-io/skip-jpeg-type-detection" 
This reverts commit 4f6e3158b4, reversing
changes made to 0ced60ac05.
 2022-03-10 09:21:13 +05:30
Abhinav 551ed5804d Merge branch 'master' into fix-heic-convert-fail 2022-03-10 09:19:31 +05:30
Abhinav e6592284aa fix bug 2022-03-09 14:28:03 +05:30
Abhinav 427d138b78 only do heic convert files who whose extension is heic/heif 2022-03-09 14:12:50 +05:30
Abhinav 6bd11414d3 updated file-type library to latest version 2022-03-09 13:27:09 +05:30
Abhinav b4e40ff036 skip jpeg type detection 2022-03-09 13:23:02 +05:30
Abhinav 9b53942257 revert to jpeg 2022-03-09 13:14:32 +05:30
Abhinav 34efd0362d limit parallel exection to one 2022-03-09 13:01:58 +05:30
Abhinav 1ef333a15c remove console logs 2022-03-09 12:46:38 +05:30
Abhinav 206b15b962 add timeout to heic conversion 2022-03-09 12:44:07 +05:30
Abhinav faad7f554e increase heic count 2022-03-09 11:15:59 +05:30
Abhinav a96c81451a bug fix 2022-03-09 11:11:56 +05:30
Abhinav 8458ae8a2c fail if thumbnail generation fails 2022-03-09 11:10:36 +05:30
Abhinav ec15d63a7f handle HIEC multiple externally , process two files at a time , using a double the size of that as worker pool also use png as it didn't crash for two parralel files and add a timeout to heic conversion 2022-03-09 11:10:20 +05:30
Abhinav Kumar d8de8bb250
Merge pull request #429 from ente-io/ios_import_support 
Support for preserving iCloud Album structure
 2022-03-08 10:52:34 +05:30
Abhinav 74ce03748a will defintely have collection in uploader, 
if not the process should fail lets not silently ignore it
 2022-03-08 10:51:03 +05:30
Abhinav 0a6cdd024a fix typo 2022-03-08 10:43:33 +05:30
Abhinav b3fd97a6e5 disable logs 2022-03-07 19:19:12 +05:30
Abhinav df3eaa8a00 improve ready 2022-03-07 19:12:44 +05:30
Abhinav c771e955c2 undo testing changes 2022-03-07 18:49:28 +05:30
Abhinav d3ce6e9a62 fix ffmpegService import 2022-03-07 18:49:16 +05:30
Abhinav de174c2a37 offload ffmpeg operations on custom worker 2022-03-07 18:48:46 +05:30
Abhinav 7b827bde68 refactor ffmpeg specfic code to ffmpeg util 2022-03-07 18:46:53 +05:30
Abhinav e1b7212496 Merge branch 'refactor-services' into ffmpeg-single-thread 2022-03-07 17:30:14 +05:30
Abhinav da53e12869 build fix 2022-03-07 17:29:21 +05:30
Abhinav 47737d2dea refactor and create typeDetectionService and readerService 2022-03-07 17:26:11 +05:30
Neeraj Gupta 05faaeb0c9
minor refactoring 2022-03-07 17:18:05 +05:30
Neeraj Gupta 3dcf337f6c
Support for preserving iCloud Album Struture 2022-03-07 16:54:27 +05:30
Abhinav a78257a3f8 uplaod video as image for testing thums 2022-03-07 15:58:10 +05:30
Abhinav a9652e5b6c single threaded ffmpeg exit running change worked 2022-03-07 13:10:44 +05:30
Abhinav ab64c84f64 Revert "update to use single use ffmpeg st" 
This reverts commit 65667ac416.
 2022-03-07 13:07:25 +05:30
Abhinav 65667ac416 update to use single use ffmpeg st 2022-03-05 18:44:44 +05:30
Abhinav 98f1f98650 add progress bar for metadata extraction 2022-03-05 13:32:57 +05:30
Abhinav 6406867770 update to use the local ffmpeg wasm package 2022-03-05 12:39:17 +05:30
Abhinav 009590ef54 add option to skip put for local 2022-03-05 07:51:02 +05:30
Abhinav ff11cb14bc verbose logs 2022-03-04 13:14:39 +05:30
Abhinav 76776fcf5d set file name for converted file 2022-03-04 11:13:38 +05:30
Abhinav 3c0dc7a63d log skipped files 2022-03-04 09:35:53 +05:30
Abhinav 9d5917516c Revert "log last attemped video metadata extraction" 
This reverts commit d72f16a884.
 2022-03-04 09:24:15 +05:30
Abhinav 685bc14899 added upload logs and download log option 2022-03-04 09:16:18 +05:30
Abhinav 72e0f7318c add upload logs 2022-03-04 08:40:14 +05:30
Abhinav d72f16a884 log last attemped video metadata extraction 2022-03-04 07:27:33 +05:30
Abhinav Kumar f24f2bd00f
Merge pull request #411 from ente-io/log-failed-files 
Log failed files
 2022-03-03 19:03:04 +05:30
Abhinav 9225cb70fa log file size along with file name 2022-03-03 17:15:02 +05:30
Neeraj Gupta 0e32b2ac89
Merge branch 'master' into settings_button 2022-03-03 16:51:52 +05:30
Abhinav 17dd56a999 log failed files only 2022-03-03 14:53:20 +05:30
Abhinav 5bf20e046e log failed files in localDB 2022-03-03 09:33:18 +05:30
Rushikesh Tote 5c40812b4c
Merge branch 'master' 2022-03-03 00:00:01 +05:30
Abhinav Kumar 913a3a65aa
Merge pull request #409 from ente-io/fix-loading 
Fix loading
 2022-03-02 22:30:44 +05:30
Abhinav ae8db26a00 fix video loader not showing issue 2022-03-02 16:17:06 +05:30
Neeraj Gupta 44eecaf92f
Increase timeout for thumbnail gen 10s -> 30s 
During testing, observed that for multiple copies of the same HEIC files, the thumbnail was blank for some files due to timeout error. 
Increasing this timeout fixed the issue.
 2022-03-02 12:43:22 +05:30
Rushikesh Tote 28197afd23
Merge branch 'master' into add-livephoto-support 2022-03-02 12:00:55 +05:30
Rushikesh Tote 890c41da69
fixed styling, live photo split 2022-03-01 18:19:07 +05:30
Abhinav a86b9134d2 also check location tag in the metadata as fallback 2022-03-01 12:50:45 +05:30
Abhinav ca13691688 extractedMetadata will allows have the properties so no need of conditional access 2022-02-28 18:27:44 +05:30
Abhinav a866d109e5 rename getUNIXTime to getUnixTimeInMicroSeconds 2022-02-28 17:44:23 +05:30
Abhinav 222167878a better handle metadata extraction failure 2022-02-28 14:03:47 +05:30
Abhinav cf1f55e68a moved types and getUnixTime to time util 2022-02-28 13:48:52 +05:30
Neeraj Gupta a6a4ecbf46
Fix minor bug in put share-url api call 2022-02-28 13:48:30 +05:30
Abhinav a98d35ad66 refactoring moved videoMetadata extraction to seperate service 2022-02-28 13:42:33 +05:30
Abhinav e49a7f9ec9 better types 2022-02-28 13:29:30 +05:30
Abhinav c488d76609 update comment for extractVideoMetadata command 2022-02-28 13:23:34 +05:30
Abhinav 5887aec373 read all available stream metadata dont need to specifically read video and audio streams 2022-02-28 13:05:14 +05:30
Neeraj Gupta 5646fff10b
collectionService: method for updating sharable url 2022-02-28 11:54:08 +05:30
Abhinav a4d81da065 Merge branch 'master' into ffmpeg-video-metadata 2022-02-28 10:47:50 +05:30
Abhinav 540620373b fix minor bugs 2022-02-28 10:34:27 +05:30
Abhinav 1af2dc24ca add metadata extraction logic 2022-02-28 10:34:16 +05:30
Rushikesh Tote 1eee5e8a74
fixed: one ffmpeg process at any time 2022-02-28 10:04:20 +05:30
Rushikesh Tote 6188c9eed4
fixed issues with video conversion 2022-02-27 12:04:51 +05:30
Rushikesh Tote 4e33ce0cf6
Merge remote-tracking branch 'origin/master' into add-livephoto-support 2022-02-26 20:16:32 +05:30
Abhinav 63d3fb34cc fix error message for verify password api 2022-02-26 18:41:52 +05:30
Abhinav bda2423930 fix variable name for get and save LocalPassword and avoid using token directly as collectionUID 2022-02-26 15:37:46 +05:30
Abhinav 0a028e7b07 dont make syncCall if already up to date with collection update time 2022-02-26 14:45:31 +05:30
Abhinav 29c7cc69c7 handle password change case 2022-02-26 14:20:31 +05:30
Abhinav c6dd93270d revert back to using ref for password token 2022-02-26 12:34:43 +05:30
Abhinav 86488ee056 minor changes and refactoring 2022-02-25 15:58:12 +05:30
Neeraj Gupta fc45b94930
switch to different api for validating link password 2022-02-24 21:09:00 +05:30
Rushikesh Tote 60fd87df8f added live photo playback support 2022-02-24 19:26:57 +05:30
Neeraj Gupta bb5b37997e
add support for showing password protected albums 2022-02-23 14:31:00 +05:30
Neeraj Gupta b93fb5167c
remove entry for publicCollection sync time while removing collection 2022-02-22 12:24:56 +05:30
Abhinav Kumar f7cb51f856
Merge pull request #253 from ente-io/live-photo-import 
add live photo upload support
 2022-02-16 20:18:38 +05:30
Abhinav 671bf0ee2d add live photo detection info section 2022-02-16 13:13:47 +05:30
Abhinav 0474d28aec remove only underscore three from the end 2022-02-16 12:30:34 +05:30
Abhinav 5744def6f2 added logic to show filename in upload progressbar 2022-02-16 12:18:07 +05:30
Abhinav 602f36ee37 use the ext returned from the fileType detection library to set the correct extension for the file 2022-02-16 12:04:16 +05:30
Abhinav 227a04a01f update file type detection function 2022-02-16 12:04:05 +05:30
Abhinav 4d74c62eb2 add try catch wrapper over extractFileMetadata and parseMetadataJSON to prevent one file breaking to progagate cause all of them to fail 2022-02-16 10:44:32 +05:30
Abhinav 3b9841b9c4 use the last index of underscore 2022-02-16 10:13:18 +05:30
Abhinav 47c6fabc05 isImageOrVideo util function and updated the areFilesLivePhotoAssets to allow only image and video as allowed assets 2022-02-16 10:06:28 +05:30
Abhinav 8534f69e96 refactor code 2022-02-16 09:57:33 +05:30
Abhinav 79455d2295 added imageType and videoType property to FleTypeInfo to handle livePhoto case 2022-02-16 09:52:00 +05:30
Abhinav 044b257972 dont need cryptoWorker for thumbnail generation convertService handles it 2022-02-16 09:29:50 +05:30
Abhinav 58dbac8ffd remove underscored suffix added to live photo name while comparing file names 2022-02-16 09:08:35 +05:30
Abhinav 0b34bcbba2 set livePhotoLocalID correctly 2022-02-15 19:21:53 +05:30
Abhinav f58108a55f fix type passed to thumbnail generation for heic detection 2022-02-15 16:32:42 +05:30
Abhinav 0ea457e5f8 fix imageMetadata intialisation 2022-02-15 09:52:17 +05:30
Abhinav c04056569b use image metadata for live photo 2022-02-15 09:48:38 +05:30
Abhinav 80db61133a refactored code to not create stream for livePhotodata assets as they zipping library doesn't support streams 2022-02-15 09:20:41 +05:30
Abhinav 8b668b924a use file name as metadata for some files may be empty 2022-02-15 09:01:19 +05:30
Abhinav 5ff8b857bc add comment and check to avoid larger than multipart file size files to be clustered as livePhoto 2022-02-15 08:12:38 +05:30
Abhinav 08cd6e72ea group same collection files together before live photo clustering 2022-02-14 16:43:56 +05:30
Abhinav 22a489be6e only generate and use image thumbnail for livePhoto 2022-02-14 13:17:10 +05:30
Abhinav ed06e1427e use video thumbnails for livePhotos 2022-02-14 13:05:22 +05:30
Abhinav dab1844223 reuse fileTypeInfo and metdata and fix case where live photos clustering 2022-02-14 12:44:42 +05:30
Abhinav 04f4fd14e3 uploadService getFileMetadata to better name extractFileMetadata to avoid confusion with getFileMetadataAndFileTypeInfo 2022-02-13 16:48:42 +05:30
Abhinav ccd730cf9f minor changes 2022-02-13 16:40:50 +05:30
Abhinav 1398ea22b3 set live photo extension as zip 2022-02-13 16:34:08 +05:30
Abhinav 86a9b86294 handle metadata null case 2022-02-13 16:26:46 +05:30
Abhinav 0fff2ee674 fix minor bugs 2022-02-13 14:40:17 +05:30
Abhinav 87d635be58 Merge branch 'refactor-metadata-extraction-v2' into live-photo-import 2022-02-13 12:59:43 +05:30
Abhinav 7b45aee757 refactor code 2022-02-13 09:43:25 +05:30
Abhinav 13d8c10a60 Merge branch 'master' into live-photo-import 2022-02-13 08:29:16 +05:30
Abhinav bbb78a6c5e update publicCollectionUID template 2022-02-12 14:03:13 +05:30
Abhinav eb23444853 create a POC working model of live photo upload 2022-02-09 10:53:43 +05:30
Abhinav 858315c897 added stage for metadata extraction 2022-02-09 10:32:14 +05:30
Abhinav Kumar c801e8225f
Merge pull request #363 from ente-io/public-link-changes 
Public link changes
 2022-02-06 16:43:34 +05:30