Commit graph

402 commits

Author SHA1 Message Date
Michael Mayer 1af2d295f0 Metadata/Nextcloud: Parse Two-Digit Years in Original File Paths #2271 2022-04-22 19:48:31 +02:00
Michael Mayer 8638929d84 Config: Automatically generate command-flag docs #1017 #2195 #2227 #2250 2022-04-22 17:38:40 +02:00
Michael Mayer f33f06933f Index: Reduce log level for empty files from error to warning #2257 2022-04-18 17:21:31 +02:00
Michael Mayer 38c6cebe21 File Formats: Disable TOML file support until there is a use for it 2022-04-18 00:14:04 +02:00
Michael Mayer 182bc09d87 CLI: Export reports as CSV/TSV for use in spreadsheets #2247 #2252 #2169 2022-04-17 12:30:33 +02:00
Michael Mayer 73be4df8f8 CLI/Docs: Improve photoprism show ... subcommands #2247 #2252 #2169 2022-04-16 19:52:53 +02:00
Michael Mayer 92e6c4fe1e Download: Add Disabled, Originals, MediaRaw & MediaSidecar Flags #2234
Extends DownloadSettings with 4 additional options:
- Name: File name pattern for downloaded files (existed)
- Disabled: Disables downloads
- Originals: Only download files stored in "originals" folder
- MediaRaw: Include RAW image files
- MediaSidecar: Include metadata sidecar files (JSON, XMP, YAML)
2022-04-15 09:42:07 +02:00
Michael Mayer 1baada39f4 Search: Remove raw and gif from stopwords.txt #590 #1994 #2079 #2169 2022-04-14 13:56:44 +02:00
Michael Mayer e4999db92e Metadata: Add IPTC 2021.1 JSON/YAML Schemas incl. test files #1143 #2252 2022-04-14 12:18:05 +02:00
Michael Mayer 0096243240 Metadata: Report supported Exiftool, XMP, and Dublin Core tags #2252
Replaces the --no-wrap flag with --md in all "photoprism show ..."
subcommands, as this is easier to understand. See also #2247.
Unused code was opportunistically removed along the way.
2022-04-14 10:49:56 +02:00
Michael Mayer b3113e006f Search: Add vector, animated, svg, and gif filters #590 #2207 2022-04-14 08:39:52 +02:00
Michael Mayer 82d61d1f93 File Types: Add experimental support for animated GIFs #590 #2207
Animated GIFs are transcoded to AVC because it is much smaller and
thus also suitable for long/large animations. In addition, this commit
adds support for more metadata fields such as frame rate, number of
frames, file capture timestamp (unix milliseconds), media type,
and software version. Support for SVG files can later be implemented in
a similar way.
2022-04-13 22:17:59 +02:00
Michael Mayer 0427163295 People: Improve face matching, audit command, and logging #2182 2022-04-13 01:59:32 +02:00
Michael Mayer e42b870c09 CLI: Update and display the list of supported file formats #2247 2022-04-12 19:14:21 +02:00
Michael Mayer 68ba289d6c CLI: Add "photoprism show config/formats" subcommands
Replaces "photoprism config", which could only display
current configuration values. Codecs and file formats
have been refactored along the way.

Signed-off-by: Michael Mayer <michael@liquidbytes.net>
2022-04-12 13:28:28 +02:00
Michael Mayer f927c68c4f UX: Skip RAW files by default when clicking Download All #2234
Signed-off-by: Michael Mayer <michael@liquidbytes.net>
2022-04-10 17:02:09 +02:00
Michael Mayer e1996f80b3 Exif: Ignore IFD1 tags with existing IFD0 values #2231
Signed-off-by: Michael Mayer <michael@liquidbytes.net>
2022-04-09 19:56:38 +02:00
Michael Mayer 9986986f8f People: Add info on the kind of face to improve matching #2182
This commit also fixes of other potential issues and improves logging.
2022-04-04 21:22:31 +02:00
Michael Mayer c74fcbf282 People: Show real name instead of uid in logs #1438 #2182
Since caching all subject data proved too complex in the time available,
this implementation uses a simple key/value lookup table to cache
subject names and perform backward searches by uid.
2022-04-04 14:21:43 +02:00
Michael Mayer ccea0d01db Metadata: Update specialwords.go for title generation in txt package
Signed-off-by: Michael Mayer <michael@liquidbytes.net>
2022-04-04 01:22:29 +02:00
Michael Mayer 41b252d820 People: Add unofficial env variables to tweak face matching #1587 #2182
Adds two unofficial env variables so advanced users can experiment:

1. PHOTOPRISM_FACE_KIDS_DIST=0.6950 (range: 0.1-1.5, -1 to disable)
2. PHOTOPRISM_FACE_IGNORE_DIST=0.86 (range: 0.1-1.5, -1 to disable)
2022-04-03 17:25:37 +02:00
Michael Mayer fab9da971c Metadata: Update specialwords.go for title generation in txt package 2022-04-03 12:27:17 +02:00
Michael Mayer a604e9a9c6 Index: Improve file size/resolution checks, add WebP support #1017 #1226
Renames the config flag to from "megapixel-limit" to "resolution-limit".
Adds native support for the WebP image file format.
2022-04-02 18:04:11 +02:00
Michael Mayer 44efdd232a Config: Improve flag descriptions of JPEG and thumbnail parameters #2215 2022-04-01 13:25:25 +02:00
Michael Mayer 9eda12ac20 Performance: Refactor database schema and UI components #995 #1438 #1811 2022-03-30 20:36:25 +02:00
Michael Mayer b8ea17d595 Package txt: Add ascii_table.txt for reference 2022-03-28 19:52:24 +02:00
Michael Mayer 9e46a66f24 Search: Make special character escaping compatible with SQLite #1994 2022-03-28 17:36:59 +02:00
Michael Mayer e77a029f13 Search: Improve input sanitation and filter queries #1994 #2079 #2181 2022-03-25 18:01:34 +01:00
Michael Mayer 680686185f Metadata: Improve parsing of (incomplete) timestamp strings #625 #2133 2022-03-25 16:31:09 +01:00
Michael Mayer 0e7fec1cec Albums: Shorten title and slug to avoid "data too long" error #2181
Related GitHub Discussions:
 - https://github.com/photoprism/photoprism/discussions/2151
 - https://github.com/photoprism/photoprism/discussions/2179
2022-03-24 21:46:25 +01:00
Michael Mayer 9d110e8b80 Search: Improve album, albums, lens, and camera filters #1994 #2079
Camera and lens can now also be searched by name. Escaping and parsing
of albums has been improved so that albums whose names start with and/or
contain numbers will be found.
2022-03-24 18:30:59 +01:00
Michael Mayer c256664a1b WebDAV: Fix upload of complete albums #1376 2022-02-27 17:32:54 +01:00
Michael Mayer c853b3cd2d Backend: Shorten source file headers and normalize email addresses 2022-02-21 15:30:18 +01:00
Michael Mayer b29ab34e9a Config: Update fs_test.go and fs/dirs.go after upgrading dev base image 2022-02-19 17:12:39 +01:00
Michael Mayer 03457bdb75 Frontend: Refactor asset loading #1648 2022-01-18 12:26:33 +01:00
Michael Mayer ff41a94051 Search: Fix query string substitutions and sanitation #1930 2022-01-17 16:07:31 +01:00
Michael Mayer 006b5bb704 Code Quality: Add "gofmt -w -s pkg internal cmd" to "make fmt-go"
gofmt formats Go source code, where -s stands for "simplify"
2022-01-08 12:12:00 +01:00
Michael Mayer 8efe473d30 Copyright: Update year in source files and footer.vue 2022-01-03 13:46:52 +01:00
Michael Mayer e858a34f3e Search: Return empty result if keywords contain stopwords only #1859 2022-01-03 13:30:34 +01:00
Michael Mayer 10d9ec78cd Config: Search /opt/photoprism for assets and media files #1821 2021-12-17 11:30:49 +01:00
Michael Mayer e8a1e85f14 Search: Improve query string normalization and sanitation #1814 #1820 2021-12-16 15:26:54 +01:00
Michael Mayer 2dedbb83dc Sanitize: Add name, query, state, and username filters #1814 2021-12-15 12:24:05 +01:00
Michael Mayer 0f0c0aaa0b API: Improve path and filename parameter sanitation #1814 2021-12-14 20:01:39 +01:00
Michael Mayer 4e94919030 API: Improve request parameter sanitation #1814 2021-12-14 18:34:52 +01:00
Michael Mayer 9a8144c046 Log4j: Sanitize user input in logs like filenames and album titles #1814 2021-12-14 17:07:38 +01:00
Michael Mayer e9fabfa8e4 Docs: Update URL 2021-12-12 20:14:07 +01:00
Michael Mayer 5be456a09f JPEG: Convert Apple "Display P3" colors to standard sRGB #1474
Other color profiles and file formats are not supported yet. Should
be easy to add though. Main difficulty will be profile name comparison:
For example "Adobe RGB (1998)" vs just "Adobe RGB".
2021-12-09 07:00:39 +01:00
Michael Mayer 57ba7a2868 Text: Add "île-de-france" to specialwords.go #1776 2021-12-06 14:12:52 +01:00
Michael Mayer c6fe4cb1ea Places: Map "France métropolitaine" to empty string #1776 2021-12-03 16:53:18 +01:00
Michael Mayer 8d9c60c6c1 Places: Map "France métropolitaine" to "Île-de-France" #1776 2021-11-30 13:32:10 +01:00
Michael Mayer 87831c0a94 Metadata: More accurate location estimates #1668 2021-11-24 16:56:57 +01:00
Michael Mayer 45d1360c18 Metadata: Improve location estimates and UTC handling #1668 2021-11-23 19:23:10 +01:00
Michael Mayer d813171204 Metadata: Estimate latitude and longitude if possible #1668 2021-11-22 18:18:41 +01:00
Michael Mayer df1ffc68cf UX: Reduce loading-animation size on slash screen 2021-11-20 16:27:30 +01:00
Michael Mayer c48310f077 Albums: Improve state geodata and sort by country #1608 #1664 2021-11-18 00:46:34 +01:00
theresa 8dcf410cc9 Tests: Add unit tests for pkg/txt 2021-11-17 18:15:07 +01:00
Jan Loewe f5dc3fae4a
Places: Add Normalization Mappings for NDS (Germany) #1664 (#1729) 2021-11-12 09:14:48 +01:00
Michael Mayer a354a17041 Auth: Normalize Username (trim space, lowercase) #1103 #1309 #1546 #1610 2021-11-12 09:10:15 +01:00
Max Nuding 666a2358fa
Places: Add Normalization Mappings for BW and RLP (Germany) #1664 #1726 2021-11-12 07:35:40 +01:00
Michael Mayer bb9442e6c4 People: Add "photoprism places update" CLI command #1664 2021-11-12 05:09:17 +01:00
Michael Mayer 72d546f8c8 Places: Add state name normalizations for AU and NZ #1664 2021-11-11 17:41:01 +01:00
Michael Mayer 0575ab2276 Places: Add state name normalizations for Germany #1664 2021-11-11 17:10:52 +01:00
Michael Mayer 6c02ee2512 Places: Improve state name normalization #1664 2021-11-11 16:00:42 +01:00
Michael Mayer e525ce5c46 Places: Don't auto capitalize state names #1664 2021-11-09 19:56:19 +01:00
Michael Mayer 051251bafb Text: Update specialwords.go 2021-11-09 16:55:47 +01:00
Michael Mayer e4fd294689 Places: Normalize US & CA state names #1664 2021-11-09 11:42:10 +01:00
Michael Mayer 25a954d568 Search: Add live and raw search filters, improve faces filter #22 2021-10-13 16:12:56 +02:00
Michael Mayer 2a119380a7 Folders: Ignore names listed in .ppignore #1609 2021-10-09 13:57:28 +02:00
Eng Zer Jun 44f7700c0c
Enable module graph pruning and deprecate io/ioutil (#1600)
* Backend: Enable Go module graph pruning and lazy module loading

This commit applies the changes by running `go mod tidy -go=1.17` to
enable module graph pruning and lazy module loading supported by Go 1.17
or higher.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* Backend: Move from io/ioutil to io and os package

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-06 07:10:50 +02:00
Michael Mayer f7153cdd21 People: Improve logging, command help, and handling of broken files #22 2021-10-05 18:42:39 +02:00
Michael Mayer 24eff21aa4 Search: Default to photo names and keywords #1517 #1560
Default to photo name when search term is too short or on the stop list.
Search full text index otherwise, which now include names of people
(requires reindexing).
2021-09-29 20:09:34 +02:00
Michael Mayer 125b05b489 Search: Remove sea. sinon, and sino from stopwords list 2021-09-25 15:56:44 +02:00
Michael Mayer 2929733f78 Index: Ignore files and folders starting with @ #1543
Synology stores hidden sidecar / thumbnail files in @eaDir.
2021-09-24 01:24:36 +02:00
Michael Mayer eb8bc7b709 Backend: Improve resilience #1544 2021-09-23 23:46:17 +02:00
Michael Mayer 3c3b19aa8e People: Add search filter for finding unclustered faces #22 2021-09-23 14:23:00 +02:00
Michael Mayer 8eb970ae8b RAW: Remove unwanted characters like quotes from file extensions #1362 2021-09-21 18:10:21 +02:00
Michael Mayer b081f27e17 RAW: Add File Extension Blacklists for Darktable and RawTherapee #1362 2021-09-21 15:33:46 +02:00
Michael Mayer 070efcbc66 Search: Improve "year", "month", and "day" filters 2021-09-20 23:32:35 +02:00
Michael Mayer 77b7fb2582 Text: Add tests for short words 2021-09-20 22:28:20 +02:00
Michael Mayer 0de2a492bb Text: Update and add tests for short words 2021-09-20 22:27:18 +02:00
Michael Mayer 04d2cfa650 People: Use slug as search filter #22 2021-09-20 12:36:59 +02:00
Michael Mayer 1f92f294dd People: Normalize names #22 2021-09-19 13:35:44 +02:00
Michael Mayer f7cc61edbc People: Shorten names in titles #22 2021-09-18 21:40:57 +02:00
Michael Mayer ed22f245db People: Add faces API endpoint & JS model #22 2021-09-18 15:32:39 +02:00
Michael Mayer 0f90258ef9 Search: Omit full text index if query is too short #1517 2021-09-17 15:52:25 +02:00
Michael Mayer c584424183 Labels: Disable client-side capitalization #1509 2021-09-06 15:00:41 +02:00
Michael Mayer cf9fb1db1a People: Combine family names in photo titles #22 2021-09-05 22:12:53 +02:00
Michael Mayer 6f5e839e2f People: Move crop package to internal #22 2021-09-05 11:05:33 +02:00
Michael Mayer 9ba396e50f Search: Detect and set filters based on search terms
Reduces the amount of typing necessary to search for faces, videos,
favorites, stacks, panoramas, scans, and monochrome pictures.
2021-09-03 20:14:11 +02:00
Michael Mayer 2bde7e5696 Panoramas: Increase projection type string limit to 32 characters #1508 2021-09-03 19:02:26 +02:00
Michael Mayer 34c8f9d7b1 People: Implement crop thumbnail API #22 2021-09-03 17:42:37 +02:00
Michael Mayer c520cb4ee4 People: Add autocomplete for selecting a person #22 2021-09-03 16:14:09 +02:00
Michael Mayer c179a790b8 People: Improve face crop caching #22 2021-09-02 23:47:37 +02:00
theresa c61a0f7d0e Tests: Add unit tests 2021-09-02 17:44:27 +02:00
Michael Mayer 9acd4a25b9 People: Generate photo titles from subject names #22 2021-09-02 14:23:40 +02:00
Michael Mayer a5f8e6149f Titles: Add German smallwords to improve capitalization 2021-09-02 09:37:23 +02:00
Michael Mayer 7f6fd7f47a Backend: Update Go build tags after upgrading to 1.17 2021-08-31 18:49:08 +02:00
Michael Mayer c0a21045fb People: Improve search query parser #22 #882 2021-08-30 11:26:57 +02:00
Michael Mayer 71372c6f87 Videos: Add .mp to known file extensions #1501 2021-08-29 16:26:20 +02:00
Michael Mayer 11d1034752 People: Add keywords, subjects, and albums search filters #22 #882 2021-08-29 16:16:49 +02:00
Michael Mayer 2e85b3cccd People: Split facial recognition into smaller functions #22
Clustering and matching have been improved along the way. This opens
the door for further optimizations while keeping the code readable.
2021-08-22 16:14:34 +02:00
Michael Mayer 29a65dbaab People: Fork clusters library with fixed dbscan implementation #1478 2021-08-19 14:24:30 +02:00
Michael Mayer 97c3e0abe8 Import: Ignore dot files listed in .ppignore #1348 2021-07-16 18:27:28 +02:00
Michael Mayer 07c711a0b4 RAW: Skip Darktable when converting Canon CR3 and Fuji RAF files #1362 2021-07-16 18:12:42 +02:00
Michael Mayer 49fd531420 People: Implement marker update API #22 2021-06-02 17:25:04 +02:00
Michael Mayer 58e3573e10 People: Add include and exclude as aliases for yes and no in search #22 2021-05-26 11:04:14 +02:00
Michael Mayer 5ecba4926a People: Show number of faces in edit dialog #22 2021-05-26 09:51:00 +02:00
Michael Mayer d3e53a89dd Metadata: Allow single quotes in keywords #1196 2021-05-05 12:32:49 +02:00
Michael Mayer 256c82fc93 Metadata: Trim trailing - from keywords #1153 2021-05-05 12:23:19 +02:00
Michael Mayer 1fea0b136e Backend: Fix potential issue in txt.Int() 2021-05-04 17:54:57 +02:00
Michael Mayer 50d2ff1e69 Backend: Upgrade dependencies and fix potential issues 2021-05-04 17:30:39 +02:00
Michael Mayer e10118a65f Metadata: Merge keywords if priority is the same #1153
Also reduces length limit for latin words to 2 letters.
2021-05-04 15:02:54 +02:00
Michael Mayer 6cbf0696c4 Metadata: Don't remove stopwords from keywords #1153 2021-04-30 17:37:37 +02:00
Michael Mayer 014642f401 Stopwords: Remove done, elsewhere, ended, exactly #1153 2021-04-27 10:02:34 +02:00
Michael Mayer 27c106863d Metadata: Merge existing keywords #1153 2021-04-25 14:17:34 +02:00
Michael Mayer 7de3713024 Backup: Improve command flags and usage docs #1190 #1193 2021-04-19 13:48:46 +02:00
Krassimir Valev 2bace7954d
Remove Niagara Falls from country mapping (#1144)
The waterfalls are on a border, so they cannot be uniquely assigned to Canada
(or USA).
2021-03-28 20:23:49 +02:00
theresa 816aca7420 Tests: Add tests for internal/api 2021-03-10 16:55:55 +01:00
theresa ce9651df10 Tests: Add tests for /pkg 2021-03-03 15:41:32 +01:00
Michael Mayer d78310c8f0 Videos: Add .m4v to known mp4 video file extensions #703 #1039 2021-02-15 13:02:55 +01:00
Krassimir Valev 58d6d23967
Support .mts and .mpo files (#1029) 2021-02-14 19:25:52 +01:00
Michael Mayer 9039774cf7 Metadata: Rename unknown country code variable 2021-02-07 17:42:52 +01:00
Michael Mayer e57e9aff70 Metadata: Improve country code from string estimate 2021-02-07 17:39:41 +01:00
Michael Mayer 789cab83c9 Metadata: Add special words for title capitalization 2021-02-05 09:13:05 +01:00
Michael Mayer 57dc591b12 Remove small words from filename title endings 2021-01-27 15:21:54 +01:00
Michael Mayer ae2f761a05 Backend: Add strings to specialwords.go 2021-01-27 11:59:41 +01:00
Michael Mayer 899419383a Metadata: Fix indexing keywords in original file name and path #920 2021-01-25 19:30:29 +01:00
Michael Mayer a564cff3af Metadata: Update countries.go 2021-01-15 11:52:12 +01:00
Krassimir Valev 405fb3c459
Metadata: Add more places to the countries mapping (#899) 2021-01-15 10:47:27 +01:00
theresa e01b193f88 Stopwords: Add dscn 2021-01-11 17:41:34 +01:00
Michael Mayer 19b2f1de13 Import: Improve handling of severe errors (panics) #261 2021-01-09 13:50:14 +01:00
Michael Mayer a8c46f616a Colors: Clean-up examples 2021-01-09 04:45:08 +01:00
Michael Mayer 9ec2381307 Update copyright after year change 2021-01-01 22:37:25 +01:00
Michael Mayer 8e22fbf8f6 Metadata: Store and compare data source for details #755 2020-12-31 13:51:31 +01:00
Michael Mayer 4614cbe4ea Frontend: Enable search for 2 character words #773 #746 2020-12-30 13:59:27 +01:00
Michael Mayer 8cfabe3205 Metadata: Cache ExifTool JSON by original file hash #755 #759 2020-12-30 13:33:47 +01:00
Michael Mayer 91acaaa573 Index shorter keywords in languages like Chinese #746
Signed-off-by: Michael Mayer <michael@liquidbytes.net>
2020-12-27 16:37:28 +01:00
Michael Mayer 02722ab861 Config: Detect case-insensitive file systems
Signed-off-by: Michael Mayer <michael@liquidbytes.net>
2020-12-26 18:06:54 +01:00
Michael Mayer d430ae24ee Indexer: Fix JSON sidecar creation using Exiftool
Signed-off-by: Michael Mayer <michael@liquidbytes.net>
2020-12-25 20:29:06 +01:00
Michael Mayer 92b99060c1 Add capitalization tests #737
Signed-off-by: Michael Mayer <michael@liquidbytes.net>
2020-12-23 22:54:46 +01:00
Michael Mayer 859e4f7d8c Indexer: Set a default color for photos with multiple files #715 2020-12-17 12:32:53 +01:00
Michael Mayer 3d7c9323d8 Indexer: Improve color & chroma classification #596 2020-12-17 11:47:31 +01:00
Michael Mayer c4b42d7e61 Refactor txt package 2020-12-16 12:34:39 +01:00
Michael Mayer 473566f404 Videos: Enforce transcoding to MPEG-4 AVC #603 2020-12-12 17:20:31 +01:00
Michael Mayer 73a00efae8 Workaround for bad image rotation in Exif headers #637 2020-12-12 13:05:58 +01:00
Michael Mayer 8a86fbc60f Indexer: Update index after renaming related sidecar files 2020-12-11 19:39:42 +01:00
Michael Mayer 424c0ce616 Indexer: Automatically rename related sidecar files 2020-12-11 19:17:07 +01:00
Michael Mayer 004400b118 Albums: Improve folder indexing 2020-12-09 13:10:21 +01:00
Michael Mayer ccdd048c47 Indexer: Ignore date in canonical file names
Signed-off-by: Michael Mayer <michael@liquidbytes.net>
2020-12-08 23:20:43 +01:00
Michael Mayer 846c635f22 Video: Transcoding to AVC1 #636 #513 #624 #603 #533 $440 2020-12-05 04:24:10 +01:00