ladybird/AK
DexesTTP e01f1c949f AK: Do not VERIFY on invalid code point bytes in UTF8View
The previous behavior was to always VERIFY that the UTF-8 bytes were
valid when iterating over the code points of an UTF8View. This change
makes it so we instead output the 0xFFFD 'REPLACEMENT CHARACTER'
code point when encountering invalid bytes, and keep iterating the
view after skipping one byte.

Leaving the decision to the consumer would break symmetry with the
UTF32View API, which would in turn require heavy refactoring and/or
code duplication in generic code such as the one found in
Gfx::Painter and the Shell.

To make it easier for the consumers to detect the original bytes, we
provide a new method on the iterator that returns a Span over the
data that has been decoded. This method is immediately used in the
TextNode::compute_text_for_rendering method, which previously did
this in a ad-hoc waay.

This also add tests for the new behavior in TestUtf8.cpp, as well
as reinforcements to the existing tests to check if the underlying
bytes match up with their expected values.
2021-06-03 18:28:27 +04:30
..
AllOf.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
AnyOf.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Array.h AK: Export integer_sequence_generate_array() 2021-05-05 19:02:51 +02:00
Assertions.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Atomic.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Badge.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Base64.cpp AK: Use calculate_base64_encoded_length in encode_base64 2021-05-22 08:54:32 +04:30
Base64.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
BinaryHeap.h AK+Userland: Use idan.horowitz@serenityos.org for my copyright headers 2021-04-22 22:42:38 +02:00
BinarySearch.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
BitCast.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Bitmap.h Bitmap: De-duplicate bitmasks 2021-05-19 23:37:10 +01:00
BitmapView.h Bitmap: De-duplicate bitmasks 2021-05-19 23:37:10 +01:00
BitStream.h AK+Userland: Use idan.horowitz@serenityos.org for my copyright headers 2021-04-22 22:42:38 +02:00
Buffered.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
ByteBuffer.h AK: Split the ByteBuffer::trim method into two methods 2021-05-31 14:49:00 +04:30
ByteReader.h LibCrypto+LibTLS: Avoid unaligned reads and writes 2021-05-14 08:39:29 +01:00
CharacterTypes.h AK: Add CharacterTypes.h 2021-06-03 13:31:46 +02:00
Checked.h AK: Make checked division also check for divide by zero 2021-06-02 16:09:16 +04:30
CheckedFormatString.h AK+Everywhere: Fix compiletime format parsing of replacement fields 2021-06-01 23:12:17 +04:30
CircularDeque.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
CircularDuplexStream.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
CircularQueue.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
CMakeLists.txt Tests: Move AK tests to Tests/AK 2021-05-06 17:54:28 +02:00
Complex.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Concepts.h AK+Kernel+LibELF: Remove the need for IteratorDecision::Continue 2021-05-16 10:36:52 +01:00
Debug.h.in AK: Add a new, spec-compliant URLParser 2021-06-01 09:28:05 +02:00
Demangle.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
DistinctNumeric.h AK: Make DistinctNumeric constexpr-capable 2021-05-04 21:32:15 +02:00
DoublyLinkedList.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Endian.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
EnumBits.h Everywhere: Use bgianf@serenityos.org for my copyright attribution 2021-04-22 21:15:54 +02:00
ExtraMathConstants.h Userland: Move non-standard math constants from math.h 2021-04-27 23:06:16 +02:00
FileStream.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Find.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
FlyString.cpp AK: Allow inlining more string functions 2021-06-03 08:06:51 +02:00
FlyString.h AK: Allow inlining more string functions 2021-06-03 08:06:51 +02:00
Format.cpp AK: Make vout() log to debug instead of VERIFY()'ing 2021-05-19 09:19:29 +02:00
Format.h AK: Add Formatter<unsigned char[Size]> 2021-05-31 17:43:54 +01:00
Forward.h AK: Turn ByteBuffer into a value type 2021-05-16 17:49:42 +02:00
Function.h AK: Add inline storage support for the Function class 2021-05-19 21:36:57 +02:00
GenericLexer.cpp AK/GenericLexer: constexpr where possible 2021-04-22 20:27:21 +02:00
GenericLexer.h AK/GenericLexer: constexpr where possible 2021-04-22 20:27:21 +02:00
HashFunctions.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
HashMap.h AK: Let HashMap export its key and value types 2021-05-18 18:48:15 +01:00
HashTable.h AK: Make HashTable::operator=(HashTable&&) clear the moved-from table 2021-05-30 14:34:32 +02:00
Hex.cpp Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Hex.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
IDAllocator.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
InlineLinkedList.h AK+Kernel+LibELF: Remove the need for IteratorDecision::Continue 2021-05-16 10:36:52 +01:00
IntrusiveList.h AK: Add reverse iterator support to AK::IntrusiveList 2021-06-03 13:27:40 +02:00
IntrusiveRedBlackTree.h Revert "AK: Remove virtual destructors from non-virtual classes" 2021-04-23 10:26:14 +02:00
IPv4Address.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
IterationDecision.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Iterator.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
JsonArray.h AK: Fix argument type for JsonArray::at and JsonArray::operator[] 2021-04-26 17:13:55 +02:00
JsonArraySerializer.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
JsonObject.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
JsonObjectSerializer.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
JsonParser.cpp AK: Use move semantics to avoid copying in JSON parser 2021-05-14 11:54:43 +02:00
JsonParser.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
JsonPath.cpp Everywhere: Use "the SerenityOS developers." in copyright headers 2021-04-29 00:59:26 +02:00
JsonPath.h Everywhere: Use "the SerenityOS developers." in copyright headers 2021-04-29 00:59:26 +02:00
JsonValue.cpp AK: Remove unused JsonValue <=> IPv4Address conversion code 2021-06-03 11:56:32 +01:00
JsonValue.h AK: Remove unused JsonValue <=> IPv4Address conversion code 2021-06-03 11:56:32 +01:00
kmalloc.h AK: Declare malloc_good_size as extern "C" 2021-05-30 11:10:47 +01:00
kstdio.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
LEB128.h AK: Handle LEB128 encoded values that are too large for the result type 2021-05-31 14:25:27 +04:30
LexicalPath.cpp LexicalPath: Reset dirname if it's empty 2021-05-27 18:21:36 +04:30
LexicalPath.h AK: Add LexicalPath::append and LexicalPath::join 2021-05-12 22:38:20 +02:00
MACAddress.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
MappedFile.cpp AK: Rename adopt() to adopt_ref() 2021-04-23 16:46:57 +02:00
MappedFile.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
MemMem.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Memory.h AK: Guard inline assembly with ARCH(I386) and provide alternative 2021-05-31 17:29:09 +01:00
MemoryStream.h Kernel+LibC: Make get_dir_entries syscall retriable 2021-05-12 12:50:23 +02:00
NeverDestroyed.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Noncopyable.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
NonnullOwnPtr.h AK+Kernel: Hide AK::adopt_own from usage in the Kernel 2021-05-29 09:04:05 +02:00
NonnullOwnPtrVector.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
NonnullPtrVector.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
NonnullRefPtr.h AK: Add missing GenericTraits<NonnullRefPtr> 2021-05-08 18:10:56 +02:00
NonnullRefPtrVector.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
NumberFormat.h Everywhere: Use "the SerenityOS developers." in copyright headers 2021-04-29 00:59:26 +02:00
NumericLimits.h AK/NumericLimits: Add lowest() for floating-point types 2021-05-22 13:38:34 +01:00
Optional.h AK: Enable direct comparsion of Optional<T> and T 2021-06-01 11:38:17 +02:00
OSError.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
OwnPtr.h AK+Kernel: Disallow implicitly lifting pointers to OwnPtr's 2021-05-31 17:09:12 +04:30
Platform.h AK: Add platform macros to detect presence of AddressSanitizer 2021-05-28 07:59:41 +02:00
PrintfImplementation.h AK: Honor variable precision argument when formatting 2021-05-30 18:45:06 +01:00
Ptr32.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Queue.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
QuickSort.h AK: Guarantee a maximum stack depth for dual_pivot_quick_sort 2021-04-28 21:38:48 +02:00
Random.cpp AK: Introduce get_random_uniform() 2021-05-14 22:24:02 +02:00
Random.h AK: Introduce get_random_uniform() 2021-05-14 22:24:02 +02:00
RedBlackTree.h Revert "AK: Remove virtual destructors from non-virtual classes" 2021-04-23 10:26:14 +02:00
RefCounted.h AK: Allow inlining ref-count functionality 2021-06-03 08:06:51 +02:00
RefPtr.h AK: Introduce adopt_ref_if_nonnull(..) to aid in Kernel OOM hardening 2021-05-13 08:29:01 +02:00
Result.h AK: Let Result<T, E> know its Value and Error types 2021-05-08 22:14:39 +02:00
ScopedValueRollback.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
ScopeGuard.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
ScopeLogger.h AK: Include String.h in ScopeLogger 2021-05-17 23:25:30 +02:00
SIMD.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Singleton.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
SinglyLinkedList.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
SinglyLinkedListWithCount.h Everywhere: Use bgianf@serenityos.org for my copyright attribution 2021-04-22 21:15:54 +02:00
SourceGenerator.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
SourceLocation.h AK+Userland: Use akaster@serenityos.org for my copyright headers 2021-05-30 14:35:34 +01:00
Span.h AK: Add a way to slice from the end of a span 2021-05-27 17:28:41 +04:30
Stack.h AK: Implement AK::Stack 2021-05-26 16:36:53 +04:30
StackInfo.cpp Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
StackInfo.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
StdLibExtraDetails.h AK: Define MakeSigned<...>::Type as void in the base struct 2021-05-26 15:34:13 +04:30
StdLibExtras.h AK: Extend round_to_power_of_two to types other than unsigned 2021-05-29 17:47:29 +01:00
Stream.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
String.cpp AK: Allow inlining more string functions 2021-06-03 08:06:51 +02:00
String.h AK: Allow inlining more string functions 2021-06-03 08:06:51 +02:00
StringBuilder.cpp AK: Remove the m_length member for StringBuilder 2021-05-31 14:49:00 +04:30
StringBuilder.h AK: Remove the m_length member for StringBuilder 2021-05-31 14:49:00 +04:30
StringHash.h AK: Make StringView::hash() constexpr 2021-05-14 15:24:32 +02:00
StringImpl.cpp AK: Include HashTable.h in StringImpl.cpp 2021-05-17 23:25:30 +02:00
StringImpl.h AK: Make StringView::hash() constexpr 2021-05-14 15:24:32 +02:00
StringUtils.cpp AK: Add trim() method to String, StringView and StringUtils 2021-06-01 09:28:05 +02:00
StringUtils.h AK: Add trim() method to String, StringView and StringUtils 2021-06-01 09:28:05 +02:00
StringView.cpp AK: Make StringView::hash() constexpr 2021-05-14 15:24:32 +02:00
StringView.h AK: Add trim() method to String, StringView and StringUtils 2021-06-01 09:28:05 +02:00
TemporaryChange.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Time.cpp Time: Remove static from function local constexpr variable 2021-05-19 16:24:02 +02:00
Time.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Traits.h AK: Add ConstPeekType to Traits 2021-05-08 18:10:56 +02:00
Trie.h Everywhere: Add missing includes for <AK/OwnPtr.h> 2021-05-19 21:36:57 +02:00
Tuple.h AK: Add deduction guides to Tuple 2021-05-18 18:48:15 +01:00
TypeCasts.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
TypedTransfer.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
TypeList.h Everywhere: "indexes" => "indices" 2021-04-29 22:23:52 +02:00
Types.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
UBSanitizer.h Kernel+AK: Move UBSanitizer to AK, and to AK namespace 2021-05-27 15:18:03 +02:00
UnicodeUtils.cpp AK: Add UnicodeUtils with Unicode-related helper functions 2021-05-20 22:10:45 +02:00
UnicodeUtils.h AK: Add UnicodeUtils with Unicode-related helper functions 2021-05-20 22:10:45 +02:00
URL.cpp Everywhere: Replace ctype.h to avoid narrowing conversions 2021-06-03 13:31:46 +02:00
URL.h AK: Move identity check from URL::operator==() to equals() 2021-06-01 12:23:16 +02:00
URLParser.cpp AK: Do not VERIFY on invalid code point bytes in UTF8View 2021-06-03 18:28:27 +04:30
URLParser.h AK: Add a new, spec-compliant URLParser 2021-06-01 09:28:05 +02:00
Userspace.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Utf8View.cpp AK: Do not VERIFY on invalid code point bytes in UTF8View 2021-06-03 18:28:27 +04:30
Utf8View.h AK: Do not VERIFY on invalid code point bytes in UTF8View 2021-06-03 18:28:27 +04:30
Utf32View.h AK: Rename Utf32CodepointIterator => Utf32CodePointIterator 2021-06-01 09:46:19 +02:00
UUID.cpp AK+Userland: Fix some compiler warnings and make variables const-ref 2021-05-16 17:49:42 +02:00
UUID.h Everything: Move to SPDX license identifiers in all files. 2021-04-22 11:22:27 +02:00
Variant.h AK+LibWasm+LibJS: Disallow Variant.has() on types that aren't contained 2021-06-02 18:02:47 +02:00
Vector.h AK/Vector: Constify find_first_index() 2021-05-22 09:34:55 +02:00
Weakable.h AK: Rename adopt() to adopt_ref() 2021-04-23 16:46:57 +02:00
WeakPtr.h AK: Rename adopt() to adopt_ref() 2021-04-23 16:46:57 +02:00