Skip to content

Rewrite ChunkedMemoryStream - #2828

Merged
JimBobSquarePants merged 11 commits into
release/3.1.xfrom
js/issue-2806
Nov 13, 2024
Merged

JimBobSquarePants merged 11 commits into
release/3.1.xfrom
js/issue-2806

Conversation

@JimBobSquarePants

@JimBobSquarePants JimBobSquarePants commented Oct 22, 2024 •

Copy link
Copy Markdown
Member

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
  • I have provided test coverage for my change (where applicable)

Description

Fixes #2806

ChunkedMemoryStream contained multiple writing bugs and was too costly to fix/maintain relative to performance benefits so I'm just ditching it.

  • Complete rewrite of ChunkedMemoryStream to simplify it and fix numerous bugs.
  • Ensure ImageEncoder uses the chunked stream when encoding non-seekable streams.

@antonfirsov

antonfirsov commented Oct 22, 2024 •

Copy link
Copy Markdown
Member

relative to performance benefits

IMO the memory benefits were significant. With the current decoder & encoder design, non seekable streams are always fully buffered into memory. A switch to MemoryStream will reintroduce significant GC allocations for large inputs all around the library, this will be a noticable regression for users who deal with http or other kinds of network streams.

Recommendations:

  • In case there is no bug in stream writing, do not remove ChunkedMemoryStream for decoders, since the largest streams are typically the input ones.
  • Instead of using MemoryStream for encoders, implement our own non-chunked MemoryStream that still uses MemoryAllocator. Although buffers over 4MB won't be pooled, in typical scenarios encoded files are smaller.

@JimBobSquarePants JimBobSquarePants changed the title Remove ChunkedMemoryStream Rewrite ChunkedMemoryStream Oct 23, 2024
@JimBobSquarePants

Copy link
Copy Markdown
Member Author

relative to performance benefits

IMO the memory benefits were significant. With the current decoder & encoder design, non seekable streams are always fully buffered into memory. A switch to MemoryStream will reintroduce significant GC allocations for large inputs all around the library, this will be a noticable regression for users who deal with http or other kinds of network streams.

Recommendations:

  • In case there is no bug in stream writing, do not remove ChunkedMemoryStream for decoders, since the largest streams are typically the input ones.
  • Instead of using MemoryStream for encoders, implement our own non-chunked MemoryStream that still uses MemoryAllocator. Although buffers over 4MB won't be pooled, in typical scenarios encoded files are smaller.

Thanks for the review @antonfirsov I've instead chosen to completely rewrite the ChunkedMemoryStream to simplify the implementation. It's much easier to maintain now!

public override void Flush()
{
_ = this.Read(this.singleByteBuffer, 0, 1);
return MemoryMarshal.GetReference<byte>(this.singleByteBuffer);

@mgravell mgravell Oct 30, 2024 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since the byte[] path goes via AsSpan(), why not use the span approach directly?

Span<byte> buffer = stackalloc byte[1];
return Read(buffer) == 1 ? buffer[0] : -1

ideally with SkipLocalsInit enabled

alt that elides a range check:

byte b = 0;
return Read(MemoryMarshal.CreateSpan(ref b, 1)) == 1 ? b : -1;

(you can't do this for the async path, though)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah... I'd completely forgotten about that method!

Rather than decorating a public mthod with [SkipLocalsInit] i've opted for the following:

/// <inheritdoc/>
public override int ReadByte()
{
    Unsafe.SkipInit(out byte b);
    return this.Read(MemoryMarshal.CreateSpan(ref b, 1)) == 1 ? b : -1;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks much better!

int offset = 0;
int count = buffer.Length;
while (count > 0)
while (bytesToRead != 0 && this.currentChunk != this.memoryChunkBuffer.Length)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know enough about the underlying implementation here; if this is doing additional downstream reads, you might prefer to exit after the first read (you're only required to read "some" data - you don't need to fill the supplied data, just return at least 1 byte or 0 for EOF); if all the data is already loaded, I wonder whether your memoryChunkBuffer is duplicating the innards of ReadOnlySequence<byte> - that already has all the Slice, CopyTo etc you might want; just a suggestion, though (I can help you grok ReadOnlySequence<T> if you're not already familiar)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, looking at MemoryChunkBuffer there is definitely some overlap. Ideally, I should be tracking the buffer and chunk indexes internally within that class.

However, this type needs to be expandable on-demand which AFAIK is not possible with ReadonlySequence<T>.

@mgravell mgravell Oct 31, 2024 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The underlying buffer-chain is as mutable as you want it to be (it is your chain, ultimately); if you want to resize it, that is usually as simple as simply creating a new ROS (which is a lightweight struct just tracking the start and end), specifying the new bounds. The chain bits aren't trivial, but not too complex. I guess if what you already have works well, it might be overkill to touch it, though.

}

return chunkBuffer.GetSpan()[this.readOffset++];
MemoryMarshal.Write(this.singleByteBuffer, ref value);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto stackalloc; or possibly even the more exotic:

byte b = 0;
var span = MemoryMarshal.CreateSpan(ref b, 1);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/// <inheritdoc/>
public override void WriteByte(byte value)
    => this.Write(MemoryMarshal.CreateSpan(ref value, 1));

/// Writes the stream contents to a byte array, regardless of the <see cref="Position"/> property.
/// </summary>
/// <returns>The <see cref="T:byte[]"/>.</returns>
/// <returns>A new <see cref="T:byte[]"/>.</returns>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this is very inefficient; I would suggest trying to deprecate this kind of API - especially if we can use ROS, for example;

[Obsolete("prefer " + nameof(AsReadOnlySequence)] public byte[] ToArray() => AsReadOnlySequence().ToArray();
public ReadOnlySequence<byte> AsReadOnlySequence() => /* magic happens */

@JimBobSquarePants JimBobSquarePants Oct 31, 2024 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, we need this to when reading XMP data for the V3 build however I wish to rewrite the XMPProfile type for V4 to avoid passing arrays around. Once this is merged to the V3 branch I'll upstream and make additional changes.

@antonfirsov

Copy link
Copy Markdown
Member

@JimBobSquarePants I believe I will have some time to also review this in the weekend.

/// and is designed to take advantage of buffer pooling when available.
/// </summary>
internal sealed class ChunkedMemoryStream : Stream
public class ChunkedMemoryStream : Stream

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider reverting this back to being sealed. I.e. is this class designed to be sub-classed? If not then sealing can allow the JITter to make certain optimisations around the method calling of the virtual/override methods

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's supposed to be internal sealed actually! I forgot to change it back after rewriting (public makes the IDE tell me to add method docs).

private readonly int allocatorCapacity;

// Has the stream been disposed.
private long length;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sidenote: IMO it's fairly conventional in C# land to prefix private field names with an underscore, to allow easy distinction from local variables, and to avoid excessive use of this.. I assume this is your personal preference, but thought I'd mention it as IMO it is somewhat non-idiomatic C#.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, but I'd rather stick to using the language as designed rather than using conventions carried over from C.

The this keyword provides important context IMO and encouraged consistancy throughout a codebase.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colgreen the project's preferred coding style is based on Framework design guidelines, and on StyleCop recommendations. The guidelines explicitly prohibit prefixing variables.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guidelines state Internal and private fields are not covered by guidelines, (I believe those guidelines are primarily related to public API surface, rather than private/internal naming). The underscore prefix for private fields is very common in my experience, e.g. it's used widely in Microsoft .NET repos.

However, this topic is probably not relevant in the context of this PR :)

// Has the stream been disposed.
private long length;
private long position;
private int currentChunk;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These names could be considered a little misleading/confusing.

E.g. currentChuck is an index into memoryChunkBuffer, so I think maybe call it currentChunkIdx.

Whereas currentChunkIdx is an offset/index /within/ the current chunk, so maybe call it intraChunkByteIdx, chunkByteIdx, or checkByteOffset? etc.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah.. good point. I've opted for bufferIndex and chunkIndex as an improvement.

chunk.Dispose();
chunk = chunk.Next;
this.Dispose(true);
GC.SuppressFinalize(this);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not necessary to call GC.SuppressFinalize(this) in a sealed class with no finalizer. This would be to cover sub-types that have a finalizer (in scenarios where there is no finalizer defined directly on the type).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Force of habit. Well spotted!

@antonfirsov antonfirsov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Previous reviewers had good points.
  • Added some nitpicking.
  • Improving funcitonal coverage would be valuable.

Looks good otherwise.

return i < 16 ? b128K * (1 << (int)((uint)i / 4)) : b4M;
}

private void Dispose(bool disposing)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given https://github.com/SixLabors/ImageSharp/pull/2828/files#r1823357071, the Dispose(bool disposing) is not even needed.

Same for MemoryChunk.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's an override of the base Stream method though. I can't implement Dispose directly. All other implementations have been simplified now though.

Comment on lines +216 to +221
if (remaining > count)
{
remaining = count;
}

Span<byte> chunkBuffer = this.writeChunk.Buffer.GetSpan();
int chunkSize = this.writeChunk.Length;
int count = buffer.Length;
int offset = 0;
while (count > 0)
int bytesToWrite = (int)remaining;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remaining is not being used after this line.

-        if (remaining > count)
-        {
-            remaining = count;
-        }

-        int bytesToWrite = (int)remaining;
+        int bytesToWrite = count;

Same for Read.

Comment on lines +446 to +450
public IEnumerator<MemoryChunk> GetEnumerator()
=> ((IEnumerable<MemoryChunk>)this.memoryChunks).GetEnumerator();

IEnumerator IEnumerable.GetEnumerator()
=> ((IEnumerable)this.memoryChunks).GetEnumerator();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any code enumerating this with foreach, so IEnumerable implementation can be deleted.

private readonly int allocatorCapacity;

// Has the stream been disposed.
private long length;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colgreen the project's preferred coding style is based on Framework design guidelines, and on StyleCop recommendations. The guidelines explicitly prohibit prefixing variables.

@@ -30,7 +30,7 @@ public class ChunkedMemoryStreamTests
[Fact]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to extend these tests to stress the corner(?) cases which were buggy in the previous implementation.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Tests have been massivly exapanded. We now test reading larger buffers and test encoding to webp for all test images.

Comment on lines 107 to 108
[InlineData(DefaultSmallChunkSize * 16)]
public void MemoryStream_ReadByteBufferSpanTest(int length)

@antonfirsov antonfirsov Nov 3, 2024 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also include lengths over DefaultSmallChunkSize * 16 and make buffer.Length a parameter. Would test cases, when buffer.Length > DefaultSmallChunkSize.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

length is already determined by the parameter but I've expanded to double the previous maximum length

@@ -167,18 +167,20 @@ public void MemoryStream_WriteToTests()
[Fact]
public void MemoryStream_WriteToSpanTests()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Write tests could stress more cases with different sizes. See my comment on the read tests.

int offset = 0;
int count = buffer.Length;
while (count > 0)
while (bytesToRead != 0 && this.currentChunk != this.memoryChunkBuffer.Length)

@antonfirsov antonfirsov Nov 3, 2024 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though current code looks good and Slice would throw for a negative number, I find this safer to maintain. Same for Write.

Suggested change
while (bytesToRead != 0 && this.currentChunk != this.memoryChunkBuffer.Length)
while (bytesToRead > 0 && this.currentChunk != this.memoryChunkBuffer.Length)

@JimBobSquarePants

Copy link
Copy Markdown
Member Author

@antonfirsov Are you happy with me pushing on with this?

@antonfirsov antonfirsov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. One small suggestion for the test code.

Comment thread tests/ImageSharp.Tests/IO/ChunkedMemoryStreamTests.cs Outdated
xadupre pushed a commit to microsoft/onnxruntime that referenced this pull request Oct 8, 2026
Updated [SixLabors.ImageSharp](https://github.com/SixLabors/ImageSharp)
from 2.1.9 to 4.1.2.

<details>
<summary>Release notes</summary>

_Sourced from [SixLabors.ImageSharp's
releases](https://github.com/SixLabors/ImageSharp/releases)._

## 4.1.2

## What's Changed
* Harden codec and metadata input validation by @​JimBobSquarePants in
SixLabors/ImageSharp#3187
* Fix TIFF decoding of legacy 8-bit color maps by @​JimBobSquarePants in
SixLabors/ImageSharp#3189


**Full Changelog**:
SixLabors/ImageSharp@v4.1.1...v4.1.2

## 4.1.1

## What's Changed
* Fix CCITT decompressor bounds checks and error handling by
@​JimBobSquarePants in SixLabors/ImageSharp#3176


**Full Changelog**:
SixLabors/ImageSharp@v4.1.0...v4.1.1

## 4.1.0

## What's Changed
* GIF: background handling & quantizer overflow fix by
@​JimBobSquarePants in SixLabors/ImageSharp#3133
* Validate PBM max pixel value by @​JimBobSquarePants in
SixLabors/ImageSharp#3134
* Add support for Apple CgBI PNG images by @​Erik-White in
SixLabors/ImageSharp#3136
* Fix GIF transparency handling and dither by @​JimBobSquarePants in
SixLabors/ImageSharp#3143
* Bump codecov/codecov-action from 6 to 7 by @​dependabot[bot] in
SixLabors/ImageSharp#3141
* Add BlendWithCoverage overloads. Optimize Rgba32 compatible shuffling.
by @​JimBobSquarePants in
SixLabors/ImageSharp#3150
* Bump actions/checkout from 6 to 7 by @​dependabot[bot] in
SixLabors/ImageSharp#3146
* Bump actions/cache from 5 to 6 by @​dependabot[bot] in
SixLabors/ImageSharp#3149
* Fix convolution sampling for bounds smaller than the kernel radius by
@​JimBobSquarePants in SixLabors/ImageSharp#3152
* Enhance pixel formats and associated-alpha operations with
optimizations by @​JimBobSquarePants in
SixLabors/ImageSharp#3154
* Bump actions/setup-dotnet from 5 to 6 by @​dependabot[bot] in
SixLabors/ImageSharp#3155
* Fix degeneracy check for resize transforms by @​JimBobSquarePants in
SixLabors/ImageSharp#3157
* Improvements to Apple CgBI PNG handling by @​Erik-White in
SixLabors/ImageSharp#3137
* Clone TIFF profiles into image metadata by @​JimBobSquarePants in
SixLabors/ImageSharp#3159
* Add AOT compiler seeds for pixel formats and ICO, CUR, QOI codecs by
@​JimBobSquarePants in SixLabors/ImageSharp#3160
* Normalize SIMD pipelines using TensorPrimitives_ by
@​JimBobSquarePants in SixLabors/ImageSharp#3161
* Add ANI decoder support by @​Poker-sang in
SixLabors/ImageSharp#2899
* Remove AntialiasThreshold from GraphicsOptions by @​JimBobSquarePants
in SixLabors/ImageSharp#3162
* Add plus and extended pixel blend modes by @​JimBobSquarePants in
SixLabors/ImageSharp#3163
* Fix 16-bit binary PBM sample byte order by @​JimBobSquarePants in
SixLabors/ImageSharp#3164
* Make MemoryAllocator limits configurable and extensible by
@​JimBobSquarePants in SixLabors/ImageSharp#3165

## New Contributors
* @​Erik-White made their first contribution in
SixLabors/ImageSharp#3136

**Full Changelog**:
SixLabors/ImageSharp@v4.0.0...v4.1.0

## 4.0.0

## What's Changed
* Update to net8 by @​stefannikolei in
SixLabors/ImageSharp#2583
* Handle dedup of local palette of 256 length - Main by
@​JimBobSquarePants in SixLabors/ImageSharp#2607
* Replace custom Crc32 by @​JimBobSquarePants in
SixLabors/ImageSharp#2611
* Sync 3.1 DrawImage fixes by @​tocsoft in
SixLabors/ImageSharp#2612
* Fix handling gif encoding for global palettes - Main by
@​JimBobSquarePants in SixLabors/ImageSharp#2615
* Bump actions/setup-dotnet from 3 to 4 by @​dependabot[bot] in
SixLabors/ImageSharp#2613
* Adjusted the casing of the Webp format name by @​jscarle in
SixLabors/ImageSharp#2623
* Fix Paeth Filter decode on platforms that do not support Ssse3 - Main
by @​JimBobSquarePants in
SixLabors/ImageSharp#2620
* Fix WebP animation speed bug by @​marklagendijk in
SixLabors/ImageSharp#2624
* Promote PixelTypeInfo to Pixel by @​stefannikolei in
SixLabors/ImageSharp#2601
* TGA: Treat 32 bit True Color images always as transparent by
@​brianpopow in SixLabors/ImageSharp#2643
* Modernize and optimize pixel format operations across platforms. by
@​JimBobSquarePants in SixLabors/ImageSharp#2645
* Cleanup SimdUtils by @​JimBobSquarePants in
SixLabors/ImageSharp#2654
* Bump actions/cache from 3 to 4 by @​dependabot[bot] in
SixLabors/ImageSharp#2648
* Bump codecov/codecov-action from 3 to 4 by @​dependabot[bot] in
SixLabors/ImageSharp#2657
* Bump NuGet/setup-nuget from 1 to 2 by @​dependabot[bot] in
SixLabors/ImageSharp#2658
* Add v3.1.x fixes #​2673 and #​2674 into main. by @​JimBobSquarePants
in SixLabors/ImageSharp#2675
* Add fixes 2668, 2676, and 2677 to main by @​JimBobSquarePants in
SixLabors/ImageSharp#2678
* Merge 2681 to v4 Main by @​JimBobSquarePants in
SixLabors/ImageSharp#2690
* Add JPEG COM marker support by @​RobertMut in
SixLabors/ImageSharp#2641
* Bump actions/upload-artifact from 3 to 4 by @​dependabot[bot] in
SixLabors/ImageSharp#2625
* Only exit JPEG scan decoding after multiple EOF hits by
@​JimBobSquarePants in SixLabors/ImageSharp#2701
* V4 Ensure VP8X alpha flag is updated correctly. by @​JimBobSquarePants
in SixLabors/ImageSharp#2703
* Fix animated png handling (issue #​2708) by @​SpaceCheetah in
SixLabors/ImageSharp#2710
* Merge latest release from v3 by @​JimBobSquarePants in
SixLabors/ImageSharp#2720
* Fix MacOS jobs by @​antonfirsov in
SixLabors/ImageSharp#2728
* Fix async-over-sync issue in Image.DecodeAsync() by @​kroymann in
SixLabors/ImageSharp#2725
* Fix overflow in MemoryAllocator.Create(options) by @​antonfirsov in
SixLabors/ImageSharp#2730
* GifDecoder: Limit lzw bits to a maximum of 12 bits by @​brianpopow in
SixLabors/ImageSharp#2744
* GifDecoder : Allow skipping bad metadata using identify by
@​JimBobSquarePants in SixLabors/ImageSharp#2749
* Add ICO and CUR file decoder. by @​frg2089 in
SixLabors/ImageSharp#2579
* v4 - Fix off-by-one error when centering a transform. by
@​JimBobSquarePants in SixLabors/ImageSharp#2761
* v4 Fix 2758 by @​JimBobSquarePants in
SixLabors/ImageSharp#2764
* Simplify Color Space Conversion APIs by @​JimBobSquarePants in
SixLabors/ImageSharp#2739
* Webp: Fix Issue 2763 by @​brianpopow in
SixLabors/ImageSharp#2767
* V4 Correctly break during Png decoding by @​JimBobSquarePants in
SixLabors/ImageSharp#2773
* V4 : Fix filtering on PNG encode. by @​JimBobSquarePants in
SixLabors/ImageSharp#2778
* Fix #​2779 buffer overrun by @​KirillAldashkin in
SixLabors/ImageSharp#2780
* Fix ImageMetadata docs typo by @​lofcz in
SixLabors/ImageSharp#2781
* Add API for metadata conversion between formats. by
@​JimBobSquarePants in SixLabors/ImageSharp#2751
* Tiff decoder: Fix issue 2679 by @​brianpopow in
SixLabors/ImageSharp#2789
* Replace PngCrcChunkHandling by @​JimBobSquarePants in
SixLabors/ImageSharp#2786
* Add tagname to debugger visualization for Exif- and Iptc-values, to
facilitate easier debugging and discovery by @​lassevk in
SixLabors/ImageSharp#2787
* V4 - Correctly handle transform spaces when building transform
matrices. by @​JimBobSquarePants in
SixLabors/ImageSharp#2795
* Allow decoding Tiff of different frame size. by @​JimBobSquarePants in
SixLabors/ImageSharp#2788
* Add progressive JPEG encoder by @​ardabada in
SixLabors/ImageSharp#2740
* Fix using dither in BmpEncoder when bit per pixel is <= 4 by @​mistoll
in SixLabors/ImageSharp#2819
* Add QuadDistortion to ProjectiveTransformBuilder by @​Socolin in
SixLabors/ImageSharp#2748
* WEBP : Use Correct Width With AlphaDecoder by @​JimBobSquarePants in
SixLabors/ImageSharp#2823
 ... (truncated)

## 3.1.12

## What's Changed
* v3 : Add Full Metadata Parsing for WEBP Animations by
@​JimBobSquarePants in SixLabors/ImageSharp#3002
* V3: Prevent negative allocation attempt for huge TIFF files by
@​JimBobSquarePants in SixLabors/ImageSharp#3004
* Backport v3 : Tiff decoder: Fix issue 2679 by @​JimBobSquarePants in
SixLabors/ImageSharp#3007


**Full Changelog**:
SixLabors/ImageSharp@v3.1.11...v3.1.12

## 3.1.11

## What's Changed
* Backport V3 - GIF: Check for end of stream when reading comments. by
@​JimBobSquarePants in SixLabors/ImageSharp#2955


**Full Changelog**:
SixLabors/ImageSharp@v3.1.10...v3.1.11

## 3.1.10

## What's Changed
* V3: Remove ExifTagValue.InteroperabilityIndex by @​JimBobSquarePants
in SixLabors/ImageSharp#2939
* V3 TIFF : Allow additional and undefined extra samples by
@​JimBobSquarePants in SixLabors/ImageSharp#2942
* V3 Use EXIF byte order for EXIF encoded strings. by
@​JimBobSquarePants in SixLabors/ImageSharp#2944
* V3 Backport : Reduce the number of memory allocations in lossless WebP
encoder by @​JimBobSquarePants in
SixLabors/ImageSharp#2947


**Full Changelog**:
SixLabors/ImageSharp@v3.1.9...v3.1.10

## 3.1.9

## What's Changed
* V3: Do not attempt to decode iDAT chunks when image is fully decoded.
by @​JimBobSquarePants in
SixLabors/ImageSharp#2927
* V3 - Do not throw exception for non-required chunks. by
@​JimBobSquarePants in SixLabors/ImageSharp#2929
* V3: Do not encode WEBP images exceeding max dimensions by
@​JimBobSquarePants in SixLabors/ImageSharp#2931


**Full Changelog**:
SixLabors/ImageSharp@v3.1.8...v3.1.9

## 3.1.8

## What's Changed
* V3 : Fix GIF, PNG, and WEBP Edge Case Handling by @​JimBobSquarePants
in SixLabors/ImageSharp#2882
* V3 Nested sub ifds parsing fix by @​JimBobSquarePants in
SixLabors/ImageSharp#2913
* V3: Undo horizontal prediction for each tile row in case of tiled
tiff's by @​JimBobSquarePants in
SixLabors/ImageSharp#2914
* V3 - Use BPP from Color Writer for Compressor by @​JimBobSquarePants
in SixLabors/ImageSharp#2915


**Full Changelog**:
SixLabors/ImageSharp@v3.1.7...v3.1.8

## 3.1.7

## What's Changed
* V3 : Decode Chunky Tile Rows Directly. by @​JimBobSquarePants in
SixLabors/ImageSharp#2874
* V3: Gracefully handle LZW overflows by @​JimBobSquarePants in
SixLabors/ImageSharp#2880
* [3.1] Set LangVersion=12 unconditionally by @​antonfirsov in
SixLabors/ImageSharp#2896


**Full Changelog**:
SixLabors/ImageSharp@v3.1.6...v3.1.7

## 3.1.6

## What's Changed
* Mono AOT decoder workaround for slow jpeg decoding. by
@​JimBobSquarePants in SixLabors/ImageSharp#2762
* V3 - Correctly handle transform spaces when building transform
matrices. by @​JimBobSquarePants in
SixLabors/ImageSharp#2791
* Backport : V3 Fix #​2779 buffer overrun by @​JimBobSquarePants in
SixLabors/ImageSharp#2824
* Backport - v3 WEBP : Use Correct Width With AlphaDecoder by
@​JimBobSquarePants in SixLabors/ImageSharp#2827
* Rewrite ChunkedMemoryStream by @​JimBobSquarePants in
SixLabors/ImageSharp#2828


**Full Changelog**:
SixLabors/ImageSharp@v3.1.5...v3.1.6

## 3.1.5

## What's Changed
* [3.1] Fix overflow in MemoryAllocator.Create(options) by @​antonfirsov
in SixLabors/ImageSharp#2731
* v3 - Backport. Handle out of bounds Gif LZW max code by
@​JimBobSquarePants in SixLabors/ImageSharp#2754
* Fix off-by-one error when centering a transform. by
@​JimBobSquarePants in SixLabors/ImageSharp#2760
* v3.1.x Fix 2758 by @​JimBobSquarePants in
SixLabors/ImageSharp#2759
* v3.1.x - Backport Webp: Fix Issue 2763/2692 by @​JimBobSquarePants in
SixLabors/ImageSharp#2768
* Correctly break during Png decoding by @​JimBobSquarePants in
SixLabors/ImageSharp#2769
* Fix filtering on PNG encode. by @​JimBobSquarePants in
SixLabors/ImageSharp#2775


**Full Changelog**:
SixLabors/ImageSharp@v3.1.4...v3.1.5

## 3.1.4

## What's Changed
* Backport - Only exit JPEG scan decoding after multiple EOF hits by
@​JimBobSquarePants in SixLabors/ImageSharp#2702
* Ensure VP8X alpha flag is updated correctly. by @​JimBobSquarePants in
SixLabors/ImageSharp#2699
* Backport APNG fix to release/3.1.x by @​SpaceCheetah in
SixLabors/ImageSharp#2713
* Limit Read Palette Indices by @​JimBobSquarePants in
SixLabors/ImageSharp#2718
* Limit all memory allocations in the MemoryAllocator layer by
@​antonfirsov in SixLabors/ImageSharp#2706
* Clear Pixel Buffers on Decode. by @​JimBobSquarePants in
SixLabors/ImageSharp#2716


**Full Changelog**:
SixLabors/ImageSharp@v3.1.3...v3.1.4

## 3.1.3

## What's Changed
* Make DrawImage processor more robust to bad input. by
@​JimBobSquarePants in SixLabors/ImageSharp#2673
* Don't skip reading data for animation chunks on identify. by
@​JimBobSquarePants in SixLabors/ImageSharp#2674
* Fix AccessViolationException and hard crash with animated webp by
@​JimBobSquarePants in SixLabors/ImageSharp#2676
* Use a smarter approach to determine the transparent index when
encoding palette Pngs. by @​JimBobSquarePants in
SixLabors/ImageSharp#2677
* Limit ancillary PNG chunk size. by @​JimBobSquarePants in
SixLabors/ImageSharp#2681


**Full Changelog**:
SixLabors/ImageSharp@v3.1.2...v3.1.3

## 3.1.2

## What's Changed
* Fix Paeth Filter decode on platforms that do not support Ssse3 by
@​JimBobSquarePants in SixLabors/ImageSharp#2618
* Backport: Fix WebP animation speed bug by @​JimBobSquarePants in
SixLabors/ImageSharp#2637


**Full Changelog**:
SixLabors/ImageSharp@v3.1.1...v3.1.2

## 3.1.1

## What's Changed
* Handle dedup of local palette of 256 length by @​JimBobSquarePants in
SixLabors/ImageSharp#2605
* Correctly calculate foreground Rectangle when negative background
location is set (3.1 fix) by @​tocsoft in
SixLabors/ImageSharp#2609
* Handle case when Foreground image overhangs bottom of background image
during DrawImage Call (3.1 target) by @​tocsoft in
SixLabors/ImageSharp#2610
* Fix handling gif encoding for global palettes. by @​JimBobSquarePants
in SixLabors/ImageSharp#2614


**Full Changelog**:
SixLabors/ImageSharp@v3.1.0...v3.1.1

## 3.1.0

## What's Changed
* Reduced intermediate allocations by @​gfoidl in
SixLabors/ImageSharp#2415
* Port colorconverter YCbCr and YCCk to arm by @​stefannikolei in
SixLabors/ImageSharp#2417
* Added and used Numerics.Vector{128|256}Count extension by @​gfoidl in
SixLabors/ImageSharp#2422
* Improvements around `fixed` by @​gfoidl in
SixLabors/ImageSharp#2418
* Create vector constants inline and not via ROS<byte> by @​gfoidl in
SixLabors/ImageSharp#2419
* More tests for Webp lossless utils by @​brianpopow in
SixLabors/ImageSharp#2424
* Fix incorrect comments by @​hey-red in
SixLabors/ImageSharp#2426
* Fixed invalid jpeg buffer width compliment for scalar color converters
by @​br3aker in SixLabors/ImageSharp#2427
* Update Readme by @​brianpopow in
SixLabors/ImageSharp#2433
* Add AdvSimd in ComponentProcessor by @​stefannikolei in
SixLabors/ImageSharp#2429
* Fix decoding tiled tiff with unequal tile width and height by
@​brianpopow in SixLabors/ImageSharp#2436
* Fix Vp8Residual costs calculation by @​brianpopow in
SixLabors/ImageSharp#2432
* Fix thresholds for Resize kernel tests by @​brianpopow in
SixLabors/ImageSharp#2440
* Tiff: Undo horizontal predictor for cmyk images by @​brianpopow in
SixLabors/ImageSharp#2459
* fix a typo in XML comment for struct `Rgb48` by @​n0099 in
SixLabors/ImageSharp#2463
* Fix #​2467 bmp encoding issue for BMP with 1 bit per pixel and more
pixels per row than divisible by 8. by @​synercoder in
SixLabors/ImageSharp#2471
* Update Directory.Build.targets by @​Pashkinz92 in
SixLabors/ImageSharp#2468
* bugfix in Rational.cs, when both Nominator and Denominator equals to 0
by @​tsardaryanCamenAI in
SixLabors/ImageSharp#2453
* Fix DrawImage offsetting issues and improve API parameter names. by
@​JimBobSquarePants in SixLabors/ImageSharp#2474
* Use more accurate bit for alpha lookup. by @​JimBobSquarePants in
SixLabors/ImageSharp#2473
* Fix for B&W pbm images with width not dividable by 8 by @​ynse01 in
SixLabors/ImageSharp#2481
* Tiff: ccitt compression, fix for issue #​2451 by @​brianpopow in
SixLabors/ImageSharp#2483
* Use `FileOptions.Asynchronous` when doing async IO by @​Neme12 in
SixLabors/ImageSharp#2488
* Allow JFXX as an App0 marker header by @​JimBobSquarePants in
SixLabors/ImageSharp#2482
* Adding QOI support by @​LuisAlfredo92 in
SixLabors/ImageSharp#2446
* Vectorize TrimTransparentPixels in GifEncoderCore by @​gfoidl in
SixLabors/ImageSharp#2500
* Prevent crafted DOS attack. by @​JimBobSquarePants in
SixLabors/ImageSharp#2501
* Reverse automatic include behavior. by @​JimBobSquarePants in
SixLabors/ImageSharp#2515
* Handle EOF in Jpeg bit reader when data is bad to prevent DOS attack.…
by @​JimBobSquarePants in
SixLabors/ImageSharp#2523
* Fix #​2518 by @​antonfirsov in
SixLabors/ImageSharp#2519
* Fix for issue 2504 - IPTC and ICC profile information being lost
during TIFF file save by @​JeffP134 in
SixLabors/ImageSharp#2535
* Preserve Gif color palettes and deduplicate frame pixels. by
@​JimBobSquarePants in SixLabors/ImageSharp#2455
* Bump actions/checkout from 3 to 4 by @​dependabot in
SixLabors/ImageSharp#2530
* Expose and conserve the color palette for indexed png images. by
@​JimBobSquarePants in SixLabors/ImageSharp#2485
* Change ExifTag TimeZoneOffset to Signed Short by @​Cheesebaron in
SixLabors/ImageSharp#2541
* Expose non-nullable configuration to remove AOT limiting null check by
@​JimBobSquarePants in SixLabors/ImageSharp#2514
* Add GPSHPositioningError exif tag by @​Cheesebaron in
SixLabors/ImageSharp#2543
* Disallow allocation attempts of unrepresentable sizes by @​antonfirsov
in SixLabors/ImageSharp#2545
* Add webp decoder option to handle the background color in ANIM chunk
by @​brianpopow in SixLabors/ImageSharp#2547
* Tiff decoding robustness improvements by @​JimBobSquarePants in
SixLabors/ImageSharp#2550
* PBM decoder robustness improvements and BufferedReadStream
observability by @​antonfirsov in
SixLabors/ImageSharp#2551
* Follow up on post-merge discussions in #​2551 by @​antonfirsov in
SixLabors/ImageSharp#2552
* APNG support by @​Poker-sang in
SixLabors/ImageSharp#2511
* WebP - Reduce the allocations in lossless encoding by
@​JimBobSquarePants in SixLabors/ImageSharp#2546
* Animated webp encoder by @​Poker-sang in
SixLabors/ImageSharp#2569
* Fix boxed quantization and update refs by @​JimBobSquarePants in
SixLabors/ImageSharp#2582
* Explicitly set BitsPerPixel and BitsPerSample to 1 for bicolor
compressed images by @​brianpopow in
SixLabors/ImageSharp#2590
* [fix]: Fixed Unknown App0 Marker add profileResolver missing by
@​JoseEliasSantos in SixLabors/ImageSharp#2566
* Png- Do not attempt to read data for chunks of length 0. by
@​JimBobSquarePants in SixLabors/ImageSharp#2561
 ... (truncated)

## 3.0.2

## What's Changed
* Handle EOF in Jpeg bit reader when data is bad to prevent DOS attack.
by @​JimBobSquarePants in
SixLabors/ImageSharp#2516


**Full Changelog**:
SixLabors/ImageSharp@v3.0.1...v3.0.2

## 3.0.1

## What's Changed
* ISSUE #​2385 - Implementing try get pattern to
TryFindFormatByFileExtension(string extension, [NotNullWhen(true)] out
IImageFormat? format) by @​Ollie-Ave in
SixLabors/ImageSharp#2386
* Remove mac specific skips by @​stefannikolei in
SixLabors/ImageSharp#2384
* Add Arm intrinsics to JpegColorConverter RGB by @​stefannikolei in
SixLabors/ImageSharp#2397
* Add arm64 intrinsics for cmyk converter by @​stefannikolei in
SixLabors/ImageSharp#2400
* Low-hanging fruit codegen optimizations by @​gfoidl in
SixLabors/ImageSharp#2401
* Webp: Dont use using statement for encodedAlphaData by @​brianpopow in
SixLabors/ImageSharp#2412
* Fixed wrong division hack by @​gfoidl in
SixLabors/ImageSharp#2413
* Port GrayscalConverter to Arm by @​stefannikolei in
SixLabors/ImageSharp#2409
* Fix RgbScalar by @​stefannikolei in
SixLabors/ImageSharp#2416

## New Contributors
* @​Ollie-Ave made their first contribution in
SixLabors/ImageSharp#2386

**Full Changelog**:
SixLabors/ImageSharp@v3.0.0...v3.0.1

## 3.0.0

## What's Changed
* Use .NET 6 only by @​JimBobSquarePants in
SixLabors/ImageSharp#2098
* Convolution: Border wrapping modes by @​ynse01 in
SixLabors/ImageSharp#2060
* Use ReadOnlySpan<byte> static data compiler optimization in more
places by @​turbedi in SixLabors/ImageSharp#2101
* Add support for decoding jpeg's with arithmetic coding by @​brianpopow
in SixLabors/ImageSharp#2073
* Replace Array.Clear(array, 0, array.Length) calls with
Array.Clear(array) by @​turbedi in
SixLabors/ImageSharp#2103
* Preserve color profile when encoding bitmaps by @​brianpopow in
SixLabors/ImageSharp#2108
* Preserve color profile when encoding webp images by @​brianpopow in
SixLabors/ImageSharp#2109
* Preserve color profile when encoding PNG images by @​brianpopow in
SixLabors/ImageSharp#2110
* Add support for decoding webp images with animations by @​brianpopow
in SixLabors/ImageSharp#1985
* Jpeg compressed tiff: jpeg decoder should handle the conversion from
YCbCr to RGB by @​brianpopow in
SixLabors/ImageSharp#2124
* Deduce color space as YCbCr if component id's are 1, 2, 3 by
@​brianpopow in SixLabors/ImageSharp#2135
* Bump actions/checkout from 2 to 3 by @​dependabot in
SixLabors/ImageSharp#2145
* Bump actions/setup-dotnet from 1 to 2 by @​dependabot in
SixLabors/ImageSharp#2143
* Bump actions/cache from 2 to 3 by @​dependabot in
SixLabors/ImageSharp#2142
* Bump actions/upload-artifact from 2 to 3 by @​dependabot in
SixLabors/ImageSharp#2146
* Bump codecov/codecov-action from 1 to 3 by @​dependabot in
SixLabors/ImageSharp#2144
* Skip invalid markers during jpeg decoding by @​br3aker in
SixLabors/ImageSharp#2147
* Tiff: Performance improvements for Fax4 decompression by @​brianpopow
in SixLabors/ImageSharp#2134
* Fix regression in TIF image loading by @​brianpopow in
SixLabors/ImageSharp#2150
* Used inline SIMD vectors if they are constants by @​gfoidl in
SixLabors/ImageSharp#2122
* Print environment info on tests start by @​JimBobSquarePants in
SixLabors/ImageSharp#2175
* Update License by @​JimBobSquarePants in
SixLabors/ImageSharp#2176
* Jpeg downscaling decoding by @​br3aker in
SixLabors/ImageSharp#2076
* Add support for decoding tiff images with CieLab color space by
@​brianpopow in SixLabors/ImageSharp#2127
* Assembly Trimming Support by @​ds5678 in
SixLabors/ImageSharp#2160
* Jpeg color space deduction fix by @​br3aker in
SixLabors/ImageSharp#2177
* Add support for decoding tiff with webp compressed data by
@​brianpopow in SixLabors/ImageSharp#2128
* Fix issue when writing lossy webp with alpha data by @​brianpopow in
SixLabors/ImageSharp#2173
* Fix run-length encode compression implementation mistake in tga
encoder by @​l1nxy in SixLabors/ImageSharp#2172
* TGA Encoder/Decoder Improvements by @​brianpopow in
SixLabors/ImageSharp#2197
* Tiff: Make sure 1 Bit compression is only used with 1 bit pixel type
by @​brianpopow in SixLabors/ImageSharp#2201
* Refactor `EnumUtils` and make it AOT-friendly. by @​teo-tsirpanis in
SixLabors/ImageSharp#2200
* Jpeg encoder complete rewrite by @​br3aker in
SixLabors/ImageSharp#2120
* Fix IPTC tags written on jpg files that contains non-English
characters can't be correctly displayed on external apps #​2212 by
@​lmerino-ep in SixLabors/ImageSharp#2213
* Identify transparency for indexed PNG correctly by @​brianpopow in
SixLabors/ImageSharp#2215
* Introduce Shared General Decoder Options plus Specialization by
@​JimBobSquarePants in SixLabors/ImageSharp#2180
* Add option to encode 2-bit Bitmap's by @​brianpopow in
SixLabors/ImageSharp#2220
* Fix AdaptiveThresholdProcessor throws IndexOutOfRangeException by
@​brianpopow in SixLabors/ImageSharp#2221
* Relaxed bmp dimensions validation by @​smorokin in
SixLabors/ImageSharp#2192
* Remove obsolete code by @​JimBobSquarePants in
SixLabors/ImageSharp#2189
* Implement Median Blur processor by @​ynse01 in
SixLabors/ImageSharp#2219
* Make TestFile Image threadsafe. by @​JimBobSquarePants in
SixLabors/ImageSharp#2225
* PNG filter improvements by @​saucecontrol in
SixLabors/ImageSharp#2226
* Use File-Scoped Namespaces and Global Implicit Usings by
@​JimBobSquarePants in SixLabors/ImageSharp#2229
* Fix for issue #​2117 by @​brianpopow in
SixLabors/ImageSharp#2230
* Use System.Half for HalfTypeHelper by @​JimBobSquarePants in
SixLabors/ImageSharp#2233
* [Chore] Remove InlineningOption from ThrowHelpers by @​blouflashdb in
SixLabors/ImageSharp#2234
* Implement AutoLevel processor - Attempt 2 by @​ynse01 in
SixLabors/ImageSharp#2235
* Add TryGetLinearlySeparableComponents and tests by @​JimBobSquarePants
in SixLabors/ImageSharp#2224
 ... (truncated)

## 2.1.13

## What's Changed
* V2 backport: 2243 - Ignore unknown chunks for WEBP by @​sonbua in
SixLabors/ImageSharp#2505

## New Contributors
* @​sonbua made their first contribution in
SixLabors/ImageSharp#2505

**Full Changelog**:
SixLabors/ImageSharp@v2.1.12...v2.1.13

## 2.1.12

## What's Changed
* Backport #​2926 to release/2.1.x by @​mus65 in
SixLabors/ImageSharp#2985

## New Contributors
* @​mus65 made their first contribution in
SixLabors/ImageSharp#2985

**Full Changelog**:
SixLabors/ImageSharp@v2.1.11...v2.1.12

## 2.1.11

## What's Changed
* Backport V2 - GIF: Check for end of stream when reading comments. by
@​JimBobSquarePants in SixLabors/ImageSharp#2971


**Full Changelog**:
SixLabors/ImageSharp@v2.1.10...v2.1.11

## 2.1.10

## What's Changed
* Backport #​2859 to release/2.1.x by @​antonfirsov in
SixLabors/ImageSharp#2890
* Backport #​2701 to 2.1.x [copy] by @​antonfirsov in
SixLabors/ImageSharp#2891


**Full Changelog**:
SixLabors/ImageSharp@v2.1.9...v2.1.10

Commits viewable in [compare
view](SixLabors/ImageSharp@v2.1.9...v4.1.2).
</details>

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=SixLabors.ImageSharp&package-manager=nuget&previous-version=2.1.9&new-version=4.1.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/microsoft/onnxruntime/network/alerts).

</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants