Skip to content

Huge tiff image (80+ mb) gets distorted when resized to a smaller jpg - huge memory (700+mb) footprint too. #2456

Closed
Closed
Huge tiff image (80+ mb) gets distorted when resized to a smaller jpg - huge memory (700+mb) footprint too.#2456
Labels
needs triage

Description

@shapeh
shapeh
opened on May 15, 2023

Prerequisites

  • I have written a descriptive issue title
  • I have verified that I am running the latest version of ImageSharp
  • I have verified if the problem exist in both DEBUG and RELEASE mode
  • I have searched open and closed issues to ensure it has not already been reported

ImageSharp version

3.0.1

Other ImageSharp packages and versions

None

Environment (Operating system, version and so on)

Windows 11 and Windows Server 2022

.NET Framework version

.NET 7.0.5

Description

Large tiff of 81.2mb gets distorted / colors off when resizing to smaller jpg size.
Huge memory consumption (700+mb) as well for Image.LoadAsync

Steps to Reproduce

        // retrieving the image from Azure using something like this...
        ...
       //  Image.LoadAsync makes VS studio profiler mem consumption go to 839 mb (program consumes 95 mb normally).
        using var image = await Image.LoadAsync(azureBlobData, token); 
        var typeOfImage = image.Metadata.DecodedImageFormat;

        // resize original image to new width:
        image.Mutate(x =>
            x.AutoOrient()
            .Resize(width, 0)
        ); 

        await image.SaveAsync(memoryStream, new JpegEncoder() { Quality = 85 }, token);
       memoryStream.Position = 0;           

        var resizedImage = await Image.LoadAsync(memoryStream, token);
        await resizedImage.SaveAsync(@$"C:\temp\{Guid.NewGuid()}.jpg", token);

Images

Hugetif.tif Image here: (too large to upload on Github):
https://1drv.ms/f/s!Aj75-9hUJpB1nZJHAY8Y0a0o46dLQA?e=4HLRVL

Imagesharp result:
hugetif660

Photoshop result:
hugetif660-photoshop

Activity

  1. shapeh
    added
    needs triage
    on May 15, 2023
  2. shapeh
    changed the title [-]Huge tiff image (80+ mb) gets distorted when resized to a smaller jpg.[/-] [+]Huge tiff image (80+ mb) gets distorted when resized to a smaller jpg - huge memory (700+mb) footprint too.[/+] on May 16, 2023

  3. dlemstra commented on May 16, 2023

    @dlemstra
    dlemstra
    on May 16, 2023
    Member

    Does it only happen during a resize or also when you just save the image as jpeg?

  4. JimBobSquarePants commented on May 17, 2023

    @JimBobSquarePants
    JimBobSquarePants
    on May 17, 2023
    Member

    This looks like a color type detection mismatch during decode to me.

    I'd really like to see the output of the memory profile if I can. A decoded image of that size will consume ~114MB without any modification so there's a lot of additional allocations I'd like to identify.

  5. brianpopow commented on May 17, 2023

    @brianpopow
    brianpopow
    on May 17, 2023
    Collaborator

    The is an issue decoding the image. This image uses lzw compression with a horizontal predictor. For CMYK the horizontal predictor is not reverted correctly. This should be an easy fix.

  6. shapeh commented on May 17, 2023

    @shapeh
    shapeh
    on May 17, 2023
    Author

    I'll revert tomorrow but I built a simple console app just to test and memory does not go through the roof for some reason. Tried with both Load and LoadAsync directly from file system and memory is okay.

    Minimal API / Azure causes mentioned problems. I'll see if I can do some memory profiling for you (not my specialty...)

  7. JimBobSquarePants commented on May 17, 2023

    @JimBobSquarePants
    JimBobSquarePants
    on May 17, 2023
    Member

    Thanks for the accurate diagnosis @brianpopow !

    @shapeh is what type is your azureBlobData variable?

  8. shapeh commented on May 18, 2023

    @shapeh
    shapeh
    on May 18, 2023
    Author

    @JimBobSquarePants
    azureBlobData is:

    System.IO.Stream {Azure.Storage.LazyLoadingReadOnlyStream<Azure.Storage.Blobs.Models.BlobProperties>}

    I will post some code later.

  9. shapeh commented on May 18, 2023

    @shapeh
    shapeh
    on May 18, 2023
    Author

    This is my code - memory is OK for all three methods below in a sample Console app which is weird, when memory in the Minimal API goes high - doing some investigation now...
    Colors are still off but it was expected according to the diagnosis by @brianpopow

    using Azure.Storage.Blobs;
using SixLabors.ImageSharp.Formats.Jpeg;

void TestLoad()
{
    using (Image image = Image.Load(@"C:\Web\tmp\hugetif.tif"))
    {
        image.Mutate(x => x.AutoOrient().Resize(660, 0));
        image.Save(@$"C:\Web\tmp\image-load-{Guid.NewGuid()}.jpg", new JpegEncoder() { Quality = 85 });
    }
    Console.WriteLine("Done Load.");

}
async Task TestLoadAsync()
{
    using (Image image = await Image.LoadAsync(@"C:\Web\tmp\hugetif.tif"))
    {
        image.Mutate(x => x.AutoOrient().Resize(660, 0));
        await image.SaveAsync(@$"C:\Web\tmp\image-load-async-{Guid.NewGuid()}.jpg", new JpegEncoder() { Quality = 85 });
    }
    Console.WriteLine("Done LoadAsync.");
}

async Task TestAzureLoad(CancellationToken token)
{
    var client = new BlobServiceClient("myconnectionstring");
    var containerClient = client.GetBlobContainerClient("mycontainer");
    var blob = containerClient.GetBlobClient("hugetif.tif");

    var azureBlobData = await blob.OpenReadAsync(null, cancellationToken: token);
    using var image = await Image.LoadAsync(azureBlobData, token);

    image.Mutate(x => x.AutoOrient().Resize(660, 0));

    await image.SaveAsync(@$"C:\Web\tmp\image-azure-{Guid.NewGuid()}.jpg", new JpegEncoder() { Quality = 85 });
    Console.WriteLine("Done Azure.");
}

TestLoad();
await TestLoadAsync();
await TestAzureLoad(CancellationToken.None);

  10. shapeh commented on May 18, 2023

    @shapeh
    shapeh
    on May 18, 2023
    Author

    Snapshot from Minimal API

    minapi

    Dependencies:
    minapi-dep

    Snapshot from Console App

    consoleapp

    Dependencies:
    consoleapp-dep

  11. JimBobSquarePants commented on May 18, 2023

    @JimBobSquarePants
    JimBobSquarePants
    on May 18, 2023
    Member

    The biggest allocation in the API snapshot is 89MB.

    You’ve got “show dead objects” checked in the first window. As I recall they’re objects that are removed by the next snapshot.

    I can see that you’re not disposing of your input stream also.

  12. shapeh commented on May 18, 2023

    @shapeh
    shapeh
    on May 18, 2023
    Author

    @JimBobSquarePants
    Ahh my bad - I mistook the Visual Studio Process Memory for the memory consumed by the app itself. Apologies for wasting everybody's time.

    Here I have done a diff on the two snapshots (before and after loading the image):
    minapi-correct

    Forgot this.

    "I can see that you’re not disposing of your input stream also."

    Memory is all good - just colors need fixing. Thanks again.

  13. brianpopow
    added a commit that references this issue on May 18, 2023

    Undo predictor for cmyk images, fixes #2456

    64fcccd
  14. JimBobSquarePants
    closed this as completedin 64fcccdon May 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions