Skip to content

Use decoded quantization tables for Jpeg encoding, store them in JpegMetadata #1705

Closed
#1706
Closed
Use decoded quantization tables for Jpeg encoding, store them in JpegMetadata#1705
#1706
Labels
needs triage

Description

@br3aker
br3aker
opened on Jul 17, 2021

Prerequisites

  • I have written a descriptive issue title
  • I have verified that I am running the latest version of ImageSharp
  • I have verified if the problem exist in both DEBUG and RELEASE mode
  • I have searched open and closed issues to ensure it has not already been reported

Description

https://user-images.githubusercontent.com/20967409/126037087-16a8bd51-8cc5-46c8-ab8a-e66d625a8595.jpg
This image gains a lot of noise after decode-encode scenario (you better download it and magnify with some app or use a histogram):

9483_comparison

Steps to Reproduce

Image above with:

using var image = Image.Load(loadPath);
JpegEncoder encoder = new JpegEncoder { };
image.SaveAsJpeg(savePath, encoder);

Cause

Given photo uses some really strange quantization tables:

image

Non-standard tables are not a problem but current QualityEvaluator misses this one quiet drastically:

// DQT #1
Quality estimation:
        Standard: False
        Quality: 70

// DQT #2
Quality estimation:
        Standard: False
        Quality: 52

// image.Metadata.GetJpegMetadata().Quality
Loaded image quality: 52

// JPEGsnoop output:
Approx quality factor = 71.19 (scaling=57.62 variance=593.35) - Luminance
Approx quality factor = 80.24 (scaling=39.51 variance=961.47) - Chrominance

So now we have 2 problems:

  1. Huge miss on quality
  2. Deducting quality twice as current code does not differ 2+ DQT markers

Proposal

Save decoded jpeg quantization tables to the jpeg metadata and use it as backup in the encoder:

// somewhere in the jpeg encoder

// always prefer explicit quality
if(options.Quality != null) 
{
    return GetStandardQuantizationTablesFromQuality(options.Quality);
}
if(metadata.UsesStandardQuantizationTables)
{
    return GetStandardQuantizationTablesFromQuality(metadata.Quality);
}
return metadata.QuantizationTables;

Activity

  1. br3aker
    added
    needs triage
    on Jul 17, 2021

  2. br3aker commented on Jul 17, 2021

    @br3aker
    br3aker
    on Jul 17, 2021
    ContributorAuthor

    I'm already working on it, just need a confirmation on public API of JpegMetadata.

  3. antonfirsov commented on Jul 17, 2021

    @antonfirsov
    antonfirsov
    on Jul 17, 2021
    Member

    I like the proposal!

  4. JimBobSquarePants commented on Jul 17, 2021

    @JimBobSquarePants
    JimBobSquarePants
    on Jul 17, 2021
    Member

    We should probably have a look at how JPEGsnoop determines quality so ours are not so inaccurate.

  5. br3aker commented on Jul 17, 2021

    @br3aker
    br3aker
    on Jul 17, 2021
    ContributorAuthor

    @JimBobSquarePants better to do as a separate PR I guess so it won't be much of a clutter.

  6. br3aker
    changed the title [-]Jpeg default encoder (drastically) alters re-encoded image quality[/-] [+]Use decoded quantization tables for Jpeg encoding, store them in JpegMetadata[/+] on Jul 17, 2021

  7. JimBobSquarePants commented on Jul 17, 2021

    @JimBobSquarePants
    JimBobSquarePants
    on Jul 17, 2021
    Member

    Wouldn't more accurate evaluation negate the need to carry the tables?

    You also need accurate quality evaluation in order to use the tables since you have to unscale these tables before copying to reuse in the encoder so that the user can re-encode them at a different quality.

  8. br3aker commented on Jul 17, 2021

    @br3aker
    br3aker
    on Jul 17, 2021
    ContributorAuthor

    The main problem with 'quality' is that there's actually no 'quality' property of jpeg. Many professional cameras use custom tables:
    https://www.impulseadventure.com/photo/jpeg-quantization.html

    Any quality-evaluation algorithm is just an approximation built around standard table from ITU specification (I didn't check JPEGsnoop implementation). Extacting quality from non-standard table and then rescaling non-standard matrix would introduce a lot of errors like in the example above.

    Main idea of this proposal is to carry the tables if we don't want to alter 'quality' level. So we can potentially have 3 scenarios:

    1. User provided 'quality' multiplier as encoder option - we must use it with standard table even if we had non-standard (this can have a discussion tbh)
    2. Standard table - we only need to store the quality multiplier as scaling default matrix with it would result in the same result -> no space wasted for standard matrix case
    3. Non-standard table - save table and estimate quality because we have it in the metadata (I would prefer to rename Quality to EstimatedQuality/ApproximateQuality marking Quality obsolete)

    P.S.
    Providing such API internally would bring the ability to alter tables on per-image/per-encoder basis but it's an another story.

  9. br3aker commented on Jul 17, 2021

    @br3aker
    br3aker
    on Jul 17, 2021
    ContributorAuthor

    Checked JPEGsnoop quality evaluation implementation - it's nice, really nice. I will port it and open a draft PR so we can discuss what to do with saving tables.

  10. JimBobSquarePants
    closed this as completedin #1706on Aug 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions