Skip to content

Fix Color Filters - #806

Merged
JimBobSquarePants merged 4 commits into
masterfrom
js/color-filters
Jan 17, 2019
Merged

JimBobSquarePants merged 4 commits into
masterfrom
js/color-filters

Conversation

@JimBobSquarePants

@JimBobSquarePants JimBobSquarePants commented Jan 9, 2019 •

Copy link
Copy Markdown
Member

Prerequisites

  • I have written a descriptive pull-request title
  • I have verified that there are no overlapping pull-requests open
  • I have verified that I am following matches the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
  • I have provided test coverage for my change (where applicable)

Description

This PR replaces the incorrect Matrix4x4 filter implementation with a new ColorMatrix transform.

All color matrices normally have a minimum of 5x4 indices which are used to transform the vector. While our implementation gave accurate output in fully opaque images, images with an alpha component were incorrect. In addition, the Hue filter was using an incorrect operator in its calculation which led to output error.

This PR.

  • Introduces a new ColorMatrix struct modelled after the Matrix4x4 struct. This contains all the properties required to implement color transforms.
  • Updates the KnownFilterMatrices generation code to reflect the new struct layout and fix bugs. HueProcessor doesn't behave as expected #802
  • Optimizes the filter process using bulk operations.

@JimBobSquarePants
JimBobSquarePants requested review from antonfirsov, dlemstra and tocsoft and removed request for antonfirsov January 9, 2019 14:00
@JimBobSquarePants

Copy link
Copy Markdown
Member Author

Starting to get really annoyed by tests randomly failing in different environments. The NET Core tests pass on my machine.

@codecov

codecov Bot commented Jan 10, 2019 •

Copy link
Copy Markdown

Codecov Report

Merging #806 into master will increase coverage by 0.04%.
The diff coverage is 95.81%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #806      +/-   ##
==========================================
+ Coverage   88.79%   88.83%   +0.04%     
==========================================
  Files        1012     1014       +2     
  Lines       43313    43703     +390     
  Branches     3124     3124              
==========================================
+ Hits        38459    38824     +365     
- Misses       4156     4178      +22     
- Partials      698      701       +3
Impacted Files Coverage Δ
src/ImageSharp/Processing/FilterExtensions.cs 100% <ø> (ø) ⬆️
...ests/ImageSharp.Tests/Helpers/Vector4UtilsTests.cs 100% <100%> (ø) ⬆️
....Tests/Processing/Processors/Filters/FilterTest.cs 100% <100%> (ø) ⬆️
...p/Processing/Processors/Filters/FilterProcessor.cs 100% <100%> (ø) ⬆️
...sts/ImageSharp.Tests/Formats/GeneralFormatTests.cs 97.46% <100%> (-0.04%) ⬇️
...ts/ImageSharp.Tests/Primitives/ColorMatrixTests.cs 100% <100%> (ø)
...ts/Processing/Processors/Filters/BrightnessTest.cs 100% <100%> (ø) ⬆️
src/ImageSharp/Common/Helpers/Vector4Utils.cs 100% <100%> (ø) ⬆️
src/ImageSharp/Processing/KnownFilterMatrices.cs 100% <100%> (ø) ⬆️
...rocessing/Processors/Filters/ColorBlindnessTest.cs 100% <100%> (ø) ⬆️
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5755779...91d1c15. Read the comment docs.

@antonfirsov antonfirsov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job! A few imprtant remarks:

  • There are issues with math correctness for certain (unused!) Matrix5x4 operations. It's easy to solve them by simply dropping them :)
  • We should consider moving Matrix5x4 to SixLabors.Core (not necessarily in this PR)
  • We should probably add tests based on images in #802. For my naive eyes they look like important special cases.

Comment thread src/ImageSharp/Primitives/Matrix5x4.cs Outdated
/// A structure encapsulating a 5x4 matrix.
/// </summary>
[StructLayout(LayoutKind.Sequential)]
public struct Matrix5x4 : IEquatable<Matrix5x4>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we move this to SixLabors.Core?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it has any practical application outside of ImageSharp.

Comment thread src/ImageSharp/Primitives/Matrix5x4.cs Outdated
/// <summary>
/// Gets the multiplicative identity matrix.
/// </summary>
public static Matrix5x4 Identity { get; } =

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The term identity matrix is only valid for square matrices! We should drop this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See below

{
#if NETCOREAPP2_1
Matrix5x4 m = KnownFilterMatrices.CreateBrightnessFilter(.5F);
HashCode hash = default;

@antonfirsov antonfirsov Jan 12, 2019 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test depends too strongly on the current implementation. IMO a unit test for GetHashCode() should be only responsible for ensuring that for equal objects the hash code is also equal. (The opposite is not necessarily true!)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just copied what they do with Numerics tests. I think it still adds value as it ensures we touch every field when calculating the hashcode

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's keep it as is for now!

Comment thread src/ImageSharp/Primitives/Matrix5x4.cs Outdated
/// <param name="value1">The first source matrix.</param>
/// <param name="value2">The second source matrix.</param>
/// <returns>The result of the multiplication.</returns>
public static Matrix5x4 operator *(Matrix5x4 value1, Matrix5x4 value2)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The result of this multiplication should be either a 5x5 or a 4x4 matrix, depending on how we define the operation.

As for non-square fixed size matrices the multiplication has usually no practical use, I suggest to drop this operation.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you touched upon below this does make sense if our struct is an optimized form of a Matrix5x5 in the manner that Matrix3x2 is an optimized Matrix3x3 which it is.

Most importantly it works!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missed that usage, thanks for pointing it out!

@antonfirsov

antonfirsov commented Jan 13, 2019 •

Copy link
Copy Markdown
Member

Re Matrix5x4:

  • Multiplication and identity actually makes sense if Matrix5x4 is an optimized form for a Matrix5x5 having an invariant (0,0,0,0,1) row/column the same way as Matrix3x2 is really just an optimized Matrix3x3.
  • We may consider renaming it to ColorMatrix and keep it inside the bounds of ImageSharp

@JimBobSquarePants

Copy link
Copy Markdown
Member Author

We should probably add tests based on images in #802. For my naive eyes they look like important special cases.

I did consider that but it only really adds regression protection which we already have covered with our sample files. I could replace the samples though.

We may consider renaming it to ColorMatrix and keep it inside the bounds of ImageSharp

This did cross my mind also, I just like the conformity of naming it currently has with Numerics. It's a coin toss.

@antonfirsov

antonfirsov commented Jan 16, 2019 •

Copy link
Copy Markdown
Member

I don't think it has any practical application outside of ImageSharp.

If we keep the class within the bounds of ImageSharp, and only use it only for colorspace transformations, it makes the most sense to name it ColorMatrix. Fight me! 😄

(The rest of my findings are obsolete.)

@JimBobSquarePants

Copy link
Copy Markdown
Member Author

Haha... I’ll rename it tomorrow then.

@JimBobSquarePants
JimBobSquarePants merged commit b74e230 into master Jan 17, 2019
@JimBobSquarePants
JimBobSquarePants deleted the js/color-filters branch January 17, 2019 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants