Not able to read jpg from stream when other data comes first -> Missing SOI #818

Prerequisites

• I have written a descriptive issue title
• I have verified that I am running the latest version of ImageSharp
• I have verified if the problem exist in both DEBUG and RELEASE mode
• I have searched open and closed issues to ensure it has not already been reported

Description

Writing a jpg to a stream and reading it back is working. But only when there is no other data in front. If we write something else to the stream first and the image data after that, we get an "missing SOI" exception while reading back (although we start at the correct stream position). When the exception is thrown, stream.Position is 4096. So my guess is, the data is read blockwise and there is one method that does not calculate the initial offset.

Steps to Reproduce

1. Open a memory stream
2. Write some data (e.g. an integer) to the stream, then write the image data (Image.SaveAsJpeg)
3. Set Stream.Position to 0
4. Get a BinaryReader on the stream
5. Read the Int32, so Stream.Position is now "4"
6. Now we should be able to read the jpg back but we get an exception ("Missing SOI"). Stream.Position is now 4096 (actual stream.Length is much longer).

This is working:

using (var stream = new MemoryStream())
{
    ((Image<Rgb24>)myImage).SaveAsJpeg(stream);
    stream.Position = 0;

    using (var reader = new BinaryReader(stream))
        Image.Load<Rgb24>(stream);
}

This is not working:

using (var stream = new MemoryStream())
using (var writer = new BinaryWriter(stream))
{
    writer.Write((int)5);
    ((Image<Rgb24>)myImage).SaveAsJpeg(stream);
    stream.Position = 0;

    using (var reader = new BinaryReader(stream))
    {
        reader.ReadInt32();
        Image.Load<Rgb24>(stream);
    }
}

System Configuration

• ImageSharp version: Beta6
• Other ImageSharp packages and versions: SixLabors.Core Beta7
• Environment (Operating system, version and so on): Windows 10, 1803, 64 Bit
• .NET Framework version: .Net Standard 2.0