More helpful exception when compressing large files in seek mode

@ericstj

When the user opens a stream in Create mode, and passes it to a ZipArchive opened in Update mode, then attempts to add a very large file to the zip, an exception will be thrown on dispose stating Stream was too long, which is confusing for the user, and frustrating if the file was large and this happens so late.

The exception is thrown because we restrict the size of a file to Int32.MaxValue. The restriction is reached because the Update mode requires a stream that allows seeking, so we internally create a MemoryStream so that we can update the archive, but the size limit for the file is Int32.MaxValue.

We do have it documented here, but it's not very obvious for the user to look for answers in that location.

We should explore two improvements:

Throw a more helpful error message when this combination of conditions meet (Stream in Create mode, ZipArchive in Update mode, file too large).
Try to detect this case much earlier, instead of waiting it to happen on Dispose.

I also opened to suggest a Roslyn analyzer that would help users avoid falling into this: #35815

This has been reported a few times already:

/cc @ericstj

@carlossanlop

@carlossanlop I am contributing after a long time and would like to attempt this.

@WinCPP

@WinCPP welcome back! I will assign this to you. Let us know if you need help.

@WinCPP

@WinCPP welcome back! I will assign this to you. Let us know if you need help.

Thanks! And yes, I do need help 🙂

As per the documentation, the file to be added to an archive in update mode cannot be larger than Int32.MaxValue in size. However, the following exception is thrown even when the size is Int32.MaxValue - 1.

      Message: 
System.OutOfMemoryException : Array dimensions exceeded supported range.

      Stack Trace: 
MemoryStream.set_Capacity(Int32 value) line 288
MemoryStream.EnsureCapacity(Int32 value) line 165
MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count) line 626
WrappedStream.Write(Byte[] buffer, Int32 offset, Int32 count) line 155
Stream.CopyTo(Stream destination, Int32 bufferSize) line 72
BufferedFileStreamStrategy.CopyTo(Stream destination, Int32 bufferSize) line 945
FileStream.CopyTo(Stream destination, Int32 bufferSize) line 508
Stream.CopyTo(Stream destination) line 51
ZipFileExtensions.DoCreateEntryFromFile(ZipArchive destination, String sourceFileName, String entryName, Nullable`1 compressionLevel) line 123
ZipFileExtensions.CreateEntryFromFile(ZipArchive destination, String sourceFileName, String entryName, CompressionLevel compressionLevel) line 78
ZipFile_ZipArchive_Create.CreateEntryFromLargeFileArchiveUpdateExtension(Boolean withCompressionLevel) line 52

Here is the line that causes the exception (link).

Digging more, I found this discussion about mscorlib code over here. that mentions that the max possible index for a byte array is 0x7FFFFFC7 although the max size can be Int32.MaxValue. When I try with a file of size 0x7FFFFFC7 it works fine, but just a byte more i.e. 0x7FFFFFC8 and I get the above-mentioned exception. So should the logic for this fix use 0x7FFFFFC7 as the maximum allowable size instead of Int32.MaxValue? Appreciate your inputs.

@WinCPP

Digging more, I found this discussion about mscorlib code over here. that mentions that the max possible index for a byte array is 0x7FFFFFC7 although the max size can be Int32.MaxValue. When I try with a file of size 0x7FFFFFC7 it works fine, but just a byte more i.e. 0x7FFFFFC8 and I get the above-mentioned exception. So should the logic for this fix use 0x7FFFFFC7 as the maximum allowable size instead of Int32.MaxValue? Appreciate your inputs.

Hi @WinCPP. I'm sorry we missed your question here. I agree with your assessment here that we should cap the size to 0x7FFFFFC7 instead of Int32.MaxValue. Let us know if you would still like to contribute to this.

@jeffhandley

Hi, @jeffhandley thanks for the reply. I am busy with some office work ATM. So I am fine if someone wants to take this ahead.

@WinCPP

Thanks, @WinCPP. And sorry again we missed your question earlier this year.

@jeffhandley

Hey @jeffhandley @carlossanlop @WinCPP, this issue is still reproducible, and the maximum size I can use remains 0x7FFFFFC7. Would decreasing the maximum size for the MemoryStream class still be a viable solution?

runtime/src/libraries/System.Private.CoreLib/src/System/IO/MemoryStream.cs

Line 37 in 0eec0be

private const int MemStreamMaxLength = int.MaxValue;

Navigation Menu

Repository navigation

More helpful exception when compressing large files in seek mode #43542

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

More helpful exception when compressing large files in seek mode #43542

Description

Activity

WinCPP commented on Jan 9, 2022

danmoseley commented on Jan 9, 2022

WinCPP commented on Jan 14, 2022

jeffhandley commented on Oct 22, 2022

WinCPP commented on Oct 22, 2022

jeffhandley commented on Oct 25, 2022

alinpahontu2912 commented on Aug 25, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions