Repository navigation
Deduplicate NOTICES #71102
Description
On 1.24.0-8.0.pre.344, the LicenseResult collected from LicenseCollector https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/asset.dart#L322 produce a lot of redundancies.
The uncompressed size of the file is relevant for #71098.
A breakdown of occurrences per library shows:
86 harfbuzz
75 boringssl
47 skia
34 turbo
34 libjpeg
29 angle
25 zlib
22 icu
19 freetype2
17 dart
12 fuchsia_sdk
9 libwebp
8 glfw
4 rapidjson
4 files
4 base
3 libcxx
3 khronos
3 double
3 cpp
3 conversion
3 abseil
2 xxhash
2 tcmalloc
2 skcms
2 root_certificates
2 ffx_spd
1 wuffs
1 wasmer
1 vulkanmemoryallocator
1 vector_math
1 test_api
1 term_glyph
1 smhasher
1 pkg
1 libsdl
1 flutter
1 engine
1 colorama
1 clock
1 charcode
1 characters
1 boolean_selector
1 async
1 StackWalker
Some examples of "different" entries:
skia
Copyright 2018 Google Inc.
...
skia
Copyright 2018 Google LLC
...
skia
Copyright 2018 Google LLC.
skia
Copyright 2018 Google, LLC
Do we need all these variations?
Activity
xster commentedon Dec 4, 2020
I wrote a thing to manually reshuffle this and deduplicate it in dart. Though I'm getting 286,281 bytes and the zip compression outcome was already 63,477. Might be worth thinking about double compressing it and finding a way to have a lightweight uncompression mechanism in the runtime.
jonahwilliams commentedon Dec 4, 2020
FYI @Hixie
Hixie commentedon Dec 4, 2020
There are subtleties to doing this which @xster and I discussed a few days ago. Reducing the post-install size (currently ~725KB), or indeed the on-the-wire size (currently about ~290KB), is definitely valuable. I'm happy to help review PRs if someone wants to work on a more targeted encoding than our current choice (namely hyphen format + UTF8 + zip).
xster commentedon Dec 16, 2020
Causes hot reload latency regression. Needs some tweaking (since this file almost never changes).
xster commentedon Feb 11, 2021
Note for future self when re-implementing, create a custom DevFSContent type in https://github.com/flutter/flutter/pull/71899/files#diff-1339eded5eeb35ab9fbc61051c530c73e307d4640328760c1af0eee36c142be2R359. Let it hold the uncompressed string. Supply the compressed string to the super's bytes getter, but otherwise, let multiple instances of itself compare the uncompressed strings first before replacing the instance in the entries list.
xster commentedon Apr 27, 2021
Rechecking stats after.
iOS went from 5MB compressed, 12.1MB uncompressed to 5MB compressed, 11.4MB uncompressed. This was the intended effect.
Android seems to have regressed a bit, from 5,073,485 to 5,082,622 bytes. Double zipping surprisingly didn't increase the size. The double compressed NOTICES is 64,459 vs 66,238 before.
But to decompress, we did end up retaining a bit more of dart:io, from 34.6kB to 41kB.
github-actions commentedon Aug 2, 2021
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.