Provide a pluggable method for asset transformation

@christopherfujino

The tool already does transformation of icon fonts at build time.

It would be good to make an interface like that open and pluggable, so that packages external to the tool could decide to transform assets. For example, package:vector_graphics will want to transform SVG files into a different kind of file at build time by invoking a Dart program to process them.

@christopherfujino @jonahwilliams for input

Some rough notes:

The icon font shaker currently operates by getting fed all the files going into the asset bundle and sniffing the mime type. It then transforms the file and outputs a file with the same name and extension.
Some transformers might want to replace existing files with brand new ones that have a different file name, but I feel like there's something I haven't thought all the way through here. Maybe keeping the file name the same is a good thing.
We'll probably want ways to allow the manifest to specify which files get transformed and by which transformers, perhaps with regexps or globs for files?

@jonahwilliams

@jonahwilliams says he might be able to write a onepager on this. We talked some more this morning and agreed we don't need to make this work for the icon shaking, but it might help support some other features the tool has had requests for like platform specific assets.

@jonahwilliams

@jonahwilliams says he might be able to write a onepager on this. We talked some more this morning and agreed we don't need to make this work for the icon shaking, but it might help support some other features the tool has had requests for like platform specific assets.

SGTM

It would be great if we could make both the icon font transformer and the license file munger components that just plug into this more generic framework.

one of these ways I tried to make this tractable was to massively simplify the scope of what we could run over. By just allowing asset -> asset transformations, we allow for a very simple API with almost no configuration required.

We must avoid making a generic build system, otherwise we'd probably be better taking something off the shelf. These usually come with severe tradeoffs and don't work in the same out of the box way that Flutter does today.

oh totally, i wasn't suggesting making a build system. the icon transformer is asset->asset, right?
i agree that the license is more assets->assets, which is less of a good fit if the goal is simplicity.

@andrewkolos

@andrewkolos I just noticed that #132985 is already merged. Do you plan work on this in 2024? Thanks a lot! :)

@andrewkolos

@andrewkolos I just noticed that #132985 is already merged. Do you plan work on this in 2024? Thanks a lot! :)

Yes, this is what I plan on working on next (other than routine maintenance work).

For anyone watching this issue or interested in how users will utilize this feature,

I'm currently drafting a PR for this. To help inform the design, I'm interested in any potential use case you might have for it.

Currently, I intend to provide a separate YAML section in the pubspec, where the user lists their transformers and what assets they will be run against:

flutter:
  # ...
  assets:
    - assets/splash/logo.svg
    - assets/chatbot/
    - path: assets/integrations/free/
      flavor: free
    - path: assets/integrations/premium/
      flavor: premium
    # ...
  # ...
  asset-transformers:
    - package: vector_graphics_compiler
      args:
        - "--tesselate"
        - "--use-half-precision-control-points"
      assets:
        - "**/*.svg"
      # Globs are not normally supported for asset lists for technical reasons.
      # However, we can support them here since this doesn't *add* any
      # files—the glob is only run against the existing assets list.

Normally, I'd be hesitant to have something outside of the assets affect assets¹, but I'm assuming that the vast majority of use cases of this feature are going to involve running a transformer against all assets of a specific file type (like in the example scenario above, where I want to run a transformer against all bundled SVG files, regardless of where they are being used in the app).

One alternative is to have asset tranformer definitions remain in their own YAML section (asset-transformers), but to not have them define an assets list. Instead, they would be given a name, and assets in the main asset list would provide any relevant transformations by name:

flutter:
  # ...
  assets:
    - path: assets/splash/logo.svg
      transformers:
        - compileSvg # Refer to later transformer definition by name.
    - assets/chatbot/
    - path: assets/integrations/free/
      flavor: free
      # new:
    - path: assets/integrations/free/logos/
      flavor: free
      transformers:
        - compileSvg
    - path: assets/integrations/premium/
      flavor: premium
      # new:
    - path: assets/integrations/premium/logos/
      flavor: free
      transformers:
        - compileSvg
    # ...
  # ...
  asset-transformers:
    # This transformer definition no longer provides its own `assets` list, but it has a
    # name that can be referred to in the main `assets` section.
    - name: compileSvg
      package: vector_graphics_compiler
      args:
        - "--tesselate"
        - "--use-half-precision-control-points"

This keeps the assets section more complete in regards to containing all information related to assets, but it also makes the task of running a transformer against all files with a given type tedious.

This makes pubspecs slightly harder to read for folks onboarding to a project and troubleshooting asset-related issues. They need to notice that some separate YAML section to have any clue that this feature is being used while all other asset-related info is kept together in assets. ↩

Do packages with entries in that section automatically executed? I imagine that being useful, although it's a bit scare with regard to security, since it basically allows arbitrary code execution. But it would be certainly nice from a DX experience. Just add a transformer as a dev dependency and it just works™️

Do transformers know whether they get executed in debug or release mode? I kinda imagine that being useful. Ex: There's a transformer that optimizes the file size of images. Depending on the amount and size of the images, that could take a significant amount of time. Therefore, I might want to only execute it in release builds.

Another use case is the automatic creation of resolution-aware images. Imagine just the high resolution image (3x) gets added. Can a transformer create the 2x and 1x version of the image?

Another question is do transformer also transform assets included by other packages?

@bdero

Thanks for the questions!

Do packages with entries in that section automatically executed? I imagine that being useful, although it's a bit scare with regard to security, since it basically allows arbitrary code execution. But it would be certainly nice from a DX experience. Just add a transformer as a dev dependency and it just works™️

Yes. In particular, they get run at flutter run/flutter build, and only if the user has configured their project to use them. Indeed, this could theoretically be exploited, but also consider the following:

Dart packages already have these privileges if imported in user code (or explicitly run through dart run <package-name>).
The upcoming Native Assets feature will (likely) allow code execution even from packages even more subtly (in a way similar to many existing ecosystems today do—Rust's Cargo and Node Package Manager (NPM) scripts come to mind).

To put it more simply, I am comfortable having users needing to trust any dependency they add to their project.

Do transformers know whether they get executed in debug or release mode? I kinda imagine that being useful. Ex: There's a transformer that optimizes the file size of images. Depending on the amount and size of the images, that could take a significant amount of time. Therefore, I might want to only execute it in release builds.

Good idea. flutter could make this available to the transformer either through an argument or through a new well-known environment variable (assuming it doesn't already exist).

Another use case is the automatic creation of resolution-aware images. Imagine just the high resolution image (3x) gets added. Can a transformer create the 2x and 1x version of the image?

Right now, the feature is designed with one-to-one asset transformation in mind. Technically, there is nothing stopping a transformer from doing such a thing since it has access to the filesystem (though there are caveats here). Still, this use case would not explicitly supported.

If anyone has design ideas for how one-to-many (or many-to-one) asset transformation might work from the user's perspective, I would be interested—not necessarily because I would want to implement this in the initial feature, but because I'd like to make the sure the current design would not be incompatible with such an idea.

(cc: @bdero, who may be interested in this)

Another question is do transformer also transform assets included by other packages?

Another good question. This is something I failed to consider carefully. As things are, transformers would apply to any asset that matches one of its globs. As a result, it's possible that users may unintentionally transform assets from package dependencies. Let me illustrate this with a hypothetical scenario:

A flutter project has some SVG assets. To make these usable by the vector_graphics package, the user sets up a transformer in their pubspec like so:

flutter:
  # ...
  asset-transformers:
    - package: vector_graphics_compiler # Suppose vector_graphics_compiler has been updated to support this feature.
      assets:
        - **.svg

The same project uses some widget that comes from a package dependency. We'll call this package my_dependency and the widget PackageWidget. PacketWidget depends on an SVG within the same package using this code: SvgPicture.asset('packages/my_dependency/assets/file.svg'). This uses the flutter_svgpackage to render its SVGs, not vector_graphics.
The user runs flutter run. The flutter tool transforms packages/my_dependency/assets/file.svg because it matches the **.svg glob. SvgPicture.asset will fail, presumably with a vague error message about packages/my_dependency/assets/file.svg being formatted incorrectly. This is both surprising and confusing to the end user, who was unaware that this file even existed, and they might not be able to figure out why this is happening.

We could certainly add some sort of guard against a user accidentally doing something like this, but it is unclear if it is worth the effort.

provide command to `FakeCommand::onRun` (#142206)

I'm planning to land support for this in pieces, see #143348 (perhaps I should have hijacked this issue instead 🤷). The only other thing to note at the moment is that I've tweaked the design slightly—asset transformations are now declared together with assets, rather than having a separate section in the pubspec.

Expose build mode in environment of asset transformer processes (#144752

Reverts "Expose build mode in environment of asset transformer proces…

Seems this can be closed, right?

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

Navigation Menu

Repository navigation

Provide a pluggable method for asset transformation #101077

19 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Provide a pluggable method for asset transformation #101077

Description

Activity

dnfield commented on Mar 31, 2022

dnfield commented on Mar 31, 2022

christopherfujino commented on Mar 31, 2022

Hixie commented on Jan 19, 2023

jonahwilliams commented on Jan 19, 2023

Hixie commented on Jan 20, 2023

19 remaining items

NashIlli commented on Dec 30, 2023

andrewkolos commented on Jan 2, 2024

andrewkolos commented on Jan 8, 2024

Footnotes

ueman commented on Jan 13, 2024

andrewkolos commented on Jan 17, 2024

andrewkolos commented on Feb 13, 2024

Levi-Lesches commented on May 16, 2024

github-actions commented on May 30, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions