Flutter SDK installed via package managers like Hermit are not detected/initialized correctly #3961

Activity

1. alecthomas

   added

   is bug

   on May 14, 2022
2. alecthomas commented on May 14, 2022

   

   alecthomas

   on May 14, 2022

   Author

   More actions

   Ah I do see that the search order seems to be deliberate, which seems a bit odd to me, but either way I think that if flutter and dart work, then the VSCode extension should work too. But maybe it has no way of finding the real SDK location?
3. DanTup commented on May 14, 2022

   

   DanTup

   on May 14, 2022

   Last edited by DanTup

   Member

   More actions

   VS Code needs to locate a real SDK because it needs the path for some of its functionality (for example when debugging, we need to be able to translate dart:xxx URIs into paths to the source files inside your SDK folder). If your shell scripts were instead symlinks to the real SDK binaries, I would expect the SDK is found (because we resolve symlinks during the search).

   Some questions:

   1. What's the purpose of the shell scripts you have wrapping the SDK?
   2. Could they be symlinks?
   3. Is it required that VS Code goes through them for other things to work?
   4. If it is required, is it required for every run, or would using the wrapper script only for the very first invocation of flutter and then using the real SDK binary directly work?
4. DanTup

   added

   awaiting infoRequires more information from the customer to progressRequires more information from the customer to progress

   on May 14, 2022
5. alecthomas commented on May 14, 2022

   

   alecthomas

   on May 14, 2022

   Author

   More actions

   Ah okay, that makes complete sense.

   1. The scripts are installed by Hermit, which is a bit like asdf.
   2. Unfortunately not.
   3. It is yes, the scripts will self-bootstrap Hermit, and auto-install the associated package (in this case Flutter).
   4. Yes this would work! On first invocation Hermit will install flutter into $CACHEDIR/hermit/pkg/flutter-3.0.0.

   Interestingly the VSCode extension log does know about Hermit's installed Flutter location.

   FTR this is Hermit's flutter package. It doesn't currently work out of the box because flutter requires its installation location to be mutable, which is slightly unfortunate, but I've been doing this manually, and I'm also thinking of adding a mutable = true option to Hermit to support this.

   Within the aforementioned constraints of Hermit, I'd be interested in figuring out a way to make VSCode work with it. Let me know if there's anything I can do to help.
6. DanTup commented on May 14, 2022

   

   DanTup

   on May 14, 2022

   Member

   More actions

   The scripts are installed by Hermit, which is a bit like asdf.

   1. I'm not familiar with either. Is it Hermit that is creating ~/Projects/flutter/bin? (Or more specifically, if other users are using Hermit, would their wrapped script be at this exact location too, or is this a custom script you have created?)

   2. Which of these paths do you have on PATH? Would it be reasonable/valid to have both of these (in this order) on PATH?

   • ~/Projects/flutter/bin
   • /Users/aat/Library/Caches/hermit/pkg/flutter-3.0.0

   This setup doesn't sound completely different from Snap on Ubuntu. After installing the snap package there is a flutter binary that is not part of a valid SDK. If we fail to find an SDK but we do find that binary, we invoke the binary (and wait for it to complete, which can be slow if it's downloading Flutter) and then re-search for the SDK (and find the real SDK).

   I wonder if we could add some setting (or env var) that would allow you to configure a path to a "flutter initialisation script" that we could invoke if no SDK is found as part of initialisation.

   3. Would that work for you? (You'd need to ensure that setting or env var can be set for any users that would need this to work).
7. alecthomas commented on May 14, 2022

   

   alecthomas

   on May 14, 2022

   Author

   More actions

   I'm not familiar with either. Is it Hermit that is creating ~/Projects/flutter/bin? (Or more specifically, if other users are using Hermit, would their wrapped script be at this exact location too, or is this a custom script you have created?)

   The bin directory is created by hermit init . and is typically checked into a repository to make the development tooling required completely by each project self-contained. You can see an example here.

   Which of these paths do you have on PATH? Would it be reasonable/valid to have both of these (in this order) on PATH?
   ~/Projects/flutter/bin
   /Users/aat/Library/Caches/hermit/pkg/flutter-3.0.0

   It's typically just the former, but I could add the latter for the Flutter package. Actually let me try that now, it might Just Work.

   I wonder if we could add some setting (or env var) that would allow you to configure a path to a "flutter initialisation script" that we could invoke if no SDK is found as part of initialisation.

   That would absolutely work, thank you. In fact I tried to do this with the Hermit package by setting FLUTTER_ROOT, so doing it with a different envar will be fine.
8. alecthomas commented on May 14, 2022

   

   alecthomas

   on May 14, 2022

   Author

   More actions

   I tried adding the real SDK to the path but it still fails, which makes sense given that it always prioritises the project directory.

   [10:06:26 pm] [General] [Info] Environment PATH:
   [10:06:26 pm] [General] [Info]     /Users/aat/Library/Caches/hermit/pkg/flutter-3.0.0/bin
   [10:06:26 pm] [General] [Info]     /Users/aat/Projects/startup_namer/bin
   [10:06:26 pm] [General] [Info]     /Users/aat/bin
   ...
   [10:06:26 pm] [General] [Info]     Found at /Users/aat/Projects/startup_namer
   [10:06:26 pm] [General] [Info]     Returning SDK path /Users/aat/Projects/startup_namer for flutter
   
9. alecthomas commented on May 14, 2022

   

   alecthomas

   on May 14, 2022

   Author

   More actions

   If we fail to find an SDK but we do find that binary, we invoke the binary (and wait for it to complete, which can be slow if it's downloading Flutter) and then re-search for the SDK (and find the real SDK).

   Interesting, have you special-cased something for Snap?
10. DanTup commented on May 16, 2022

    

    DanTup

    on May 16, 2022

    Member

    More actions

    I'm not familiar with either. Is it Hermit that is creating ~/Projects/flutter/bin? (Or more specifically, if other users are using Hermit, would their wrapped script be at this exact location too, or is this a custom script you have created?)

    The bin directory is created by hermit init . and is typically checked into a repository

    I'm a little confused.. I thought ~/Projects/flutter was outside of your project/repo (eg. a central location for all), but now I'm not so sure. If we added some env var (say FLUTTER_INIT_SCRIPT) that would allow VS Code to find and run that script if it didn't find an SDK, what would hat path be set to? (considering it presumably needs to be a single value for the whole system, and not per-project)

    I tried adding the real SDK to the path but it still fails, which makes sense given that it always prioritises the project directory.

    I'm not sure I understand what you tested here - what is /Users/aat/Projects/startup_namer? I thought the issue was that the Flutter SDK was not being located when not initialised, but this suggests you have two SDKs and the wrong one is being selected.

    Interesting, have you special-cased something for Snap?

    Yep, although it's not something I really want to extend in a non-generic way. I'd prefer to come up with something general that could be supported by many package managers without having each of them specifically coded here.
11. alecthomas commented on May 16, 2022

    

    alecthomas

    on May 16, 2022

    Author

    More actions

    The only SDK present is at /Users/aat/Library/Caches/hermit/pkg/flutter-3.0.0. ~/Projects/startup_namer and ~/Projects/flutter` are both sample Flutter project using Hermit.

    If we added some env var (say FLUTTER_INIT_SCRIPT) that would allow VS Code to find and run that script if it didn't find an SDK

    The only thing that concerns me here is that it's a fallback after the fairly slow failure to search for Flutter. Personally I would prefer an environment variable that specifies exactly where Flutter is, and that be used immediately. Perhaps FLUTTER_SDK_ROOT or something.. I'm not sure.
12. DanTup commented on May 16, 2022

    

    DanTup

    on May 16, 2022

    Member

    More actions

    The only SDK present is at /Users/aat/Library/Caches/hermit/pkg/flutter-3.0.0.
    ~/Projects/startup_namer and ~/Projects/flutter are both sample Flutter project using Hermit

    Are you able to post (or send to me at [email protected]) the full log that includes the search? We should not return a folder like ~/Projects/startup_namer as the SDK if it is not a valid SDK.

    The only thing that concerns me here is that it's a fallback after the fairly slow failure to search for Flutter.

    My intention for FLUTTER_INIT_SCRIPT was something that would only ever occur when Flutter has not been bootstrapped:

    If Flutter is not bootstrapped:

    • searching PATH finds no valid SDK
    • FLUTTER_INIT_SCRIPT is defined, so it is run
    • search is re-run and finds the Flutter SDK that's now been set up (and is in PATH)

    If Flutter is already bootstrapped:

    • searching PATH finds Flutter

    I'm happy for FLUTTER_SDK_ROOT to be included in the search (if it's not already), but I don't think that solves the problem here. We need to be a real SDK (or a symlink to) in order for things to work, we can't point at wrapper scripts and not know where an SDK is.
13. alecthomas commented on May 16, 2022

    

    alecthomas

    on May 16, 2022

    Last edited by alecthomas

    Author

    More actions

    Are you able to post (or send to me at [email protected]) the full log that includes the search? We should not return a folder like ~/Projects/startup_namer as the SDK if it is not a valid SDK.

    I included the full log for ~/Projects/flutter in a collapsed section in the original ticket body. The only thing in ~/Projects/flutter/bin is theflutter and dart shell wrappers, no other Flutter-related files.
14. alecthomas commented on May 16, 2022

    

    alecthomas

    on May 16, 2022

    Last edited by alecthomas

    Author

    More actions

    I've subsequently deleted ~/Projects/flutter but here's what's in startup_namer and the actual SDK root:

    🐚 ~/Projects/startup_namer $ ls -l bin                                                                                                                    master
    total 32
    -rw-------  1 aat  staff  270 14 May 19:56 README.hermit.md
    -rwx------  1 aat  staff  562 14 May 19:56 activate-hermit*
    lrwxr-xr-x  1 aat  staff   18 14 May 16:16 dart@ -> .flutter-3.0.0.pkg
    lrwxr-xr-x  1 aat  staff   18 14 May 16:16 flutter@ -> .flutter-3.0.0.pkg
    -rwx------  1 aat  staff  804 14 May 19:56 hermit*
    -rw-------  1 aat  staff   87 14 May 22:04 hermit.hcl
    🐚 ~/Projects/startup_namer $ ls -l ~/Library/Caches/hermit/pkg/flutter-3.0.0                                                                              master
    total 168
    -rw-------   1 aat  staff   3606 12 May 04:38 AUTHORS
    -rw-------   1 aat  staff    732 12 May 04:38 CODEOWNERS
    -rw-------   1 aat  staff   2924 12 May 04:38 CODE_OF_CONDUCT.md
    -rw-------   1 aat  staff   5764 12 May 04:38 CONTRIBUTING.md
    -rw-------   1 aat  staff   1519 12 May 04:38 LICENSE
    -rw-------   1 aat  staff   1107 12 May 04:38 PATENT_GRANT
    -rw-------   1 aat  staff   5875 12 May 04:38 README.md
    -rw-------   1 aat  staff  18786 12 May 04:38 TESTOWNERS
    -rw-------   1 aat  staff  11722 12 May 04:38 analysis_options.yaml
    drwx------   8 aat  staff    256 12 May 21:20 bin/
    -rw-------   1 aat  staff   1829 12 May 04:38 dartdoc_options.yaml
    drwx------  19 aat  staff    608 12 May 21:20 dev/
    drwx------  16 aat  staff    512 12 May 21:20 examples/
    -rw-------   1 aat  staff   1731 12 May 04:38 flutter_console.bat
    -rw-------   1 aat  staff    296 12 May 04:38 flutter_root.iml
    drwx------  13 aat  staff    416 12 May 21:20 packages/
    -rw-------   1 aat  staff      5 15 May 18:57 version
    
15. DanTup commented on May 16, 2022

    

    DanTup

    on May 16, 2022

    Member

    More actions

    Sorry, I missed the original log. I think I see what's happening

    • We find what looks like a Flutter binary at /Users/aat/Projects/flutter/bin/flutter
    • We see that it is a symlink, so we follow it (this is to catch cases where something like /usr/bin/flutter is a symlink to the real SDK location, which we need)
    • The symlink actually just points at hermit in the same directory (/Users/aat/Projects/flutter/bin/hermit)
    • We trim the bin/executable off the end to get /Users/aat/Projects/flutter and then ask "Does this look like a Flutter SDK?" and because it has a flutter binary in it, it does, so we return it as the SDK (but it's not)

    So the problem here is that when we followed the symlink, we didn't realise that it didn't point at a flutter binary but hermit. This is actually a lot like snap (where it points at a snap binary) so rather than detecting snap with a full path, I'm going to try detecting just the binary name.

    I'll publish a pre-release build you can test with once the changes are made to confirm it all works (no promises, but hopefully later today).
16. DanTup

    added a commit that references this issue on May 16, 2022

    Handle Snap initialization more generically to support other package …

    ...

    This issue will close once commit ab864c1 is merged into the 'master' branch.

    ab864c1
17. DanTup

    closed this as completedin 4bc199aon May 16, 2022
18. DanTup commented on May 16, 2022

    

    DanTup

    on May 16, 2022

    Member

    More actions

    @alecthomas can you try upgrading both extensions to the Pre-Release versions (v2.41.20220516) and restart VS Code and let me know if you still see any issues?

    It should handle situations correctly now:

    • When Flutter is not initialized (eg. you have bin/flutter but /Users/aat/Library/Caches/hermit/pkg/flutter-3.0.0 has not been set up yet) in which case you should see VS Code say "Initializing Flutter" while it does that
    • When Flutter is already initialized (eg. /Users/aat/Library/Caches/hermit/pkg/flutter-3.0.0 has a complete SDK) it'll detect and use that

    You shouldn't need to set FLUTTER_ROOT.

    If anything doesn't work, please capture a new log from this version. Thanks!
19. DanTup

    removed

    awaiting infoRequires more information from the customer to progressRequires more information from the customer to progress

    on May 16, 2022
20. DanTup

    added this to the v3.42.0 milestone on May 16, 2022
21. DanTup

    changed the title [-]VSCode extension unable to find Dart/Flutter using FLUTTER_ROOT if `./bin/dart`/`./bin/flutter` are present[/-] [+]Flutter SDK installed via package managers like Hermit are not detected/initialized correctly[/+] on May 16, 2022
22. DanTup

    added

    in flutterRelates to running Flutter appsRelates to running Flutter apps

    on May 16, 2022
23. DanTup commented on May 16, 2022

    

    DanTup

    on May 16, 2022

    Member

    More actions

    You can switch to the pre-release versions from inside VS Code here:

    

    It may take a few minutes for them to show up as they've only just been published.
24. alecthomas commented on May 16, 2022

    

    alecthomas

    on May 16, 2022

    Author

    More actions

    Success!! Works absolutely flawlessly, thank you :)
25. DanTup commented on May 16, 2022

    

    DanTup

    on May 16, 2022

    Member

    More actions

    Great! Thanks for confirming!
26. keertip

    added a commit that references this issue on Sep 2, 2025

    Handle Snap initialization more generically to support other package …

    ...

    This issue will close once commit 4bc199a is merged into the 'master' branch.

    4bc199a