Skip to content

Add libclang to LUCI bots so that we can run package:ffigen #105695

Open
Open
Add libclang to LUCI bots so that we can run package:ffigen#105695
Assignees
dcharkes
Labels
dependency: dartDart team may need to help us

Description

@dcharkes
dcharkes
opened on Jun 9, 2022

In order to run package:ffigen on bots, we need libclang.

Without libclang available, package:ffigen cannot find the dynamic library provided by libclang.

Installation instructions: https://github.com/dart-lang/ffigen#installing-llvm

I'm unfamiliar with the setup on our Flutter LUCI bots, so I cannot recommend how to get/package them. Maybe @athomas has suggestions if the bots are sufficiently similar to the Dart bots.

The install location does not matter, default locations are the best. We can add these to package:ffigen and automatically detect it. Alternatively, we can also pass in a path to the test as a commandline argument.

Activity

  1. dcharkes
    added
    team-infraOwned by Infrastructure team
    on Jun 9, 2022

  2. keyonghan commented on Jun 9, 2022

    @keyonghan
    keyonghan
    on Jun 9, 2022
    Contributor

    @dcharkes Luci bots have been relying on package binaries stored in chromium CIPD. Could you help prepare an executable binary that we can upload to cipd? Then we can reference the package from LUCI recipes in LUCI bots.

  3. godofredoc
    assigned
    dcharkes
    on Jun 10, 2022

  4. dcharkes commented on Jun 15, 2022

    @dcharkes
    dcharkes
    on Jun 15, 2022
    ContributorAuthor

    Some more exploration:

    • On Cirrus CI, libclang is already availble and the test using it succeeds.
    • For the LUCI bots @athomas and I tried installing it in the Linux Bionic image, but that has libclang 6 and ffigen needs libclang >= 9. So that would bring us back to CIPD package which we would have to maintain.
    • Running a GitHub action and installing libclang there works, however the way the tool-tests are written require ../../bin/cache/dart-sdk/bin/dart. So we would need to write the test in another way.

  5. keyonghan commented on Jun 21, 2022

    @keyonghan
    keyonghan
    on Jun 21, 2022
    Contributor

    I am removing the infra ticket queue project. Please loop back if anything is needed from flutter infra side.

  6. dcharkes commented on Jun 28, 2022

    @dcharkes
    dcharkes
    on Jun 28, 2022
    ContributorAuthor

    @dcharkes Luci bots have been relying on package binaries stored in chromium CIPD. Could you help prepare an executable binary that we can upload to cipd? Then we can reference the package from LUCI recipes in LUCI bots.

    The latest release of libclang is 14.0.6 https://github.com/llvm/llvm-project/releases/tag/llvmorg-14.0.6. (We need at least version 9.)

    However, those downloads do not provide a libclang for all 3 operating systems.
    So, We'd get them via other means. I'd usually get them through a package manager for the operating system.

    MacOS

    $ mdfind -name libclang.dylib
...

# The version of one of these.
$ /Users/dacoharkes/Library/Android/sdk/ndk/24.0.8215888/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang -v
Android (8075178, based on r437112b) clang version 14.0.1 (https://android.googlesource.com/toolchain/llvm-project 8671348b81b95fc603505dfc881b45103bee1731)
...

# libclang is self-contained and does not require dependencies (as far as I can see).
$ objdump --reloc --dynamic-reloc /Users/dacoharkes/Library/Android/sdk/ndk/24.0.8215888/toolchains/llvm/prebuilt/darwin-x86_64/lib64/libclang.dylib
/Users/dacoharkes/Library/Android/sdk/ndk/24.0.8215888/toolchains/llvm/prebuilt/darwin-x86_64/lib64/libclang.dylib:

    Linux

    $ which clang
...
$ cd /usr/lib/llvm-13
...

# It is not standalone on Linux
/usr/lib/x86_64-linux-gnu$ readelf -d libclang-13.so.13.0.1 

Dynamic section at offset 0x1fd3bf8 contains 34 entries:
  Tag        Type                         Name/Value
...
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libLLVM-13.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
...

    (To be on the save side I'd include every .so file in lib/ from the Linux LLVM download for arm64 on that GitHub downloads page.)

    Windows

    To be explored.

    @keyonghan Just to confirm. You'd like me to provide 3 archives (for each OS one) that contain libclang and its dependencies. (This seems horrible compared to using a package manager.)

  7. keyonghan commented on Jun 28, 2022

    @keyonghan
    keyonghan
    on Jun 28, 2022
    Contributor

    That is correct. Packages that are platform dependent come with separate archives in cipd. One example: https://chrome-infra-packages.appspot.com/p/flutter/device_doctor

  8. keyonghan
    added
    passed secondary triage
    on Sep 22, 2022
  9. Hixie
    removed
    passed secondary triage
    on Jul 8, 2023
  10. auto-submit
    added a commit that references this issue on Sep 14, 2023

    Update plugin_ffi generated file to match FFIgen 9.0.0 (#134614)

    77a5a5d
  11. Mairramer
    added a commit that references this issue on Oct 10, 2023

    Update plugin_ffi generated file to match FFIgen 9.0.0 (flutter#134614)

    acae221
  12. christopherfujino
    added
    triaged-infraTriaged by Infrastructure team
    on Sep 9, 2024
  13. github-project-automation
    added this to [DEPRECATED] Infra Ticket Queueon Sep 9, 2024
  14. github-project-automation
    moved this to New in [DEPRECATED] Infra Ticket Queueon Sep 9, 2024
  15. christopherfujino
    added
    P2Important issues not at the top of the work list
    on Sep 9, 2024

  16. matanlurey commented on Feb 22, 2025

    @matanlurey
    matanlurey
    on Feb 22, 2025
    Contributor

    This is something the Dart team would have to own if it's important, we don't have the resources do this ourselves.

    (If not, feel free to close)

  17. matanlurey
    added
    dependency: dartDart team may need to help us
    and removed
    team-infraOwned by Infrastructure team
    P2Important issues not at the top of the work list
    triaged-infraTriaged by Infrastructure team
    on Feb 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

dependency: dartDart team may need to help us

Type

No type

Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions