Repository navigation
Loading Native Libraries is broken #9488
Description
Activity
Aditya Patwardhan (@adityapatwardhan) can you take a look at this?
Reacted by Rain Sallow (/u/ta11ow)You could look used paths with
procmonutility and then try to fix with HintPathI'll give that a look but given this isn't a library I'm loading directly (it's loaded by the main SkiaSharp.dll itself) I'm not that confident in that doing too much. The fact that there are four possible library files to load depending on platform also complicates things.
But regardless, this doesn't seem like an intentional change so the regression should probably be addressed.
MSFT team works hard on vNext p/invokes in .Net Core 3.0 and perhaps they back-ported something to 2.1. I don't remember that we changed anything in PowerShell related dll loading.
There was #8073 and several
Internal-marked changes in 6.2.0-preview2. I have no idea what else may have changed during those. 😕The PR change is to load module dll from module folder first then from GAC. Your problem is that the module dll reference second dll and auto dll loading (it does Core not PowerShell ) doesn't found the dll. My suggestion is to explicitly add hintpath to the dll in project file.
Ilya (@iSazonov) I added some HintPath references to my csproj, but all I get is a warning when I
dotnet publish. There is no change in behaviour of the module once loaded.From a bit of reading up on it, it looks like the HintPath is used to reference external assemblies that aren't included in the project build itself. The SkiaSharp assemblies are included, so adding a HintPath for compile/build doesn't affect the Skia DLLs or the paths they look up at runtime.
For reference, this is my csproj:
<Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> <TargetFramework>netstandard2.0</TargetFramework> </PropertyGroup> <ItemGroup> <PackageReference Include="PowerShellStandard.Library" Version="5.1.1" PrivateAssets="All" /> <PackageReference Include="SkiaSharp" Version="1.68.0" /> <PackageReference Include="SkiaSharp.NativeAssets.Linux" Version="1.68.0" /> </ItemGroup> </Project>
I added the following lines to no effect (other than a build/publish time warning that they can't be found, because, well, they don't exist until after the publish is completed anyway):
<ItemGroup> <Reference Include="libSkiaSharp"> <HintPath>win-x86/libSkiaSharp.dll</HintPath> <HintPath>win-x64/libSkiaSharp.dll</HintPath> <HintPath>osx/libSkiaSharp.dylib</HintPath> <HintPath>linux-x64/libSkiaSharp.dylib</HintPath> </Reference> </ItemGroup>
From what I've read on HintPath, it doesn't seem to have any effect on what paths are searched when loading a DLL, or that DLL tries to load other DLLs for p/invoke purposes.
That HintPath is a compile-time thing only.
Reacted by Rain Sallow (/u/ta11ow)The way you say that makes it sound like there's another I missed?
Rain Sallow (/u/ta11ow) (@vexx32) If you have simple repo code you could ask in CoreCLR repository.
I'm unsure if I can rule out a Powershell-specific change just yet. I might have to put a small test together for a console app and see how that goes.
Thanks to a tip from Joel Bennett (@Jaykul), I've managed to put together a stop-gap solution. Executing this from the PSM1 before importing the cmdlet and the main Skia DLL seems to resolve the issue.
Add-Type -TypeDefinition @" using System.Runtime.InteropServices; public class DllLoadPath { [DllImport("kernel32", CharSet=CharSet.Unicode)] public static extern int SetDllDirectory(string NewDirectory); } "@ # ... [DllLoadPath]::SetDllDirectory($NativeRuntimeFolder)
However, the fact that it requires additional p/invoke to make this work is kind of ridiculous. I'm also very confused as to what changed and removed the loaded SkiaSharp.dll's own directory path from the standard search locations as documented here.
Reacted by Flavien MICHALECZEK53 remaining items
Currently the folder structure of PSWordCloud is not general, no. That can be changed, of course.
Yeah, I can absolutely put together a simple test bed for that. Pester is a bit tricky to do, just based on the nature of the output I suppose, but I think when it fails it produces an exception you can catch, though that exception is thrown from the type initializer if I recall correctly.
Would it be appropriate to have the test itself pull the Skia DLLs via dotnet, or would it be better to package them in the repo? I guess if we pull them each time it's not so big a deal if it's flagged as a feature test once it's no longer pending? 🤔
If you could implement a test nodule with indirect dependence with Skia dlls it would be nice. And yes we could install Skia from nuget. Although the test would be more reliable if we directly put the dlls with the test module in
our test module folderan assert folder.
It seems Nuget best practice is to put native dlls in RID named folders. Yes?Reacted by Rain Sallow (/u/ta11ow)Yep, I can do that! I'll have a look at it tommorow. 😊
Reacted by IlyaI updated my comment - please put the test module in an test assert folder.
Reacted by Rain Sallow (/u/ta11ow)Rob Holt (@rjmholt) I made some attempts to use the code you provided to make it work for PSWordCloud, but it's still not working at all. The load operation completes successfully, but loading the managed SkiaSharp.dll after the unmanaged library still fails to make use of the loaded DLL on Unix platforms. It does manage to work for Windows, though, somehow.
See my WIP branch here https://github.com/vexx32/PSWordCloud/tree/FixPInvokes -- run
build.ps1to build the module and put all the files in the right places. It'll import the module for you, too. CallingNew-WordCloudafter running the build works on Windows, but fails completely on Mac OS.Patrick Meinecke (@SeeminglyScience) you see anything I did wrong there? :/
GitHub
Create pretty word clouds with PowerShell! Contribute to vexx32/PSWordCloud development by creating an account on GitHub.Steve Lee (@SteveL-MSFT) what's the plan for handling native libs at present? 😕
Attempting to load a module that requires p/invoke from a third-party native library fails to load the library from the same folder as the DLL doing the p/invoking.
This used to work in PowerShell Core 6.1.0 but regressed between 6.2.0-preview1 and 6.2.0-preview2. It still works in Windows PowerShell.
The native library it's failing to locate is located in the same folder as the SkiaSharp.dll that is trying to locate it. See #8861 (comment) and that issue thread for why it was done in that fashion.
Steps to reproduce
Expected behavior
Actual behavior
Querying for the base exception with
$Error[0].Exception.GetBaseException() | Format-List * -Fyields the following:Environment data
Note: The same was tested in a Mac environment (thanks Stephen Valdinger (@steviecoaster)!) and also failed in the same way. I suspect it will be identical on Linux distros as well.
/cc Steve Lee (@SteveL-MSFT)