Repository navigation
Invoke-Webrequest is missing some properties, like .ParsedHtml and .AllElements #2867
Description
Hello Everyone
PsCore 6.x's Invoke-Webrequest is missing on the most 'loved' Ps 'classic' features, ie the ability to parse web pages and turn them into explorable objects.
Steps to reproduce
$test=invoke-webrequest -url http://www.github.com
Expected behavior
$test.ParsedHtml
$test.AllElements
(amont others) are missing:
Microsoft Powershell :
Core Poweshell:
Actual behavior
these properties are missing
Environment data
CLRVersion
SerializationVersion 1.1.0.1
PSRemotingProtocolVersion 2.3
PSEdition Core
WSManStackVersion 3.0
BuildVersion 3.0.0.0
GitCommitId v6.0.0-alpha.13
PSVersion 6.0.0-alpha
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
I have hope to have web parsing capabilities in PSCore!
Thanks everyone!
Activity
BrucePay commentedon Feb 18, 2017
Joey Aiello (@joeyaiello) FYI: Full PowerShell essentially uses Internet Explorer to do the HTML parsing. This is likely to be problematic to replicate on .NET core.
joeyaiello commentedon Mar 1, 2017
Bruce Payette (@BrucePay) I'd still like to investigate after we look at whatever ends up shipping .NET Standard 2.0
jakerobinson commentedon May 3, 2017
What's the best alternative if this is low priority? Any low effort workarounds for parsing HTML in core?
SteveL-MSFT commentedon May 3, 2017
.Net Std 2.0 doesn't solve this. There is another issue where we discussed the need to leverage an OSS html parser rather than rely on a web browser being available (#2867), doing that work should also resolve this one.
37 remaining items
iSazonov commentedon Mar 14, 2020
Since new Microsoft Edge is based on Chromium we could discover Chromium engine on system and utilize the API.
jdelforno commentedon Jul 28, 2020
Ran into this today as well.
Name Value
PSVersion 7.0.3
PSEdition Core
GitCommitId 7.0.3
OS Microsoft Windows 10.0.19041
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0`
fMichaleczek commentedon Jul 28, 2020
I think Tsubasa (@kamome283) module is a better way with AngleSharp than try to communicate with an external process.
https://github.com/kamome283/AngleParse
SteveL-MSFT commentedon Oct 19, 2020
Seems that the community has helped fill in this gap with modules on PowerShellGallery to specifically handle parsing html.
francisco-lafe commentedon Sep 13, 2021
So, simply put, this Invoke-WebRequest cmdlet is not compatible for PS7 if I developed a script on PS5?
Given that all the online information is mixed and you never know when they're talking about PS5 or PS7.
wisemoth commentedon Sep 13, 2021
To be fair the documentation hosted at Microsoft makes it very easy to choose between (Windows) Powershell 5.1 and (say) Powershell 7.1; for example:
https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-7.1
https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-5.1
There you can (for example) see different text for
-UseBasicParsingserialhex commentedon Dec 12, 2021
wisemoth did you intentionally make 2 links to the same thing for invoke-webrequest? because both of those links point to the same URL and give the same information (for PowerShell 7,2). If you're trying to say that the information is not mixed then doing this disproves your point...
francisco-lafe commentedon Dec 13, 2021
You're using the link previews created by github....
serialhex commentedon Dec 13, 2021
Huh, i guess I am... I sit corrected. Why isn't the preview of the complete link? ehhh... automated technology failing us all again! Sorry about that.