Repository navigation
Invoke-Webrequest is missing some properties, like .ParsedHtml and .AllElements #2867
Description
Activity
Joey Aiello (@joeyaiello) FYI: Full PowerShell essentially uses Internet Explorer to do the HTML parsing. This is likely to be problematic to replicate on .NET core.
- added this to the milestone
on Feb 28, 2017 Bruce Payette (@BrucePay) I'd still like to investigate after we look at whatever ends up shipping .NET Standard 2.0
What's the best alternative if this is low priority? Any low effort workarounds for parsing HTML in core?
.Net Std 2.0 doesn't solve this. There is another issue where we discussed the need to leverage an OSS html parser rather than rely on a web browser being available (#2867), doing that work should also resolve this one.
- modified the milestones: ,
on May 15, 2017 37 remaining items
Since new Microsoft Edge is based on Chromium we could discover Chromium engine on system and utilize the API.
Reacted by Greg Martin, Andrew Johnson and AnthonyPerrinmdsRan into this today as well.
Name Value
PSVersion 7.0.3
PSEdition Core
GitCommitId 7.0.3
OS Microsoft Windows 10.0.19041
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0`I think Tsubasa (@kamome283) module is a better way with AngleSharp than try to communicate with an external process.
https://github.com/kamome283/AngleParseGitHub
HTML parsing and processing tool for PowerShell. Contribute to kamome283/AngleParse development by creating an account on GitHub.Seems that the community has helped fill in this gap with modules on PowerShellGallery to specifically handle parsing html.
Reacted by Flavien MICHALECZEKReacted by Derrick Berg, BubiBalboa, francisco-lafe, jsbasra and Pete BatardSo, simply put, this Invoke-WebRequest cmdlet is not compatible for PS7 if I developed a script on PS5?
Given that all the online information is mixed and you never know when they're talking about PS5 or PS7.Given that all the online information is mixed and you never know when they're talking about PS5 or PS7.
To be fair the documentation hosted at Microsoft makes it very easy to choose between (Windows) Powershell 5.1 and (say) Powershell 7.1; for example:
https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-7.1
https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-5.1There you can (for example) see different text for
-UseBasicParsingThe Invoke-WebRequest cmdlet sends HTTP and HTTPS requests to a web page or web service. It parses the response and returns collections of links, images, and other significant HTML elements. This cmdlet was introduced in PowerShell 3.0. Beginning in PowerShell 7.0, Invoke-WebRequest supports proxy configuration defined by environment variables. See the Notes section of this article. Important The examples in this article reference hosts in the contoso.com domain. This is a fictitious domain used by Microsoft for examples. The examples are designed to show how to use the cmdlets. However, since the contoso.com sites do not exist, the examples do not work. Adapt the examples to hosts in your environment.The Invoke-WebRequest cmdlet sends HTTP, HTTPS, FTP, and FILE requests to a web page or web service. It parses the response and returns collections of forms, links, images, and other significant HTML elements. This cmdlet was introduced in Windows PowerShell 3.0. Note By default, script code in the web page may be run when the page is being parsed to populate the ParsedHtml property. Use the -UseBasicParsing switch to suppress this. Important The examples in this article reference hosts in the contoso.com domain. This is a fictitious domain used by Microsoft for examples. The examples are designed to show how to use the cmdlets. However, since the contoso.com sites do not exist, the examples do not work. Adapt the examples to hosts in your environment.Reacted by Brad Voriswisemoth did you intentionally make 2 links to the same thing for invoke-webrequest? because both of those links point to the same URL and give the same information (for PowerShell 7,2). If you're trying to say that the information is not mixed then doing this disproves your point...
wisemoth did you intentionally make 2 links to the same thing for invoke-webrequest? because both of those links point to the same URL and give the same information (for PowerShell 7,2). If you're trying to say that the information is not mixed then doing this disproves your point...
You're using the link previews created by github....
Reacted by wisemothHuh, i guess I am... I sit corrected. Why isn't the preview of the complete link? ehhh... automated technology failing us all again! Sorry about that.
Hello Everyone
PsCore 6.x's Invoke-Webrequest is missing on the most 'loved' Ps 'classic' features, ie the ability to parse web pages and turn them into explorable objects.
Steps to reproduce
$test=invoke-webrequest -url http://www.github.com
Expected behavior
$test.ParsedHtml
$test.AllElements
(amont others) are missing:
Microsoft Powershell :
Core Poweshell:
Actual behavior
these properties are missing
Environment data
I have hope to have web parsing capabilities in PSCore!
Thanks everyone!