Power Shell Web Scraping SSL/TSL Issue
This one liner will ignore SSL certificate errors:
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
Errors regarding self-signed untrusted certificates, mismatching names or expiration will be ignored after this is executed.
Samuel Meddows
A full time IT student in Brisbane Australia @ QUT. I specialize in web development and networking.
Updated on June 15, 2022Comments
-
Samuel Meddows almost 2 years
I am wanting to run a web scraping script on a server.
The current script collects the html on the specified page.
$url = "http://websms" [net.httpWebRequest] $request = [net.webRequest]::create($url) [net.httpWebResponse] $response = $request.getResponse() $responseStream = $response.getResponseStream() $sr = new-object IO.StreamReader($responseStream) $result = $sr.ReadToEnd() $result
This works fine on a typical web page. However I am wanting to run it on a servers admin page which of course requires a login.
I thought before I attempt to login I will try and scrape the login page of the server. Running the above script I get the following result.
Exception calling "GetResponse" with "0" argument(s): "The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel." At C:\temp\web3.ps1:3 char:56 + [net.httpWebResponse] $response = $request.getResponse <<<< () + CategoryInfo : NotSpecified: (:) [], MethodInvocationException + FullyQualifiedErrorId : DotNetMethodException
Any idea of how to get around this issue or perhaps if you could point me a different direction so I can scrape elements from the admin html page of the sever.
Thanks Guys!