Download URL Contents Directly into String (VB6) WITHOUT Saving to Disk

Question

Basically, I want to download the contents of a particular URL (basically, just HTML codes in the form of a String) into my VB6 String variable. However, there are some conditions.

I know about the URLDownloadToFile Function - however, this requires that you save the downloaded file/HTML onto a file location on disk before you can read it into a String variable, this is not an option for me and I do not want to do this.

The other thing is, if I need to use an external library, it must already come with all versions of Windows from XP and onwards, I cannot use a control or library that I am required to ship, package and distribute even if it is free, this is not an option and I do not want to do this. So, I cannot use the MSINET.OCX (Internet Transfer) Control's .OpenURL() function (which simply returns contents into a String), as it does not come with Windows.

Is there a way to be able to do this with the Windows API, URLMON or something else that is pre-loaded into or comes with Windows, or a way to do it in VB6 (SP6) entirely?

If so, I would appreciate direction, because even after one hour of googling, the only examples I've found are references to URLDownloadToFile (which requires saving on disk before being ale to place into a String) and MsInet.OpenURL (which requires that I ship and distribute MSINET.OCX, which I cannot and don't want to do).

Surely there has got to be an elegant way to be able to do this? I can do it in VB.NET without an issue, but obviously don't have the luxury of the .NET framework in VB6 - any ideas?

Update:

I have found this: http://www.freevbcode.com/ShowCode.asp?ID=1252 however it says that the displayed function may not return the entire page and links to a Microsoft bug report or kb article explaining this. Also, I understand this is based off wininet.dll - and I'm wondering which versions of Windows does WinInet.dll come packaged with? Windows XP & beyond? Does it come with Windows 7 and/or Windows 8?

Solution 1

If you want to do this with pure VB, and no IE, then you can take advantage of a little-used features of the VB UserControl - async properties.

Create a new UserControl, and call it something like UrlDownloader. Set the InvisibleAtRuntime property to True. Add the following code to it:

Option Explicit

Private Const m_ksProp_Data         As String = "Data"

Private m_bAsync                    As Boolean
Private m_sURL                      As String

Public Event AsyncReadProgress(ByRef the_abytData() As Byte)
Public Event AsyncReadComplete(ByRef the_abytData() As Byte)

Public Property Let Async(ByVal the_bValue As Boolean)
    m_bAsync = the_bValue
End Property

Public Property Get Async() As Boolean
    Async = m_bAsync
End Property

Public Property Let URL(ByVal the_sValue As String)
    m_sURL = the_sValue
End Property

Public Property Get URL() As String
    URL = m_sURL
End Property

Public Sub Download()

    UserControl.AsyncRead m_sURL, vbAsyncTypeByteArray, m_ksProp_Data, IIf(m_bAsync, 0&, vbAsyncReadSynchronousDownload)

End Sub

Private Sub UserControl_AsyncReadComplete(AsyncProp As AsyncProperty)

    If AsyncProp.PropertyName = m_ksProp_Data Then
        RaiseEvent AsyncReadComplete(AsyncProp.Value)
    End If

End Sub

Private Sub UserControl_AsyncReadProgress(AsyncProp As AsyncProperty)

    If AsyncProp.PropertyName = m_ksProp_Data Then
        Select Case AsyncProp.StatusCode
        Case vbAsyncStatusCodeBeginDownloadData, vbAsyncStatusCodeDownloadingData, vbAsyncStatusCodeEndDownloadData
            RaiseEvent AsyncReadProgress(AsyncProp.Value)
        End Select
    End If

End Sub

To use this control, stick it on a form and use the following code:

Option Explicit

Private Sub Command1_Click()

    XDownload1.Async = False
    XDownload1.URL = "http://www.google.co.uk"
    XDownload1.Download

End Sub

Private Sub XDownload1_AsyncReadProgress(the_abytData() As Byte)

    Debug.Print StrConv(the_abytData(), vbUnicode)

End Sub

Suffice to say, you can customise this to your hearts content. It can tell (using the AyncProp object) whether the file is cached, and other useful information. It even has a special mode in which you can download GIF, JPG and BMP files and return them as a StdPicture object!

Solution 2

This is how I did it with VB6 a few years ago:

Private Function GetHTMLSource(ByVal sURL As String) As String
Dim xmlHttp As Object
    Set xmlHttp = CreateObject("MSXML2.XmlHttp")
    xmlHttp.Open "GET", sURL, False
    xmlHttp.send
    GetHTMLSource = xmlHttp.responseText
    Set xmlHttp = Nothing
End Function

Solution 3

One alternative is using Internet Explorer.

Dim ex As InternetExplorer
Dim hd As HTMLDocument
Dim s As String

Set ex = New InternetExplorer

With ex
    .Navigate "http://donttrack.us/"
    .Visible = 1
    Set hd = .Document
    s = hd.body.innerText ' assuming you just want the text
    's = hd.body.innerHTML ' if you want the HTML
End With

EDIT: For the above early binding to work you need to set references to "Microsoft Internet Controls" and "Microsoft HTML Object Library" (Tools > References). You could also use late binding, but to be honest, I forget what the proper class names are; maybe someone smart will edit this answer :-)

Comments

[Siddharth Rout]

+ 1. I recommend xmlHttp as well :)

[Siddharth Rout]

+ 1 Another nice way. You might want to switch over to latebinding or mention that the reference needs to be added to IE?

[Erx_VB.NExT.Coder]

Thanks heaps antagony, this is for sure be useful to many. upvoted.

[Erx_VB.NExT.Coder]

I just wanted to say, this is very interesting, here I am thinking I know VB6 inside out and wam - you hit me with this awesome solution. Just a question, I can just include this as a User Control file in my project right? Don't have to compile and reference it as a dll or ocx? (thanks again, upvoted)

[Erx_VB.NExT.Coder]

thanks mate, this is another good idea, despite my years of experience with the webbrowser control, this never even occured to me for some reason :). Also wanted to add that your method of using Internet Explorer instead of the WebBrowser control is a better idea because WB control will use IE7 or IE4 depending on which versions you have installed, and doesnt use latest installed IE version. But, when you use IE directly, you're always using the latest IE version installed on the client computer. Thanks again, upvoted.

[Mark Bertenshaw]

Yes, that's correct. The UserControl can just be set as Public = False. Just a note - despite what I mentioned above, this is probably reliant on components that come with Internet Explorer. If you look at the VB6 release notes, it requires a minimum of Internet Explorer 5. The cache it refers to is probably Temporary Internet Files. Perhaps I should have said "references-free" :-) .

[Antagony]

I also found this very interesting, but I couldn't get it to work as it stands. It was throwing a type mismatch error on AsyncProp.Value on the RaiseEvent line. To get it to work I had to change the event arguments to variants and drop the array parentheses, both there and in the Debug.Print line. It worked then though, so +1 very good. :-)

[Mark Bertenshaw]

Well, that's odd. I've taken the code from the page above, and followed my own instructions - but it seems to work without modification.

[Antagony]

Well I've just tried it again, this time in an XP VM machine I keep up to date for testing and it's exactly the same. Could we be running slightly different versions of VBRUN maybe? I'm running VB6/SP6 with MSVBVM60.dll version 6.00.98.15

[Erx_VB.NExT.Coder]

I will mark this as the answer because it has the least dependants or referencing needed, I'm hoping it works on WinXP pre-SP2?