How to use WndProc as a class function

Question

I'm trying to create a class that includes the WndProc, but I'm getting an error :

Error 2 error C2440: '=' : cannot convert from 'LRESULT (__stdcall Client::* )(HWND,UINT,WPARAM,LPARAM)' to 'WNDPROC'

I searched the web for it, and seen that you need to make the WndProc static, but then, it compiles and everything is great, though if I want to change something, it doesnt let me :

Error 3 error C2352: 'Client::CreateMen' : illegal call of non-static member function

(CreateMen is a function in the class that creates the menu, using HMENU and such).

this is my function title:

LRESULT CALLBACK WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam);

What can I do? I'm really confused...

Thanks!

Solution 1

A non-static class method has a hidden this parameter. That is what prevents the method from being used as a WndProc (or any other API callback). You must declare the class method as static to remove that this parameter. But as you already noticed, you cannot access non-static members from a static method. You need a pointer to the object in order to access them.

In the specific case of a WndProc callback, you can store the object pointer in the HWND itself (using either SetWindowLong/Ptr(GWL_USERDATA) or SetProp()), then your static method can retrieve that object pointer from the hWnd parameter (using GetWindowLong/Ptr(GWL_USERDATA) or GetProp()) and access non-static members using that object pointer as needed. For example:

private:
    HWND m_Wnd;
    static LRESULT CALLBACK WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam);

LRESULT CALLBACK Client::WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
{
    Client *pThis;

    if (msg == WM_NCCREATE)
    {
        pThis = static_cast<Client*>(reinterpret_cast<CREATESTRUCT*>(lParam)->lpCreateParams);

        SetLastError(0);
        if (!SetWindowLongPtr(hwnd, GWL_USERDATA, reinterpret_cast<LONG_PTR>(pThis)))
        {
            if (GetLastError() != 0)
                return FALSE;
        }
    }
    else
    {
        pThis = reinterpret_cast<Client*>(GetWindowLongPtr(hwnd, GWL_USERDATA));
    }

    if (pThis)
    {
        // use pThis->member as needed...
    }

    return DefWindowProc(hwnd, msg, wParam, lParam);
}

m_Wnd = CreateWindowEx(..., this);

Solution 2

Unfortunately you cannot use a class function as a wndproc because as the compiler tries to tell you the calling convention differs, even though the two functions have the same signature, a class function expects the this pointer to be passed to it. On 64 bit builds it will expect it to be in the RCX/ECX registry while on 32 bit builds it will expect the this pointer to be the last argument pushed on the stack. The window code won't do that when calling your WndProc essentially turning this into a function call on a garbage pointer.

What you can do is make a static method that does something like the following:

LRESULT Client::CreateMen(HWND hwnd, UINT msg, WPARAM wparam, LPARAM lparam)
{
    // The OS makes sure GWLP_USERDATA is always 0 before being initialized by the application
    Client* client = (Client*)GetWindowLongPtr(hwnd, GWLP_USERDATA);

    if(msg == WM_INIT)
    {
        client = new Client();
        SetWindowLongPtr(hwnd, GWLP_USERDATA, client);
    }

    if(msg == WM_DESTROY)
    {
        client = (Client*)GetWindowLongPtr(hwnd, GWLP_USERDATA, client);
        SetWindowLongPtr(hwnd, GWLP_USERDATA, NULL);
        delete client;
        client = NULL;
    }

    if(client)
    {
        // Do stuff with the client instance
    }

    return DefWindowProc(hwnd, msg, wparam, lparam);
}

I haven't tested this, so it might have some bugs, but let me know if you have any problems with it and I'll refine it if need be.

Comments

[Remy Lebeau]

Why have a Client object (that is having CreateMen()` called on it) create a second Client object? You can pass the original Client object to CreateWindow/Ex() directly and then have the WndProc call SetWindowLong() while the window is being created. That way, GWL_USERDATA is ready for subsequent messages, and you don't need to use a custom INIT message.

[David Heffernan]

The this pointer doesn't come through that register for a stdcall function

[Radu Chivu]

@RemyLebeau CreateMen is not called on a Client object, it's the implementation for the static window procedure. I thought it was better to keep the object inside the window procedure so external code doesn't touch it.

[Radu Chivu]

@DavidHeffernan Thanks for the correction, I've spent more time debugging 64 bit code than 32, so I was under the impression that the same convention of passing arguments in registers whenever possible applied, I'm going to correct that in my post, thanks.

[Amit]

Thanks! it worked very well!

[Admin]

NOTE : put WndProc in public area not private because you can't use private member as callback input in functions :)

[Remy Lebeau]

@PinnedObject private works just fine in my example, there is no need for it to be public.

[Admin]

@RemyLebeau can you provide your code to see how you use it? I get E0265 function "Class::WndProc" is inaccessible in vs2019, AFAIK it only works if window and window class be created in class functions itself not outside.

[Remy Lebeau]

@PinnedObject Yes, the window class that is passed to CreateWindowEx() has to be registered inside of Client in order for a private WndProc to work. It doesn’t make much sense to pass this to CreateWindowEx() and register the window class outside of Client.

[Admin]

@RemyLebeau Yes, you're right but I don't mean using CreateWindowEx, You only register a WindowClass once not every time in each class and WindowClass needs WndProc and this is where issues comes up and there's two solution to this, making WndProc public or delegate it using a fake WndProc

[Remy Lebeau]

@PinnedObject if you are using a per-class WndProc, it makes sense to register it in the same class that uses it. Then private works fine. If you register outside of the class then of course the WndProc will have to be made accessible to outside code, via public or friend, etc.

[IOviSpot]

pThis returns nullptr...

[Remy Lebeau]

@IOviSpot when used properly, it will only be nullptr until WM_NCCREATE is received (it is not the first message received) and then afterwards it should not be nullptr anymore.