What is the different between API functions AllocConsole and AttachConsole(-1)?

Question

Could you please explain me, what is the different between API functions AllocConsole and AttachConsole(-1) ? I mean if AttachConsole gets ATTACH_PARENT_PROCESS(DWORD)-1.

Solution 1

Well, the fundamental difference is:

• AllocConsole() will create a new console (and attach to it)
• AttachConsole( ATTACH_PARENT_PROCESS /* -1 */) will not create a new console, it will attach to the existing console of the parent process.

In the first case you get a whole new console window, in the second case, you use an existing console window.

Of course, if you're already attached to a console (ie., you're a console mode program launched from cmd.exe) there's not much difference - you'll get an error with either API.

Also note that just because you detach from a console doesn't mean the detached console will be useful - for example, if you're a console process launched from a cmd window, that window essentially blocks until your process ends.

Some code to play with:

int main( int argc, char* argv[])
{
    int ch;
    BOOL bResult;

    printf( "default console\n");
    ch = getchar();

    bResult = FreeConsole();
    bResult = AllocConsole();    
    printf( "AllocConsole()\n");
    ch = getchar();

    bResult = FreeConsole();
    bResult = AttachConsole( ATTACH_PARENT_PROCESS);    
    printf( "AttachConsole( ATTACH_PARENT_PROCESS)\n");
    ch = getchar();

    return 0;
}

Solution 2

I don't think there's a function called CreateConsole, but there's AllocConsole.

Assuming that's what you meant, I think the difference is that AttachConsole(ATTACH_PARENT_PROCESS) can return ERROR_INVALID_HANDLE if the parent process doesn't have a console.

Try running this code from both a command prompt and Start -> Run:

#include <windows.h>
#pragma comment ( lib, "user32.lib" )

int main()
{
    BOOL b;
    char msg[1024];

    b = FreeConsole();
    sprintf(msg, "%d", b);
    MessageBox(NULL, msg, "FreeConsole", 0);

    b = AttachConsole(ATTACH_PARENT_PROCESS);
    sprintf(msg, "%d", b);
    MessageBox(NULL, msg, "AttachConsole", 0);

    return 0;
}

When run from a command prompt, two message boxes containing a 1 are displayed, meaning both calls succeeded. When run from Start -> Run, the first box contains 1 and the second contains 0, meaning that only the first call succeeded. The second one fails because explorer.exe (which is the parent of a process launched from Start -> Run) doesn't have a console.

Solution 3

On Windows 7, when you execute cmd.exe, CreateProcess will have the CREATE_NEW_CONSOLE flag, which will allocate a new console rather than being attached to the parent console (which is default behaviour when the PE header contains Subsystem = 3 i.e. IMAGE_SUBSYSTEM_WINDOWS_CUI indicating that it is a console application). This means that AllocConsole will be called before the .exe image entry point in the current process address space.

AllocConsole creates a new conhost.exe instance, which paints the GUI window, handles the mouse and keyboard events and maintains and writes to the the input buffer and maintains and reads from the screen buffer, and when there is a keyboard event, it updates the input buffer. AllocConsole also sets the stdin handle in the ParameterBlock in the cmd.exe process PEB to the console input buffer and the stdout and stderr to the the console pseudohandles, and sets the ConsoleHandle in the ParameterBlockto the PID of theconhost.exe` instance that it is attached to.

cmd.exe is a console application written in C, like diskpart.exe or setx.exe, which displays the command prompt to the screen buffer (stdout of cmd.exe) and reads a command from the stdin of cmd.exe and interprets the key press events to display to stdout as well as determine what command to call and what to display to stdout as a result (which might be a file in the handle in the PEB and not sent to conhost). A command itself has an stdin of either nothing, the read side of an anonymous pipe, a file or the con file.

cmd.exe calls functions like WriteFile which will call WriteConsole (which is an alias of WriteConsoleA) if it is writing to a standard handle (otherwise it calls NtWriteFile). This will initiate an ALPC call to conhost.exe PID in ParameterBlock->ConsoleHandle, passing the console pseudohandle and the buffer to write the result to (or the buffer to read from).

When you execute cmd /c from inside cmd.exe, it creates a child cmd.exe, but does not supply CREATE_NEW_CONSOLE, which results in the child cmd.exe attaching to the same visible console i.e. conhost.exe instance and AttachConsole is called before the entry point. This is not the case with a child created with admin /c (admin version of cmd.exe) if done in a non-elevated cmd.exe, which needs to have a new conhost.exe instance because it has different privileges. Similarly, when starting a program, start supplies CREATE_NEW_CONSOLE, which opens a new conhost.exe for its child process, but call and specifying the program filename + extension as a raw command do not open another console window, but attach to the parent. cmd /c diskpart creates a cmd.exe child that attaches to the console that the parent is attached to and then that child creates its own child diskpart.exe, which attaches to the console that the parent is attached to.

When you execute a GUI application, it is not allocated or attached to a console. If you use AllocConsole within the application, it will create a new conhost.exe console window (which can be hidden by the application, i.e. blender has window > toggle system console, and it does this by getting a handle to the conhost.exe console window and then using using ShowWindow with SW_HIDE on it), but will not create a child process, because it is the console window of the current process, so now there will be 2 windows. You can also instead attach to a console (using AttachConsole) belonging to a process of pid. AttachConsole(-1) attaches to the parent pid. This will use the conhost that that process is attached to.

You cannot AttachConsole if you are already attached to a console without using FreeConsole and you can't AttachConsole to connect to the parent's console if you create the process with DETACHED_PROCESS. CREATE_NO_WINDOW has the same effect as DETACHED_PROCESS in that it does not attach to or allocate a console for the console application but does allow it to attach to the parent console.

Comments

[configurator]

Can this be used in C#.Net to create a console window for an executable compiled as a Windows Program (and not a Console Program)?

[Michael Burr]

@configuratoir - as far as I know you can do this but I think you'll have to use p/invoke to call the Win32 APIs (not a big deal for these APIs) - I'm not sure if there's a framework equivalent.

[abatishchev]

@nobugz - Yes, you can. See more at stackoverflow.com/questions/472282/…

[7vujy0f0hy]

@abatishchev: Nice link to a dead link... not.

[abatishchev]

@7vujy0f0hy oh, I'm sorry that a linked posted 8 years ago doesn't work for you anymore. Never happened before and please rest assured won't happen again. Please accept my sincere apology. P.S. Thanks for the edit. Cheers!

[Joseph Sible-Reinstate Monica]

"you can't AttachConsole to connect to the parent's console if you create the process with DETACHED_PROCESS" Are you sure? I made a little test program that tried this and it seemed to work for me.

[Lewis Kelsey]

stackoverflow.com/a/10753521/7194773 I read it there, didn't ever test it. Maybe you have a different version of Windows. I saw your comment there, will have to investigate.