【转载】Disabling Escape in modal dialog boxes

2008-4-6 Nie.Meining Coding

Today I spent a lot of time on a seemingly simple task. I’m writing a tool I need for Jayden using the Win32 API. In general, I quite like the Win32 API, but I ran into an annoying quirk today. I finally found the solution and I’m documenting it here to help other people that might run into the same problem and to make sure I don’t have to go through all this trouble again if I run into this problem in the future.

What I want sounds quite simple: I want to make it impossible for the user to close my modal dialog box by pressing Escape. Normally, this would break expected behaviour, but if you’re using a dialog box as your main window, it actually makes sense. I’ll go over the procedure to do this step by step. Don’t leave before we get to multiline edit controls, though, because that’s where it really gets weird.

The behaviour of the Escape key

When you press Escape in a dialog box, the default behaviour is to ignore all changes you made and close the dialog box. The Win32 API facilitates this behaviour by sending a WM_COMMAND to your dialog window with wParam set to IDCANCEL. You can handle this message any way you want, but typically you would call EndDialog() to close the dialog box. If you don’t want to close the dialog box when the user presses Escape, you just ignore WM_COMMAND when wParam is set to IDCANCEL. Simple.

Adding a Cancel button

Most dialog boxes contain a Cancel button, so the user can cancel the dialog box using the mouse instead of the keyboard. Chances are that you want a click on the Cancel button and a press on the Escape key to behave exactly the same. The easiest way make this happen, is to give your Cancel button an id of IDCANCEL. This means that whenever the user clicks Cancel, your dialog window will receive a WM_COMMAND message with wParam set to IDCANCEL. Sound familiar? Indeed, it’s the same message you get when the user presses Escape.

If you want the Cancel button to behave differently from Escape, it’s possible to do that, but since that’s a Bad Idea I’m not going to describe it. Either you have a Cancel button and pressing Escape is equivalent to clicking on it, or you ignore Escape and you don’t have a Cancel button. For an application that uses a dialog box as its main window, the latter is a logical choice.

The WM_CLOSE message

At this point, I’ll tell you a bit about the WM_CLOSE message, because we are going to need this information later on. The WM_CLOSE message is sent to your (dialog) window whenever your window needs to close. I can think of three user actions that result in a WM_CLOSE message:

The user clicks the X in the upper-right corner of the window.
The user choses Close from the system menu.
The user presses Alt-F4.
There are more reasons your window might receive a WM_CLOSE message, but they are not the direct result of an action taken by the user.

The WM_CLOSE message tells you that you should close the window, but it doesn’t do it for you. You still need to call DestroyWindow() or EndDialog() to do that. With a modal dialog box, you call EndDialog() to close it, so a typical, bare-bones DialogProc looks something like this.

static INT_PTR CALLBACK DialogProc(HWND dialog,

    UINT message, WPARAM wParam, LPARAM lParam)

{

    switch (message)

    {

    case WM_CLOSE:

        EndDialog(dialog, 0);

        return TRUE;

    }

 

    return FALSE;

}


Of course, you can do all kinds of stuff in response to WM_CLOSE. For example, you can ask the user if she’s really sure she wants to close the window. Or you can warn her about unsaved data. You can even ignore the message altogether. Now, that’s a stupid thing to do. Most of the time…

In comes the multiline edit control

So far, I didn’t tell you anything extraordinary. Everything I described is standard Windows stuff and well documented. When we add a multiline edit control to the mix, however, things start getting quirky and the documentation stays silent. Add a multiline edit control to your dialog box, set focus to the edit control, press Escape and see what happens. Well, the dialog box closes, even if you don’t handle WM_COMMAND + IDCANCEL! Now that is quirky.

For some reason, whenever you press Escape in a multiline edit control it sends a WM_CLOSE message to the dialog window without sending a WM_COMMAND + IDCANCEL message. If you have special code for when the user cancels the dialog box, you’ll be unpleasantly surprised by this, because that code won’t run, even though the user did cancel the dialog box by pressing Escape. You can move all that code to the handler for WM_CLOSE, of course. That’s okay if you want to treat the X button, Alt-F4 and the Close option from the system menu like a click on the Cancel button, but if you don’t, you have a problem.

Also, this makes it bloody hard to make sure Escape doesn’t close your dialog box. Just ignoring WM_COMMAND + IDCANCEL isn’t enough anymore. You can’t even be certain that the WM_CLOSE message is really a request to close your window and that is really screwed up.

For your information, both the regular edit control and the rich edit control have this problem. Set them to multiline mode and things go awry.

The solution

After quite a bit of experimenting, I found out that the behaviour of the multiline edit controls returns back to normal when you remove your WM_CLOSE handler. That’s right, just don’t handle the WM_CLOSE message at all and the multiline edit control doesn’t just stop sending you the WM_CLOSE message when the user presses Escape, it also gives you the expected WM_COMMAND + IDCANCEL message again. Now, that is truly weird. How does the edit control even know the handler isn’t there? Does it inspect my code?

Now that we have the multiline edit controls behaving normally again, we are stuck with another problem: we still need to close the window when the user asks us to. In other words, we need to respond to the three user actions I mentioned above: the X in the upper-right corner, the Close option in the system menu and Alt-F4. Fortunately, there is another way to respond to those actions: the WM_SYSCOMMAND message.

Whenever one of the described actions occurs, your window will receive a WM_SYSCOMMAND with wParam set to SC_CLOSE. We can now solve our problems by changing the previous code to this:

static INT_PTR CALLBACK DialogProc(HWND dialog,

    UINT message, WPARAM wParam, LPARAM lParam)

{

    switch (message)

    {

    case WM_SYSCOMMAND:

        if (wParam == SC_CLOSE)

        {

            EndDialog(dialog, 0);

            return TRUE;

        }

 

        break;

    }

 

    return FALSE;

}

 

A word of warning

Although we have taken care of all user actions that lead to a WM_CLOSE message, we ignore all other causes. For example, your window also receives a WM_CLOSE message when you try to stop your application by selecting End Task in the Task Manager. Since you are not handling the WM_CLOSE message, nothing happens. Eventually, the Task Manager will show you a dialog box telling you that the application is not responding. (Interestingly enough, using the Task Manager to close your application also lead to a WM_COMMAND + IDCANCEL message.)

There might be lots of other reasons for Windows to send you a WM_CLOSE message, but if you use the above code, you ignore them all. I haven’t run into trouble yet, but you should be aware of this problem when you decide to use the code I presented here.

An alternative solution

There’s another way to deal with the misbehaving multiline edit controls. You can subclass them, intercept the press of Escape and then send the WM_COMMAND + IDCANCEL manually. This way, you can still handle WM_CLOSE normally. It is a bit more cumbersome, however. Subclassing is described in the Win32 API documentation, but I’ll show you how to do it, just to be complete.

Subclassing a control basically means that you can intercept any message that is send to that control before it reaches the control. All messages you don’t want to intercept, you just send through to the control. There are two ways to do subclassing: the old way and the new way. The old way works on all versions of Windows, but you can’t use it of you already subclassed the control for other purposes or if you want to share a subclass between several controls. The new way allows to install as much subclasses as you want to multiple controls, but it only works on Windows XP. O, those trade-offs. I’m going to show you the old way.

First thing we need to do is write a new window procedure for the control, which I will call ControlProc. This window procedure handles all messages we are interested in (WM_KEYDOWN in our case) and sends all other messages to the control’s own window procedure by calling CallWindowProc().

static LRESULT CALLBACK ControlProc(HWND control,

    UINT message, WPARAM wParam, LPARAM lParam)

{

    switch (message)

    {

    case WM_KEYDOWN:

        // do something

        break;

    }

 

    // call the control's own window procedure

    return CallWindowProc(OldControlProc, control,

        message, wParam, lParam);

}

 

Now we need to tell Windows that we want to use our window procedure instead of the control’s own window procedure. We need to do that as soon as the dialog box is created, so we handle WM_INITDIALOG in the dialog’s window procedure.

static WNDPROC OldWindowProc;

 

static INT_PTR CALLBACK DialogProc(HWND dialog,

    UINT message, WPARAM wParam, LPARAM lParam)

{

    switch (message)

    {

    case WM_INITDIALOG:

        // get the window handle of the multiline

        // edit control

        // TODO: replace IDC_EDIT with the actual id

        // of your edit control

        HWND hwnd_edit = GetDlgItem(dialog, IDC_EDIT);

 

        // subclass the edit control

        OldWindowProc = (WNDPROC) SetWindowLong(

            hwnd_edit, GWL_WNDPROC, (LONG) ControlProc);

 

        return TRUE;

    }

 

    return FALSE;

}

 

Of course, we still need to fill in the code to handle the WM_KEYDOWN message. WM_KEYDOWN is sent to the edit control whenever the user presses a key while the control has focus. wParam contains the virtual-key code of the key that has been pressed and the virtual -key code for Escape is VK_ESCAPE. When we detect a press of Escape, we should respond by sending the WM_COMMAND message to the dialog window with wParam set to IDCANCEL. Here goes.

static LRESULT CALLBACK ControlProc(HWND control,

    UINT message, WPARAM wParam, LPARAM lParam)

{

    switch (message)

    {

    case WM_KEYDOWN:

        if (LOWORD(wParam) == VK_ESCAPE)

        {

            HWND hwnd_parent = GetAncestor(control, GA_PARENT);

            SendMessage(hwnd_parent, WM_COMMAND, IDCANCEL,

                (LPARAM) control);

 

            return 0;

        }

 

        break;

    }

 

    // call the control's own window procedure

    return CallWindowProc(OldControlProc, control, message,

        wParam, lParam);

}

 

And there you have it, two solutions worked out for you. I find the WM_SYSCOMMAND solution easier to deal with, but it does suffer from not responding to WM_CLOSE. In that respect, subclassing is a better solution, but it gets tricky when you have more than one multiline edit control and don’t want to give up support for previous versions of Windows.

Hopefully, this will be helpful to someone, because it took me the better part of the day to figure all this out and write about it.

发表评论:

Powered by emlog