There's a bug in vnc4server. When you have gnome installed and you install vnc4server you'll experience it. Pressing 'D' wherever & whenever inside gnome will cause all the windows to minimize.
Change System->Preferences->Keyboard shortcuts; find the shortcut:
"Hide all normal windows and set focus to the desktop"
and change ‘D’ to something else.