The Robot class - How to move the mouse cursor and type keys automatically from your Java program?

Introduction

This article demonstrates the various functionality offered by the java.awt.Robot class. From the Java API documentation "This class is used to generate native system input events for the purposes of test automation, self-running demos, and other applications where control of the mouse and keyboard is needed. The primary purpose of Robot is to facilitate automated testing of Java platform implementations.

Using the class to generate input events differs from posting events to the AWT event queue or AWT components in that the events are generated in the platform's native input queue. For example, Robot.mouseMove will actually move the mouse cursor instead of just generating mouse move events.

Note that some platforms require special privileges or extensions to access low-level input control. If the current platform configuration does not allow input control, an AWTException will be thrown when trying to construct Robot objects. For example, X-Window systems will throw the exception if the XTEST 2.2 standard extension is not supported (or not enabled) by the X server.

Applications that use Robot for purposes other than self-testing should handle these error conditions gracefully. "

Application

• Automated testing: As the Java API docs said, the primary intended usage for this class is to support automated testing. For example your Java program could do certain things on a mouse click on a JButton or a key release in a JTextField. When it comes to automated testing of this Java program, you can use this Robot class to simulate these user actions in a way that will be as close as possible to a real user action.

• Demonstration: May be you want to demonstrate a particular GUI app which involves user interactions...

Features

The following are the key features/functionality offered by this Robot class.

• Simulate/perform the following mouse events.
  • Mouse Move - Moves the mouse pointer to a given screen's coordinates.
  • Mouse press - Presses the specified mouse button at the mouse pointer's current location.
  • Mouse release - Releases the specified mouse button at the mouse pointer's current location.
  • Mouse wheel - Rotates the mouse scroll wheel to a given amount.
• Simulate/perform the following keyboard events.
  • Key Press - Press a specified key in the keyboard.
  • Key Release - Release a specified key in the keyboard.
• Simulate/perform a screen capture - "Creates an image containing pixels read from the screen. This image does not include the mouse cursor."
• Get pixel color - Gets the color of the pixel at a given screen co-ordinate.

Usage

• Instantiation: The first step to do in order to use this class's features to instantiate it. Instantiation is straight forward and you can use the default parameterless constructor.

Robot robot = new Robot();

• Simulate Mouse move: To move the mouse over a text field - textField

Point locOnScreen = textField.getLocationOnScreen();
robot.mouseMove(locOnScreen.x, locOnScreen.y);

• Simulate Mouse button press and release at the current mouse pointer position

// Press and release mouse button 1
    robot.mousePress(InputEvent.BUTTON1_MASK);
    robot.mouseRelease(InputEvent.BUTTON1_MASK);
 
// Press and release mouse button 2
    robot.mousePress(InputEvent.BUTTON2_MASK);
    robot.mouseRelease(InputEvent.BUTTON2_MASK);

• Simulate Keyboard key presses and releases.

// Types the word 'DEMO'
    robot.keyPress(KeyEvent.VK_D);
    robot.keyRelease(KeyEvent.VK_D);
 
    robot.keyPress(KeyEvent.VK_E);
    robot.keyRelease(KeyEvent.VK_E);
 
    robot.keyPress(KeyEvent.VK_M);
    robot.keyRelease(KeyEvent.VK_M);
 
    robot.keyPress(KeyEvent.VK_O);
    robot.keyRelease(KeyEvent.VK_O);

Note: These methods could be seen as dumb methods which does exactly what you asked it to do. For example if you try to simulate a key press for the character ':' - colon like

robot.keyPress(KeyEvent.VK_COLON)

and in you keyboard the only way you can manually type a colon is to hold shift and press semi-colon ';', then the above line would throw an IllegalArgumentException. The correct way to simulate these characters is to exactly do what can be done with the connected keyboard i.e.

robot.keyPress(KeyEvent.VK_SHIFT);
robot.keyPress(KeyEvent.VK_SEMICOLON);
robot.keyRelease(KeyEvent.VK_SEMICOLON);
robot.keyRelease(KeyEvent.VK_SHIFT);