History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: SRC-183
Type: New Feature New Feature
Status: Resolved Resolved
Resolution: Fixed
Priority: Critical Critical
Assignee: Dan Fabulich
Reporter: Dan Fabulich
Votes: 3
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
Selenium Remote Control

Add support for sending OS-level keystroke commands with java.awt.Robot

Created: 15/Nov/06 12:47 PM   Updated: 26/Nov/08 03:52 PM
Component/s: Server
Affects Version/s: None
Fix Version/s: 1.0 beta 1

Issue Links:
Relationship
This issue Relates to:
SRC-377 Add support for typeKeysNative Major Open
 


 Description  « Hide
This would allow us to work around SEL-63 (support for <input type="file">). While we're at it, we might even add in support for taking screenshots...?

 All   Comments   Change History      Sort Order:
Dan Fabulich - 22/Nov/06 01:03 PM
This would also be nice for clicking on browser security pop-ups and bringing up the browser's context menu.

Jason Huggins - 23/Jul/07 11:21 PM
How about this for an implementation?

1) Use the Robot library built into Java (java.awt.Robot) to send OS-level keystrokes and mouse events
2) Add a method to toggle the use of the Robot library. For example:
se.useNativeKeyboardAndMouse("true") /// default is "false"... or boolean (1 or 0), perhaps?
3) Modify the "typeKeys" method to use Robot when useNativeKeyboardAndMouse is set to true. For example,
se.typeKeys("", "The string to enter")

Note: If the locator argument is an empty string (""), then it is assumed it will just "blindly" type the keyboard events. The keys will be sent (though only incidentally) to the window that has the focus. In the future, if Selenium ever gains native OS widget recognition, perhaps we could properly do something like this:


Jason Huggins - 24/Jul/07 02:34 PM
Hmmm... "KeyboardAndMouse" are a bit wordy...

How about:
se.setNativeInput("true") // or "false"
se.getNativeInput() // returns "true" or "false"

and for typeKeys, how about:
se.typeKeys("window","{ENTER}")

The "window" locator is already a valid dom expression, so that's nice... Also, I'd like to reuse Microsoft's "SendKeys" API for the syntax of sending keys. Read here for more info: http://msdn2.microsoft.com/en-us/library/8c6yea83.aspx


Dan Fabulich - 24/Oct/07 03:35 PM
I checked in a basically working version of this in revision 2061. I didn't follow Jason's suggestion re: the formatting of the commands because that would require coordinating state between the JS browser and the server, making the implementation much more complicated.

Instead, I implemented separate keyDownNative, keyUpNative and keyPressNative commands that do not accept element locators, and require you to pass in integer java.awt.event.KeyEvent keycodes. http://java.sun.com/javase/6/docs/api/java/awt/event/KeyEvent.html

These commands do not support the regular shiftKeyDown, altKeyDown, metaKeyDown and controlKeyDown commands; instead you have to separately send a keyDownNative(KeyEvent.VK_SHIFT) to hold down shift in a "native" way. You also have to manually focus on the element first using the "focus" command.

Finally, I did not implement typeKeysNative; I filed that separately as SRC-377, because it's a lot harder than it looks to convert a given string into native key events.


Andras Hatvani - 26/Nov/08 03:52 PM
I think the focus function makes the test case TestMouseEvents fail:
Expected: '{mouseover(theTextbox)} {mouseover(theButton)} {mousedown(theTextbox ctrlKeyDown)} {mousedown(theButton)} {focus(theButton)} {contextmenu(theButton)} {mousedown(theButton)}'

Actual: '{mouseover(theTextbox)} {mouseover(theButton)} {mousedown(theTextbox ctrlKeyDown)} {mousedown(theButton)} {focus(theButton)} {contextmenu(theButton)} {focus(theButton)} {mousedown(theButton)}' <-- the command before the last will now be recorded as event.

Is this the correct behavior and only the test case has to be adapted?