Automate Screen or Button Taps via Tasker : Simulating keypress events

When using Tasker, sometimes we want to do some automation on screen e.g. screen or button taps. 

At present, there are some ways can reach it. 

1."Input" action

We can use actions in "Input" category(especially "Dpad" action and "type" Action) to do some simple click/tap.

An example: http://tasker.wikidot.com/auto-usb-mass-storage 

Auto USB mass storage when connected
With Tasker, use a trick to automatically enable USB mass storage when USB cable plugged.

Require:
- Android 2.3+
- USB debbuging off
- In fact, the button “Turn on USB storage” must show up when USB cable plugged.

Download this file

Otherwise, you can create the profile yourself with the following steps:

1 Click New and select State
Select Hardware Action then USB plugged
2 Click + in the bottom left to add an action
Select category Task and Wait
Set 250 ms and press Done
3 Click + in the bottom left to add an action
Select category Input and DPad
Set Button Down and press done
4 Click + in the bottom left to add an action
Select category Input and DPad
Set Button Press and press done
5 Click + in the bottom left to add an action
Select category Task and Wait
Set 250 ms and press Done
6 Click + in the bottom left to add an action
Select category Input and DPad
Set Button Right and press done
7 Click + in the bottom left to add an action
Select category Input and DPad
Set Button Press only If %WIN ~ Turn off USB storage < replace it with your langage

Explanation:
Tasker simulates a tap on the button “Turn on USB storage” when it appears.

 

2. Use script

Use some type of Script to do this. Yes, script may be the best solution for automation.

Be careful! Sleep values might need some tweaking!  

  
  2.1 Monkey Script

 

Did you remember the "typewriter monkeys" theory? Certainly, the monkey immigrant to Android now.

You can reference Android Develop Document for details if you want know parameters of monkey script.

Or you can use the app named "Action Box"(Tasker plugin) directly.

It providers a somehow "IDE" for monkey script.

     You may need to calculate the coordinate in screen. It's so hard.

In order to get the correct coordinates, on your device go to Settings -> dev options and turn on "show pointer location" then you can go into your app and tap on the correct location and it will display the x/y coordinates of the tap which you can input on that script. 

     Some docs about the monkey tool>>>.

  2.2 "Input" command in shell

      With "input" command in shell, we can simulate keypress event on Android. The keycode sucks!

      More info about "Input" command, here.

Simulating keypress events on Android

  2.3 Sendevent and Getevent command in Shell 

      Android shell support sendevent and getevent command to grab and generate tap/swipe events on screen.

So we can make a way to do this. The method "listens" to your input actions (screen taps, button presses, etc)

and turns them into a script you can use to reproduce them.

Think of it as something similar to Excel's Macro Recording feature.

Of course, you need have knowledge of basic screen events(refer to tutorial).

TUTORIAL: Remote Control Touch Screen

  Solution 1:

1. run getevent ini adb shell

2. copy/paste it's output to file on PC

3. run script which contents one line cat $1 | gawk --non-decimal-data '{print $1, ("0x"$2)+0, ("0x"$3)+0, ("0x"$4)+0}' | sed 's/://g;s/^/sendevent /g' as convert.sh file.in > file out

4. put file.out on SD-card

5. Source it with Run Shell action

 

Note: the solution comes from here. Thanks to Vladimir Oz .

  Solution 2:

     You can use "Sendevent->Getevent Toolbox" to find a way out.

Simulating keypress events on Android

Modern day smart-phones have already begun to migrate from the traditional "a-button-for-every-need" approach to the "huge-display-cum-touchscreen" form-factors. Android phones  are no exception. But, traditional buttons are still reqd. for a few oft used functions (power,back,home,menu etc.) And smart-phones continue to have them alongwith the primary touch-based-UI.

 

Android-OS provides a very easy method to simulate key/button press/release events via software. You might ask why do we need a software to generate the events when a hardware button is already present on the device. Here's why:

• During development/testing of the button-drivers itself.
• To implement automated rigorous tests. ( MonkeyTest? )
• To implement/interface additional custom software keyboards.
• Just because we can!

The reason why i am doing it today is REASON NUMBER 4.

 

Now, the basic goal of this exercise is extremely simple:

Q. How to generate a hardware-button-press event
    WITHOUT actually pressing any key on the device?

 Let us first understand what happens when a hardware button is pressed. Here is what happens when you press a button on an Android device:

1. The h/w button triggers an interrupt.
2. The ISR of the corresponding driver gets called (in kernel).
3. In the ISR, the driver generates an input-event.
4. Android-framework (in userspace) gets the notification of the event.
5. Android reads the input-event code & type.
6. Compares it with the proper "keymap/keylayout" file.
7. The proper button is identified.

To simulate button presses we enter this procedure at STEP3.

Instead of the regular driver generating the input-event, we generate an input-event ourselves using a pre-built userspace binary. This will then be notified to the Android-framework and the rest continues as above.

 

So now all depends on the the pre-built userspace binary, which is...
[ drum-roll... ]

...Input. (How convenient!) The syntax of input is as follows:

input keyevent <event_code>

Now, before we try to send any keyevent, we need to find out the event-code that maps to the h/w key we want to simulate.

The following table summarizes all the supported keycodes in Android:

Additionally we can look up the keylayout (.kl) file on the device. Personally, I find keylayout a misnomer, as we are not talking about different keyboards here, but different mappings of an input-event-value to its functionality.

 

Anyways you can always find the file in system/usr/keylayout/xyz.kl

NOTE: For each keyboard device xyz, the android.keylayout.xyz system property must be set. If a keylayout file is not specified, Android will default to/system/usr/keylayout/qwerty.kl

 

Now to generate a event of a specific keycode, we simply execute this on the terminal/serial console:

input keyevent <keycode>

 The value of keycode can be any of the integer values from the above table.

In case a serial-console is NOT available on your device, you can always run the command via adb as follows:

adb shell input keyevent <keycode>

 

That's how a hardware-button-press is simulated in software!!

simulate screen tap with tasker

run shell -> input tap x y  -> ensure use root is ticked
x y are the coordinates (activate them to show on screen in developer menu in settings)

run shell -> input tap 500 500

run shell -> input swipe 500 500 500 800

run shell -> input swipe 500 500 500 500 s

Emulating touch in Android is apparently possible with shell commands, using Tasker’s Run Shell action.

The command you use is

input tap x y

 

Where X and Y are coordinates. Root must be checked in the action options, and obviously Tasker needs root access to do this.

As for how you find X and Y, that’s easy:

Go to system settings, developer options, and enable “Show pointer location”.

Use that option to find the X and Y coordinates of whatever you need the touch emulation to press, and write them down.

Then use them in the above command.

You can also emulate swipes, using the following command:

input swipe X1 Y1 X2 Y2

Where X1 Y1 are the star coordinates and X2 Y2 are the end coordinates.

As an example, “input swipe 400 400 100 400″ on my phone swipes the image thumbnails in a Google Play listing.

This works great on my device, but might be dependent on device and/or OS version/ROM.

For long tap use , using the following command:

input swipe x1 y1 x2 y2 s

Set x1 and x2 to the same value and y1 and y2 to the same value to keep your swipe in the same place and set s to the number of milliseconds you want your long press to last.

TIP:

don't forget to turn off the "Show Pointer Location" and "Show Touches" BEFORE running the task.

That caused several reboots for me. Oops!

 

This is fantastic, only issue I'm having is that it doesn't seem to work with the screen off.. maybe if I remove keyguard?

yeah, have to wake disable key guard/ disable pattern lock, run the emulation,

enable keyguard/enable pattern lock, but totally worth it