Remote control for Samsung DeX

From Knox 3.4.1 onwards, you can use remote control with Samsung DeX on devices running Android 10 (Q) or higher. This enables dual screen viewing, to remotely view either the:

• device screen
• DeX screen
• device and DeX screens collectively

This feature uses two shared buffers compared to a single shared buffer used earlier for just the device screen.

You can also inject pointer or key events into a DeX screen, separate from the device screen. Currently, trackball injection is not supported for DeX screens.

To remotely control a DeX screen

Use the following C APIs for remote viewing, and Java APIs for remote injection:

To capture a DeX screen

Initially, the dexStateChanged callback notifies you that DeX mode is on.

When DeX mode is on, use APIs mentioned in Remote viewing to capture the device screen. To capture the DeX screen, use DeX remote capture APIs following these steps —

1. Call IRemoteDesktop::getDefaultDexScreenInfo() to get the default hardware screen dimensions of DeX screen.
2. Call IRemoteDesktop::setDexScreenInfo() to set the desired DeX screen dimensions.
3. Call IRemoteDesktop::getDexScreenInfo() to get the DeX framebuffer size.
4. Call IRemoteDesktop::getDexBufInfo() to get the file descriptor (FD) for a DeX buffer.
5. Call mmap() to map DeX file descriptor to memory.

Now, on calling captureScreen(), you can capture a DeX screen using the file descriptor.

Look at this diagram to get a better idea of how DeX screen capture works.

To inject events to a DeX screen

Injecting events to the DeX screen is similar to injecting to device screen, as described in Remote injection.

The class RemoteInjection in the package com.samsung.android.knox.remotecontrol enables you to:

• Inject a keyboard event, using the API method injectKeyEventDex

• Inject a mouse event, using the API method injectPointerEventDex

The key events are defined by the Android class KeyEvent, however some key events like the power key apply only to the device screen.

This example simulates the keyboard entry of the character 0 to the DeX screen:

int keycode = KeyEvent.KEYCODE_0;
remoteInjection.injectKeyEventDex(new KeyEvent(KeyEvent.ACTION_DOWN, keycode), true);

This example simulates a pointer click at screen pixel location 100 x 200 on the DeX screen:

int x = 100, y = 200,flags = 0;
remoteInjection.injectPointerEventDex(MotionEvent.obtain(SystemClock.uptimeMillis(),
    SystemClock.uptimeMillis(), MotionEvent.ACTION_DOWN, x, y, flags), true);