
Figure 1. Example of Image Keyboard Support
Users often want to communicate using emojis, stickers, and other kinds of rich content. In previous versions of Android, soft keyboards (also known as input method editors or IMEs) could send only unicode emoji to apps. For rich content, apps had to either build app-specific APIs that couldn't be used in other apps or use workaround like sending images through Easy Share Action or the clipboard.
With Android 7.1 (API level 25), the Android SDK includes the Commit Content API, which provides a universal way for IMEs to send images and other rich content directly to a text editor in an app. The API is also available in v13 Support Library as of revision 25.0.0. We recommend using the Support Library because it runs on devices as early as Android 3.2 (API Level 13), and it contains helper methods that simplify implementation.
With this API, you can build messaging apps that accept rich content from any keyboard, as well as keyboards that can send rich content to any app. The Google Keyboard and apps like Google Messenger support the Commit Content API in Android 7.1 (see figure 1).
This page shows you how to implement the Commit Content API in both IMEs and apps.
How it works
Keyboard image insertion requires participation from both the IME and the app. The following sequence describes each step in the image insertion process:
When the user taps on an
EditText
, the editor sends a list of MIME content types that it accepts inEditorInfo.contentMimeTypes
.The IME reads the list of supported types and displays content in the soft keyboard that the editor can accept.
When the user selects an image, the IME calls
commitContent()
and sends anInputContentInfo
to the editor. ThecommitContent()
call is analogous to thecommitText()
call, but for rich content.InputContentInfo
contains an URI that identifies the content in a content provider.
Adding image support to apps
To accept rich content from IMEs, an app must tell IMEs what content types it
accepts and specify a callback method that is executed when content is received.
The following example demonstrates how to create an
EditText
that
accepts PNG images:
Kotlin
val editText = object : EditText(this) { override fun onCreateInputConnection(editorInfo: EditorInfo): InputConnection { val ic: InputConnection = super.onCreateInputConnection(editorInfo) EditorInfoCompat.setContentMimeTypes(editorInfo, arrayOf("image/png")) val callback = InputConnectionCompat.OnCommitContentListener { inputContentInfo, flags, opts -> val lacksPermission = (flags and InputConnectionCompat.INPUT_CONTENT_GRANT_READ_URI_PERMISSION) != 0 // read and display inputContentInfo asynchronously if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N_MR1 && lacksPermission) { try { inputContentInfo.requestPermission() } catch (e: Exception) { return@OnCommitContentListener false // return false if failed } } // read and display inputContentInfo asynchronously. // call inputContentInfo.releasePermission() as needed. true // return true if succeeded } return InputConnectionCompat.createWrapper(ic, editorInfo, callback) } }
Java
EditText editText = new EditText(this) { @Override public InputConnection onCreateInputConnection(EditorInfo editorInfo) { final InputConnection ic = super.onCreateInputConnection(editorInfo); EditorInfoCompat.setContentMimeTypes(editorInfo, new String [] {"image/png"}); final InputConnectionCompat.OnCommitContentListener callback = new InputConnectionCompat.OnCommitContentListener() { @Override public boolean onCommitContent(InputContentInfoCompat inputContentInfo, int flags, Bundle opts) { // read and display inputContentInfo asynchronously if (BuildCompat.isAtLeastNMR1() && (flags & InputConnectionCompat.INPUT_CONTENT_GRANT_READ_URI_PERMISSION) != 0) { try { inputContentInfo.requestPermission(); } catch (Exception e) { return false; // return false if failed } } // read and display inputContentInfo asynchronously. // call inputContentInfo.releasePermission() as needed. return true; // return true if succeeded } }; return InputConnectionCompat.createWrapper(ic, editorInfo, callback); } };
There’s a lot going on, so let’s explain what's going on.
This example uses the support library, so there are some references to
android.support.v13.view.inputmethod
instead ofandroid.view.inputmethod
.This example creates an
EditText
and overrides itsonCreateInputConnection(EditorInfo)
method to modify theInputConnection
. TheInputConnection
is the communication channel between an IME and the app that is receiving its input.The call
super.onCreateInputConnection()
preserves the built-in behavior (sending and receiving text) and gives you a reference to theInputConnection
.setContentMimeTypes()
adds a list of supported MIME types toEditorInfo
. Make sure to callsuper.onCreateInputConnection()
beforesetContentMimeTypes()
.callback
is executed whenever the IME commits content. The methodonCommitContent()
has a reference toInputContentInfoCompat
which contains a content URI.- You should request and release permissions if your app is running on API
Level 25 or higher and the
INPUT_CONTENT_GRANT_READ_URI_PERMISSION
flag was set by the IME. Otherwise, you should already have access to the content URI either because it was granted by the IME or because the content provider does not restrict access. For more information, see Adding Image Support to IMEs.
- You should request and release permissions if your app is running on API
Level 25 or higher and the
createWrapper()
wraps theInputConnection
, the modifiedEditorInfo
, and the callback into a newInputConnection
and returns it.
Here are some recommended practices:
Editors that do not support rich content should not call
setContentMimeTypes()
and leave theirEditorInfo.contentMimeTypes
set tonull
.Editors should ignore the content if the MIME type specified in
InputContentInfo
does not match any of types it accepts.Rich content does not affect and is not affected by the position of the text cursor. Editors can ignore cursor position when working with content.
In the editor's
OnCommitContentListener.onCommitContent()
method, you can returntrue
asynchronously, even before loading the content.Unlike text which can be edited in the IME before being committed, rich content is committed immediately. Be aware that if you want to provide users the ability to edit or delete content, you must implement logic yourself.
To test your app, make sure your device or emulator has a keyboard that is able to send rich content. You can use the Google Keyboard in Android 7.1 or higher.
Adding image support to IMEs
IMEs that want to send rich content to apps must implement the Commit Content API as shown below:
- Override
onStartInput()
oronStartInputView()
and read the list of supported content types from the target editor. The following code snippet shows how to check whether the target editor accepts GIF images.
Kotlin
override fun onStartInputView(editorInfo: EditorInfo, restarting: Boolean) { val mimeTypes: Array<String> = EditorInfoCompat.getContentMimeTypes(editorInfo) val gifSupported: Boolean = mimeTypes.any { ClipDescription.compareMimeTypes(it, "image/gif") } if (gifSupported) { // the target editor supports GIFs. enable corresponding content } else { // the target editor does not support GIFs. disable corresponding content } }
Java
@Override public void onStartInputView(EditorInfo info, boolean restarting) { String[] mimeTypes = EditorInfoCompat.getContentMimeTypes(editorInfo); boolean gifSupported = false; for (String mimeType : mimeTypes) { if (ClipDescription.compareMimeTypes(mimeType, "image/gif")) { gifSupported = true; } } if (gifSupported) { // the target editor supports GIFs. enable corresponding content } else { // the target editor does not support GIFs. disable corresponding content } }
- Commit content to the app when the users selects an image. Avoid calling
commitContent()
when there is any composing text because it might cause the editor to lose focus. The following code snippet shows how to commit a GIF image.
Kotlin
/** * Commits a GIF image * * @param contentUri Content URI of the GIF image to be sent * @param imageDescription Description of the GIF image to be sent */ fun commitGifImage(contentUri: Uri, imageDescription: String) { val inputContentInfo = InputContentInfoCompat( contentUri, ClipDescription(imageDescription, arrayOf("image/gif")), null ) val inputConnection = currentInputConnection val editorInfo = currentInputEditorInfo var flags = 0 if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N_MR1) { flags = flags or InputConnectionCompat.INPUT_CONTENT_GRANT_READ_URI_PERMISSION } InputConnectionCompat.commitContent(inputConnection, editorInfo, inputContentInfo, flags, null) }
Java
/** * Commits a GIF image * * @param contentUri Content URI of the GIF image to be sent * @param imageDescription Description of the GIF image to be sent */ public static void commitGifImage(Uri contentUri, String imageDescription) { InputContentInfoCompat inputContentInfo = new InputContentInfoCompat( contentUri, new ClipDescription(imageDescription, new String[]{"image/gif"}), null ); InputConnection inputConnection = getCurrentInputConnection(); EditorInfo editorInfo = getCurrentInputEditorInfo(); Int flags = 0; if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N_MR1) { flags |= InputConnectionCompat.INPUT_CONTENT_GRANT_READ_URI_PERMISSION; } InputConnectionCompat.commitContent( inputConnection, editorInfo, inputContentInfo, flags, null); }
As an IME author, you will most likely have to implement your own content provider to respond to content URI requests. The exception is if your IME supports content from existing content providers like
MediaStore
. For information on building content providers, see the Content Provider and File Provider documentation.If you are building your own content provider, we recommend that you don't export it (set android:exported to
false
). Instead, enable permission granting in the provider by setting android:grantUriPermission totrue
. Then, your IME can grant permissions to access the content URI when the content is committed. There are two ways to do this:On Android 7.1 (API Level 25) and higher, when calling
commitContent()
, set the flag parameter toINPUT_CONTENT_GRANT_READ_URI_PERMISSION
. Then, theInputContentInfo
object that the app receives can request and release temporary read permissions by callingrequestPermission()
andreleasePermission()
.On Android 7.0 (API Level 24) and lower,
INPUT_CONTENT_GRANT_READ_URI_PERMISSION
is ignored, so you need to manually grant permission to the content. One way to do this is withgrantUriPermission()
, but you can implement your own mechanism that satisfies your own requirements.
To test your IME, make sure your device or emulator has an app that is able to receive rich content. You can use the Google Messenger app in Android 7.1 or higher.