1. Introduction

As more and more large and folding screen devices appear, many applications do not have UI adaptations for devices of different sizes, when the application chooses to display in a specific aspect ratio (although Google does not recommend this, the official hope that developers can adapt the layout of different screen sizes ~), when the application’s aspect ratio and its container ratio is not compatible, it will be opened in Letterbox mode.

sobyte

Letterbox mode displays the interface at a specified scale, and the surrounding blank area can be filled with wallpaper or colors. The appearance of the Letterbox can be influenced by the following factors.

  • config_letterboxActivityCornersRadius : the size of the rounded corners of the interface
  • config_letterboxBackgroundType : background fill type, respectively.
    • LETTERBOX_BACKGROUND_APP_COLOR_BACKGROUND : Color is affected by android:colorBackground
    • LETTERBOX_BACKGROUND_APP_COLOR_BACKGROUND_FLOATING : color is affected by android:colorBackgroundFloating
    • LETTERBOX_BACKGROUND_SOLID_COLOR : color is affected by config_letterboxBackgroundColor
    • LETTERBOX_BACKGROUND_WALLPAPER : Show wallpaper, this option is similar to FLAG_SHOW_WALLPAPER and will cause the wallpaper window to be displayed
  • config_letterboxBackgroundWallpaperBlurRadius : the wallpaper blurring level
  • config_letterboxBackgroundWallpaperDarkScrimAlpha : how dark the wallpaper is

2. when to trigger

Letterbox is generally triggered by.

  • android:resizeableActivity=false and when the declared aspect ratio of the application is not compatible with the container (e.g. the screen width exceeds android:maxAspectRatio ).
  • setIgnoreOrientationRequest(true) After the system is set to ignore the screen orientation, an interface with a forced vertical screen is opened in landscape mode.

3. implementation scheme

The implementation of Letterbox display is not complicated, Android 12 added LetterboxUiController in ActivityRecord to control the layout and display of Letterbox, let’s see the SurfaceFlinger state when it is in Letterbox mode.

sobyte

As you can see, compared to the normal case, in addition to the size and position of the interface itself being scaled to a specified proportion, there are two more Layers around, hanging under the ActiviRecord node.

Here is a brief analysis of the code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

// LetterboxUiController.java
void updateLetterboxSurface(WindowState winHint) {
    final WindowState w = mActivityRecord.findMainWindow();
    if (w != winHint && winHint != null && w != null) {
        return;
    }
    // 对界面四周需要显示的 Layer 进行位置计算
    layoutLetterbox(winHint);
    if (mLetterbox != null && mLetterbox.needsApplySurfaceChanges()) {
        // 对 Surface 执行创建、参数设置等操作
        mLetterbox.applySurfaceChanges(mActivityRecord.getSyncTransaction());
    }
}

void layoutLetterbox(WindowState winHint) {
    final WindowState w = mActivityRecord.findMainWindow();
    if (w == null || winHint != null && w != winHint) {
        return;
    }
    updateRoundedCorners(w);
    updateWallpaperForLetterbox(w);
    // 是否进入 Letterbox 模式的关键判断
    if (shouldShowLetterboxUi(w)) {
        if (mLetterbox == null) {
            // 把具体逻辑委托给 Letterbox 
            mLetterbox = new Letterbox(() -> mActivityRecord.makeChildSurface(null),
                    mActivityRecord.mWmService.mTransactionFactory,
                    mLetterboxConfiguration::isLetterboxActivityCornersRounded,
                    this::getLetterboxBackgroundColor,
                    this::hasWallpaperBackgroudForLetterbox,
                    this::getLetterboxWallpaperBlurRadius,
                    this::getLetterboxWallpaperDarkScrimAlpha);
            mLetterbox.attachInput(w);
        }
        mActivityRecord.getPosition(mTmpPoint);
        // Get the bounds of the "space-to-fill". The transformed bounds have the highest
        // priority because the activity is launched in a rotated environment. In multi-window
        // mode, the task-level represents this. In fullscreen-mode, the task container does
        // (since the orientation letterbox is also applied to the task).
        final Rect transformedBounds = mActivityRecord.getFixedRotationTransformDisplayBounds();
        final Rect spaceToFill = transformedBounds != null
                ? transformedBounds
                : mActivityRecord.inMultiWindowMode()
                        ? mActivityRecord.getRootTask().getBounds()
                        : mActivityRecord.getRootTask().getParent().getBounds();
        // 位置计算
        mLetterbox.layout(spaceToFill, w.getFrame(), mTmpPoint);
    } else if (mLetterbox != null) {
        mLetterbox.hide();
    }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

// Letterbox.LetterboxSurface.java
public void applySurfaceChanges(SurfaceControl.Transaction t) {
    if (!needsApplySurfaceChanges()) {
        // Nothing changed.
        return;
    }
    mSurfaceFrameRelative.set(mLayoutFrameRelative);
    if (!mSurfaceFrameRelative.isEmpty()) {
        if (mSurface == null) {
            // 创建挂在 ActivityRecord 节点下的 Surface，设置为 ColorLayer 类型
            createSurface(t);
        }
        // 设置颜色、位置、裁剪
        mColor = mColorSupplier.get();
        t.setColor(mSurface, getRgbColorArray());
        t.setPosition(mSurface, mSurfaceFrameRelative.left, mSurfaceFrameRelative.top);
        t.setWindowCrop(mSurface, mSurfaceFrameRelative.width(),
                mSurfaceFrameRelative.height());

        // 对壁纸背景设置透明度和模糊度
        mHasWallpaperBackground = mHasWallpaperBackgroundSupplier.get();
        updateAlphaAndBlur(t);

        t.show(mSurface);
    } else if (mSurface != null) {
        t.hide(mSurface);
    }
    if (mSurface != null && mInputInterceptor != null) {
        mInputInterceptor.updateTouchableRegion(mSurfaceFrameRelative);
        t.setInputWindowInfo(mSurface, mInputInterceptor.mWindowHandle);
    }
}

4. Summary

This article only briefly analyzed the trigger conditions and the general logic of displaying Letterbox mode, there are many details not covered, such as the detailed trigger logic judgment can check the LetterboxUiController#shouldShowLetterboxUi method. s