Android Camera rotation angle

Posted by gabrielkolbe on Mon, 03 Jan 2022 01:54:59 +0100

First, understand info Orientation official interpretation

Official definition: orientation indicates the direction of the camera image. Its value is the angle when the camera image rotates clockwise to the natural direction of the device. For example, suppose the device is vertical. The rear camera sensor is installed horizontally. When you face the screen, if the top edge of the rear camera sensor is parallel to the right of the natural direction of the device, the orientation of the rear camera is 90. If the top edge of the front camera sensor is parallel to the right side of the natural direction of the device, the orientation of the front camera is 270.

I drew a simple sketch. I tested with two mobile phones, one is Huawei glory 6plus and the other is a customized T6A. During the test, the mobile phone was fixed as a vertical screen application

Another Android T6A has a special camera position. I got info orientation = 0; That means I don't have to rotate. The captured data is consistent with the direction of the screen

Special note:

For the rear camera, only rotate the orientation of the rear camera, i.e. 90, to keep consistent with the screen direction;  

For the preview direction of the front camera, the image previewed by the camera is the mirror image of the image collected by the camera. Because the system mirrors the image collected by the front camera, it needs to rotate 270-180 degrees, also 90 degrees, to be consistent with the screen direction.
 

1, Adaptation target

According to the camera rotation angle and the screen display rotation angle, select the rotation angle of the preview data displayed on the View, so that the real picture directly seen by the eyes is the same as that displayed on the mobile phone screen.

  • Camera rotation angle: the rotation angle of the camera imaging relative to the mobile phone. If the device has a camera installed, the rotation angle of the camera relative to the device is fixed.

    • Camera API acquisition method
     Camera.CameraInfo info = new Camera.CameraInfo();
 Camera.getCameraInfo(cameraId, info);
 Log.i(TAG, "orientation: " + info.orientation);
    • Camera2 API acquisition method
    CameraCharacteristics characteristics = manager.getCameraCharacteristics(cameraId);
sensorOrientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
Log.i(TAG, "orientation: " + sensorOrientation);

  • The screen shows the rotation angle: Activity#getWindowManager().getDefaultDisplay().getRotation() The value of can be ROTATION_0,ROTATION_90,ROTATION_180,ROTATION_270，

  • Preview data display rotation angle: according to the camera rotation angle and screen display rotation angle, we can calculate the preview data display rotation angle. Clockwise rotation direction.

2, Discovery law

Select the rear camera and front camera respectively, hold the phone at various angles, and obtain the camera rotation angle, screen display rotation angle and preview data as follows. Then, for the mirror image difference of the front and rear camera after the preview data, we can summarize the preview data display rotation angle:

cameraCamera rotation angleThe screen shows the rotation anglePreview frame dataThe preview data shows the rotation angle
Post90Surface.ROTATION_0 (portrait)

 

Post_ portrait

90
Post90Surface.ROTATION_90 (landscape)

Post_ landscape

0
Post90Surface.ROTATION_180 (reverse-portrait)

Post_ reverse-portrait

270
Post90Surface.ROTATION_270 (reverse-landscape)

Post_ reverse-landscape

180
Front270Surface.ROTATION_0 (portrait)

(non mirrored image after acquisition) front_ portrait

(internal mirror first) 90
Front270Surface.ROTATION_90 (landscape)

(non mirrored image after acquisition) front_ landscape

(internal mirror) 0
Front270Surface.ROTATION_180 (reverse-portrait)

(non mirrored image after acquisition) front_ reverse-portrait

(internal mirror) 270
Front270Surface.ROTATION_270 (reverse-landscape)

 

(non mirrored image after acquisition) front_ reverse-landscape

(internal mirror) 180

  • Internal mirror:
    For the rear camera, the preview data needs to be rotated before it can be displayed into a normal effect. Take the vertical screen as an example:

    Original drawingThe preview data shows the rotation angledesign sketch

    Post_ portrait

    		90

    Normal Preview

    For the front camera, we need to mirror the left and right before rotating to get the desired result. Take the vertical screen as an example:

    Original drawingMirror imageThe preview data shows the rotation angledesign sketch

     

    Front_ portrait

     

     

    Data after mirroring

     

     

     

    		90

    normal

3, Summary

After traversing and analyzing all the above situations, we also get the following results:

  • Mapping relationship:

    cameraCamera rotation angleThe screen shows the rotation angleThe preview data shows the rotation angle
    Post90Surface.ROTATION_0 (portrait)90
    Post90Surface.ROTATION_90 (landscape)0
    Post90Surface.ROTATION_180 (reverse-portrait)270
    Post90Surface.ROTATION_270 (reverse-landscape)180
    Front270Surface.ROTATION_0 (portrait)90
    Front270Surface.ROTATION_90 (landscape)0
    Front270Surface.ROTATION_180 (reverse-portrait)270
    Front270Surface.ROTATION_270 (reverse-landscape)180

  • Summary function:
    If we simply summarize the above numerical mapping relationship, we can find that the rotation angle of preview data display seems to be only related to the rotation angle of screen display, so we can get the following function:

     private int getCameraOri(int rotation) {
      switch (rotation) {
          case Surface.ROTATION_0:
              return 90;
          case Surface.ROTATION_90:
              return 0;
          case Surface.ROTATION_180:
              return 270;
          case Surface.ROTATION_270:
              return 180;
          default:
              return 0;
      }
  }

    But is it really enough? It can be seen that the input parameter of this function is only rotation, without considering cameraId and cameraOrientation. Let's rethink:

    • Rear camera
      For the rear camera, the rotation angle is 90 degrees without considering the mirror relationship, that is:

      • The rotation is surface ROTATION_ 0, the preview data needs to be rotated 90 degrees clockwise (cameraOrientation);
      • The rotation is surface ROTATION_ At 90, the preview data does not need to be rotated, i.e. 0 degrees (cameraOrientation - 90);
      • The rotation is surface ROTATION_ 180, the preview data needs to be rotated 270 degrees clockwise (cameraOrientation - 180) + 360;
      • The rotation is surface ROTATION_ At 270, the preview data needs to be rotated 180 degrees clockwise (cameraOrientation - 270) + 360;

        The screen shows that when the rotation angle increases by 90 degrees, the preview data shows that the rotation angle decreases by 90 degrees.
        Therefore, the adaptation code of the rear camera is as follows:
          int degrees = rotation * 90;
    switch (rotation) {
        case Surface.ROTATION_0:
            degrees = 0;
            break;
        case Surface.ROTATION_90:
            degrees = 90;
            break;
        case Surface.ROTATION_180:
            degrees = 180;
            break;
        case Surface.ROTATION_270:
            degrees = 270;
            break;
        default:
            break;
    }
    // result is in camera Parameter to setdisplayoorientation (int)
    result = (info.orientation - degrees + 360) % 360;

    • front camera
      For the front camera, the rotation angle is 270 degrees, that is:

      • The rotation is surface ROTATION_ 0, the preview data needs to be rotated 270 degrees clockwise after the left and right mirror images (cameraOrientation);
      • The rotation is surface ROTATION_ At 90, the preview data does not need to be rotated after the left and right images are mirrored, that is, 0 degrees (cameraOrientation + 90) - 360;
      • The rotation is surface ROTATION_ 180, the preview data needs to be rotated 90 degrees clockwise (cameraOrientation + 180) - 360 after left and right mirroring;
      • The rotation is surface ROTATION_ 270, the preview data needs to be rotated 180 degrees clockwise (cameraOrientation + 270) - 360 after left and right mirroring;
        The system has already handled the mirroring operation for us (see the comments in the second code below). We only need to pass in the rotation angle.

    • To sum up, the Camera rotation angle adaptation code is as follows:

   displayOrientation = getCameraOri(rotation，mCameraId);
   camera.setDisplayOrientation(displayOrientation);

   private int getCameraOri(int rotation, int cameraId) {
        int degrees = rotation * 90;
        switch (rotation) {
            case Surface.ROTATION_0:
                degrees = 0;
                break;
            case Surface.ROTATION_90:
                degrees = 90;
                break;
            case Surface.ROTATION_180:
                degrees = 180;
                break;
            case Surface.ROTATION_270:
                degrees = 270;
                break;
            default:
                break;
        }

        // result is in camera Parameter to setdisplayoorientation (int)
        int result;
        Camera.CameraInfo info = new Camera.CameraInfo();
        Camera.getCameraInfo(cameraId, info);
        if (info.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
            result = (info.orientation + degrees) % 360;
            result = (360 - result) % 360;
        } else {
            result = (info.orientation - degrees + 360) % 360;
        }
        return result;
    }

In fact, Android hardware. The camera class has already implemented the adaptation scheme for us, and the annotation of public native final void setdisplayoorientation (int degrees) states that a mirror operation will be performed before rotation:

Set the clockwise rotation of preview display in degrees. This affects
the preview frames and the picture displayed after snapshot. This method
is useful for portrait mode applications. Note that preview display of
front-facing cameras is flipped horizontally before the rotation, that
is, the image is reflected along the central vertical axis of the camera
sensor. So the users can see themselves as looking into a mirror.

    /**
     * Set the clockwise rotation of preview display in degrees. This affects
     * the preview frames and the picture displayed after snapshot. This method
     * is useful for portrait mode applications. Note that preview display of
     * front-facing cameras is flipped horizontally before the rotation, that
     * is, the image is reflected along the central vertical axis of the camera
     * sensor. So the users can see themselves as looking into a mirror.
     *
     * <p>This does not affect the order of byte array passed in {@link
     * PreviewCallback#onPreviewFrame}, JPEG pictures, or recorded videos. This
     * method is not allowed to be called during preview.
     *
     * <p>If you want to make the camera image show in the same orientation as
     * the display, you can use the following code.
     * <pre>
     * public static void setCameraDisplayOrientation(Activity activity,
     *         int cameraId, android.hardware.Camera camera) {
     *     android.hardware.Camera.CameraInfo info =
     *             new android.hardware.Camera.CameraInfo();
     *     android.hardware.Camera.getCameraInfo(cameraId, info);
     *     int rotation = activity.getWindowManager().getDefaultDisplay()
     *             .getRotation();
     *     int degrees = 0;
     *     switch (rotation) {
     *         case Surface.ROTATION_0: degrees = 0; break;
     *         case Surface.ROTATION_90: degrees = 90; break;
     *         case Surface.ROTATION_180: degrees = 180; break;
     *         case Surface.ROTATION_270: degrees = 270; break;
     *     }
     *
     *     int result;
     *     if (info.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
     *         result = (info.orientation + degrees) % 360;
     *         result = (360 - result) % 360;  // compensate the mirror
     *     } else {  // back-facing
     *         result = (info.orientation - degrees + 360) % 360;
     *     }
     *     camera.setDisplayOrientation(result);
     * }
     * </pre>
     *
     * <p>Starting from API level 14, this method can be called when preview is
     * active.
     *
     * <p><b>Note: </b>Before API level 24, the default value for orientation is 0. Starting in
     * API level 24, the default orientation will be such that applications in forced-landscape mode
     * will have correct preview orientation, which may be either a default of 0 or
     * 180. Applications that operate in portrait mode or allow for changing orientation must still
     * call this method after each orientation change to ensure correct preview display in all
     * cases.</p>
     *
     * @param degrees the angle that the picture will be rotated clockwise.
     *                Valid values are 0, 90, 180, and 270.
     * @throws RuntimeException if setting orientation fails; usually this would
     *    be because of a hardware or other low-level error, or because
     *    release() has been called on this Camera instance.
     * @see #setPreviewDisplay(SurfaceHolder)
     */
    public native final void setDisplayOrientation(int degrees);

The operation effect is indeed the same. We can also turn to the android source code to see the internal implementation:

  • android.hardware.Camera.java
public native final void setDisplayOrientation(int degrees);
  • frameworks\base\core\jni\android_hardware_Camera.cpp
static void android_hardware_Camera_setDisplayOrientation(JNIEnv *env, jobject thiz,
        jint value)
{
    ALOGV("setDisplayOrientation");
    sp<Camera> camera = get_native_camera(env, thiz, NULL);
    if (camera == 0) return;

    if (camera->sendCommand(CAMERA_CMD_SET_DISPLAY_ORIENTATION, value, 0) != NO_ERROR) {
        jniThrowRuntimeException(env, "set display orientation failed");
    }
}
  • frameworks\av\camera\CameraClient.cpp
status_t CameraClient::sendCommand(int32_t cmd, int32_t arg1, int32_t arg2) {
    ...
    if (cmd == CAMERA_CMD_SET_DISPLAY_ORIENTATION) {
        // Mirror the preview if the camera is front-facing.
        orientation = getOrientation(arg1, mCameraFacing == CAMERA_FACING_FRONT);
        if (orientation == -1) return BAD_VALUE;

        if (mOrientation != orientation) {
            mOrientation = orientation;
            if (mPreviewWindow != 0) {
                mHardware->setPreviewTransform(mOrientation);
            }
        }
        return OK;
    } 
    ...
    return mHardware->sendCommand(cmd, arg1, arg2);
}

int CameraClient::getOrientation(int degrees, bool mirror) {
    if (!mirror) {
        if (degrees == 0) return 0;
        else if (degrees == 90) return HAL_TRANSFORM_ROT_90;
        else if (degrees == 180) return HAL_TRANSFORM_ROT_180;
        else if (degrees == 270) return HAL_TRANSFORM_ROT_270;
    } else {  // Do mirror (horizontal flip)
        if (degrees == 0) {           // FLIP_H and ROT_0
            return HAL_TRANSFORM_FLIP_H;
        } else if (degrees == 90) {   // FLIP_H and ROT_90
            return HAL_TRANSFORM_FLIP_H | HAL_TRANSFORM_ROT_90;
        } else if (degrees == 180) {  // FLIP_H and ROT_180
            return HAL_TRANSFORM_FLIP_V;
        } else if (degrees == 270) {  // FLIP_H and ROT_270
            return HAL_TRANSFORM_FLIP_V | HAL_TRANSFORM_ROT_90;
        }
    }
    ALOGE("Invalid setDisplayOrientation degrees=%d", degrees);
    return -1;
}

Here is a key code to set the horizontal mirror image, vertical mirror image and rotation angle for different rotation angles of the front camera.

        if (degrees == 0) {           // FLIP_H and ROT_0
            return HAL_TRANSFORM_FLIP_H;
        } else if (degrees == 90) {   // FLIP_H and ROT_90
            return HAL_TRANSFORM_FLIP_H | HAL_TRANSFORM_ROT_90;
        } else if (degrees == 180) {  // FLIP_H and ROT_180
            return HAL_TRANSFORM_FLIP_V;
        } else if (degrees == 270) {  // FLIP_H and ROT_270
            return HAL_TRANSFORM_FLIP_V | HAL_TRANSFORM_ROT_90;
        }

Including HAL_TRANSFORM_XXX is defined as follows:

/**
 * Transformation definitions
 *
 * IMPORTANT NOTE:
 * HAL_TRANSFORM_ROT_90 is applied CLOCKWISE and AFTER HAL_TRANSFORM_FLIP_{H|V}.
 *
 */
typedef enum android_transform {
    /* flip source image horizontally (around the vertical axis) */
    HAL_TRANSFORM_FLIP_H    = 0x01,
    /* flip source image vertically (around the horizontal axis)*/
    HAL_TRANSFORM_FLIP_V    = 0x02,
    /* rotate source image 90 degrees clockwise */
    HAL_TRANSFORM_ROT_90    = 0x04,
    /* rotate source image 180 degrees */
    HAL_TRANSFORM_ROT_180   = 0x03,
    /* rotate source image 270 degrees clockwise */
    HAL_TRANSFORM_ROT_270   = 0x07,
    /* don't use. see system/window.h */
    HAL_TRANSFORM_RESERVED  = 0x08,
} android_transform_t;
  • frameworks\av\services\camera\libcameraservice\device1\CameraHardwareInterface.cpp
status_t CameraHardwareInterface::setPreviewTransform(int transform) {
    int rc = OK;
    mPreviewTransform = transform;
    if (mPreviewWindow != nullptr) {
        rc = native_window_set_buffers_transform(mPreviewWindow.get(),
                mPreviewTransform);
    }
    return rc;
}
  • frameworks\native\libs\nativewindow\include\system\window.h
/*
 * native_window_set_buffers_transform(..., int transform)
 * All buffers queued after this call will be displayed transformed according
 * to the transform parameter specified.
 */
static inline int native_window_set_buffers_transform(
        struct ANativeWindow* window,
        int transform)
{
    return window->perform(window, NATIVE_WINDOW_SET_BUFFERS_TRANSFORM,
            transform);
}
  • frameworks/native/libs/gui/Surface.cpp
// The ANativeWindow::perform function points to the local hook_perform function
Surface::Surface(
        const sp<IGraphicBufferProducer>& bufferProducer,
        bool controlledByApp)
    : mGraphicBufferProducer(bufferProducer),
      mCrop(Rect::EMPTY_RECT),
      mGenerationNumber(0),
      mSharedBufferMode(false),
      mAutoRefresh(false),
      mSharedBufferSlot(BufferItem::INVALID_BUFFER_SLOT),
      mSharedBufferHasBeenQueued(false)
{
    ...
    ANativeWindow::perform          = hook_perform;
    ...
}
int Surface::hook_perform(ANativeWindow* window, int operation, ...) {
    va_list args;
    va_start(args, operation);
    Surface* c = getSelf(window);
    int result = c->perform(operation, args);
    va_end(args);
    return result;
}
int Surface::perform(int operation, va_list args)
{
    int res = NO_ERROR;
    switch (operation) {
    ...
    case NATIVE_WINDOW_SET_BUFFERS_TRANSFORM:
        res = dispatchSetBuffersTransform(args);
        break;
    ...
    }
}
int Surface::dispatchSetBuffersTransform(va_list args) {
    uint32_t transform = va_arg(args, uint32_t);
    return setBuffersTransform(transform);
}

// Finally, the value of mTransform is set
int Surface::setBuffersTransform(uint32_t transform)
{
    ATRACE_CALL();
    ALOGV("Surface::setBuffersTransform");
    Mutex::Autolock lock(mMutex);
    mTransform = transform;
    return NO_ERROR;
}
// Let's see where mTransform is used
int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {
    ...
    IGraphicBufferProducer::QueueBufferInput input(timestamp, isAutoTimestamp,
            mDataSpace, crop, mScalingMode, mTransform ^ mStickyTransform,
            fence, mStickyTransform);
        if (mConnectedToCpu || mDirtyRegion.bounds() == Rect::INVALID_RECT) {
        input.setSurfaceDamage(Region::INVALID_REGION);
    } else {
        // Here we do two things:
        // 1) The surface damage was specified using the OpenGL ES convention of
        //    the origin being in the bottom-left corner. Here we flip to the
        //    convention that the rest of the system uses (top-left corner) by
        //    subtracting all top/bottom coordinates from the buffer height.
        // 2) If the buffer is coming in rotated (for example, because the EGL
        //    implementation is reacting to the transform hint coming back from
        //    SurfaceFlinger), the surface damage needs to be rotated the
        //    opposite direction, since it was generated assuming an unrotated
        //    buffer (the app doesn't know that the EGL implementation is
        //    reacting to the transform hint behind its back). The
        //    transformations in the switch statement below apply those
        //    complementary rotations (e.g., if 90 degrees, rotate 270 degrees).

        int width = buffer->width;
        int height = buffer->height;
        bool rotated90 = (mTransform ^ mStickyTransform) &
                NATIVE_WINDOW_TRANSFORM_ROT_90;
        if (rotated90) {
            std::swap(width, height);
        }

        Region flippedRegion;
        for (auto rect : mDirtyRegion) {
            int left = rect.left;
            int right = rect.right;
            int top = height - rect.bottom; // Flip from OpenGL convention
            int bottom = height - rect.top; // Flip from OpenGL convention
            switch (mTransform ^ mStickyTransform) {
                case NATIVE_WINDOW_TRANSFORM_ROT_90: {
                    // Rotate 270 degrees
                    Rect flippedRect{top, width - right, bottom, width - left};
                    flippedRegion.orSelf(flippedRect);
                    break;
                }
                case NATIVE_WINDOW_TRANSFORM_ROT_180: {
                    // Rotate 180 degrees
                    Rect flippedRect{width - right, height - bottom,
                            width - left, height - top};
                    flippedRegion.orSelf(flippedRect);
                    break;
                }
                case NATIVE_WINDOW_TRANSFORM_ROT_270: {
                    // Rotate 90 degrees
                    Rect flippedRect{height - bottom, left,
                            height - top, right};
                    flippedRegion.orSelf(flippedRect);
                    break;
                }
                default: {
                    Rect flippedRect{left, top, right, bottom};
                    flippedRegion.orSelf(flippedRect);
                    break;
                }
            }
        }

        input.setSurfaceDamage(flippedRegion);
    }

    status_t err = mGraphicBufferProducer->queueBuffer(i, input, &output);
    if (err != OK)  {
        ALOGE("queueBuffer: error queuing buffer to SurfaceTexture, %d", err);
    }
    ...
}

 

Hot Topics

©2025 Programmer Think