Add Camera Support

添加相机支持

This lesson teaches you to

本节课您将学到

  1. Connect a CSI-2 peripheral

  2. 连接一个 CSI-2 外设

  3. Add camera features to an app
  1. 在应用中添加相机功能
  2. Handle I/O threading
  1. 处理 I/O 线程
  2. Trigger still image capture
  1. 触发静态图像拍摄
  2. Receive image data from the camera
  1. 接收相机图片数据

Try it out

试试看

A smart doorbell should capture an image of who (or what) is at the door, allowing the owner to remotely make a decision about whether or not to answer.

一个智能的门铃应该在门外有人或物时自动拍摄图像，使主人能够远程查看，并决定是否应答。

Because Android Things builds on the Android framework, you can access the same robust camera APIs that an Android mobile developer would use. In this lesson, you’ll access the camera peripheral using Android camera APIs and capture an image for later processing.

由于 Android Things 基于 Android 框架进行构建，您可以使用 Android 移动开发者所使用的成熟的相机 APIs 。在本节课，您将使用 Android 相机 APIs 来控制相机外设然拍摄一张图片以便后续处理。

Connect the camera

连接摄像头

Connect a supported camera module to the CSI-2 camera port on your board. Ensure that the cable is fully inserted and sits evenly before closing the connector latch.

将一个开发板所支持的相机模块连接到您的开发板的 CSI-2 相机端口。在固定连接器之前确认线缆完全插入并且结合处平整。

""

Add permissions and required features

添加权限和必要的功能

Add the required permissions to your app’s manifest file:

在您应用的 manifest 文件中添加必要的权限：

  1.     <uses-permission android:name="android.permission.CAMERA" />

Set up an I/O thread

创建一个 I/O 线程

Communicating with peripheral hardware introduces blocking operations into the flow of your app. To avoid blocking the app’s main thread, and thus delaying framework events, create a background worker thread to process input and handle commands. The HandlerThread works very nicely for this purpose.

与外设硬件通信将阻塞您的应用。为了避免应用的主线程被阻塞，从而导致框架事件被延迟，需要新开启一个后台线程来处理输入和处理指令。在这种情况下， HandlerThread 将会是一个很好的选择。

  1. public class DoorbellActivity extends Activity {
  2.     /**
  3.      * A Handler for running tasks in the background.     
  4.      */
  5.     private Handler mCameraHandler;
  6.     /**
  7.      * An additional thread for running tasks that shouldn't block the UI. 
  8.      *    
  9.      */
  10.     private HandlerThread mCameraThread;
  12.     @Override
  13.     protected void onCreate(Bundle savedInstanceState) {
  14.         super.onCreate(savedInstanceState);
  15.         // Creates new handlers and associated threads for camera and networking operations.     
  16.         mCameraThread = new HandlerThread("CameraBackground");
  17.         mCameraThread.start();
  18.         mCameraHandler = new Handler(mCameraThread.getLooper());
  19.     }
  21.     @Override
  22.     protected void onDestroy() {
  23.         super.onDestroy();
  24.         mCameraThread.quitSafely();
  25.     }
  26. }

Initialize the camera session

初始化相机会话

The first step to capture a camera image is to discover the hardware and open a device connection. To connect to a camera device:

捕捉图像的第一步是找到硬件并且打开设备连接。为了连接相机设备，您需要：

  1. Use the CameraManager system service to discover the list of available camera devices with getCameraIdList().

  1. 使用 CameraManager 系统服务中的 getCameraIdList() 方法找到的可用的相机设备列表。

  2. Create an ImageReader instance to process the raw camera data and produce a JPEG-encoded image to your app. The reader processes data asynchronously, and invokes the provided OnImageAvailableListener when the image is ready.

  1. 在您的应用中创建一个 ImageReader 实例用于处理原始的相机数据并生成一个 JPEG 编码图像。读取数据是异步的操作，当图像准备好将会调用 OnImageAvailableListener 方法。
  2. Open a connection to the appropriate camera device using openCamera().
  1. 使用 openCamera() 方法来连接一个合适的相机。
  2. The CameraDevice.StateCallback reports that the camera was opened successfully through the onOpened() callback method.
  1. 利用 CameraDevice.StateCallback 中的回调方法 onOpened() 来通知相机已经被成功打开。
  2. Close the CameraDevice when it is not in use to free the system resources:
  1. 当不再使用相机时，调用 CameraDevice 方法关闭相机，释放系统资源：
  1. public class DoorbellCamera {
  3.     // Camera image parameters (device-specific)
  4.     private static final int IMAGE_WIDTH  = ...;
  5.     private static final int IMAGE_HEIGHT = ...;
  6.     private static final int MAX_IMAGES   = ...;
  8.     // Image result processor
  9.     private ImageReader mImageReader;
  10.     // Active camera device connection
  11.     private CameraDevice mCameraDevice;
  12.     // Active camera capture session
  13.     private CameraCaptureSession mCaptureSession;
  15.     // Initialize a new camera device connection
  16.     public void initializeCamera(Context context,
  17.                                  Handler backgroundHandler,
  18.                                  ImageReader.OnImageAvailableListener imageAvailableListener) {
  20.         // Discover the camera instance
  21.         CameraManager manager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
  22.         String[] camIds = {};
  23.         try {
  24.             camIds = manager.getCameraIdList();
  25.         } catch (CameraAccessException e) {
  26.             Log.d(TAG, "Cam access exception getting IDs", e);
  27.         }
  28.         if (camIds.length < 1) {
  29.             Log.d(TAG, "No cameras found");
  30.             return;
  31.         }
  32.         String id = camIds[0];
  33.         Log.d(TAG, "Using camera id " + id);
  35.         // Initialize image processor
  36.         mImageReader = ImageReader.newInstance(IMAGE_WIDTH, IMAGE_HEIGHT,
  37.                 ImageFormat.JPEG, MAX_IMAGES);
  38.         mImageReader.setOnImageAvailableListener(imageAvailableListener, backgroundHandler);
  40.         // Open the camera resource
  41.         try {
  42.             manager.openCamera(id, mStateCallback, backgroundHandler);
  43.         } catch (CameraAccessException cae) {
  44.             Log.d(TAG, "Camera access exception", cae);
  45.         }
  46.     }
  48.     // Callback handling devices state changes
  49.     private final CameraDevice.StateCallback mStateCallback =
  50.             new CameraDevice.StateCallback() {
  52.         @Override
  53.         public void onOpened(CameraDevice cameraDevice) {
  54.             mCameraDevice = cameraDevice;
  55.         }
  57.         ...
  58.     };
  60.     // Close the camera resources
  61.     public void shutDown() {
  62.         if (mCameraDevice != null) {
  63.             mCameraDevice.close();
  64.         }
  65.     }
  66. }

Trigger an image capture

触发图像拍摄事件

Once the camera device connection is active, create a CameraCaptureSession to facilitate image requests from the hardware. To open a capture session:

一旦相机设备连接处于活动状态，创建一个 CameraCaptureSession 类来处理硬件拍摄图片的请求。开始一个拍摄会话：

  1. Build a new CameraCaptureSession instance with the createCaptureSession() method.

  2. 创建一个新的 CameraCaptureSession 实例并实现 createCaptureSession() 方法。

  3. Pass the method a list of potential target surfaces for individual image requests. For this example, pass the surface connected to the ImageReader constructed previously.

  4. 将一系列独立图像请求的预览所组成的列表传递给一个方法。在本例中，我们将这个预览列表传递给之前所构建的的 ImageReader 。

  5. Attach a CameraCaptureSession.StateCallback to report when the session is configured and active. The callback invokes the onConfigured() method if everything is successful.

  6. 当配置和激活成功后将会回调 CameraCaptureSession.StateCallback 中的 onConfigured() 方法。

  1. public class DoorbellCamera {
  3.     ...
  5.     public void takePicture() {
  6.         if (mCameraDevice == null) {
  7.             Log.w(TAG, "Cannot capture image. Camera not initialized.");
  8.             return;
  9.         }
  11.         // Here, we create a CameraCaptureSession for capturing still images.
  12.         try {
  13.             mCameraDevice.createCaptureSession(
  14.                     Collections.singletonList(mImageReader.getSurface()),
  15.                     mSessionCallback,
  16.                     null);
  17.         } catch (CameraAccessException cae) {
  18.             Log.d(TAG, "access exception while preparing pic", cae);
  19.         }
  20.     }
  22.     // Callback handling session state changes
  23.     private CameraCaptureSession.StateCallback mSessionCallback =
  24.             new CameraCaptureSession.StateCallback() {
  26.             @Override
  27.             public void onConfigured(CameraCaptureSession cameraCaptureSession) {
  28.                 // The camera is already closed
  29.                 if (mCameraDevice == null) {
  30.                     return;
  31.                 }
  33.                 // When the session is ready, we start capture.
  34.                 mCaptureSession = cameraCaptureSession;
  35.                 triggerImageCapture();
  36.             }
  38.             @Override
  39.             public void onConfigureFailed(CameraCaptureSession cameraCaptureSession) {
  40.                 Log.w(TAG, "Failed to configure camera");
  41.             }
  42.     };
  43. }

Your app can now use the active CameraCaptureSession to request image data from the camera hardware. To begin an image capture request inside the capture session:

现在，您的应用可以使用激活的 CameraCaptureSession 对象来请求相机硬件的图像数据。为了在图像会话中开始一个图像拍摄请求，您需要：

  1. Initialize a new CaptureRequest using the builder interface. To capture a single still image, use the TEMPLATE_STILL_CAPTURE parameter.

  2. 使用建造者接口初始化一个新的 CaptureRequest 实例，并通过设置 TEMPLATE_STILL_CAPTURE 参数来拍摄一张静态图像。

  3. Indicate the target surface for the request. For this example, this is the same ImageReader surface provided to the capture session.

  4. 指出请求的目标图像。在本例中，同获取图像预览和拍照会话是同一个 ImageReader。

  5. Set any additional capture parameters, such as auto-focus and auto-exposure, using the request builder.

  6. 设置额外的拍摄参数，例如自动对焦和自动曝光。

  7. Initiate the capture request on the CameraCaptureSession using the capture() method.

  8. 使用 CameraCaptureSession 的 capture() 方法发起拍摄请求。

  9. When the capture is complete, close the active session.

  10. 当拍摄完成，关闭会话。

  1. public class DoorbellCamera {
  3.     // Active camera device connection
  4.     private CameraDevice mCameraDevice;
  5.     // Active camera capture session
  6.     private CameraCaptureSession mCaptureSession;
  7.     // Image result processor
  8.     private ImageReader mImageReader;
  9.     ...
  11.     private void triggerImageCapture() {
  12.         try {
  13.             final CaptureRequest.Builder captureBuilder =
  14.                     mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);
  15.             captureBuilder.addTarget(mImageReader.getSurface());
  16.             captureBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON);
  17.             Log.d(TAG, "Session initialized.");
  18.             mCaptureSession.capture(captureBuilder.build(), mCaptureCallback, null);
  19.         } catch (CameraAccessException cae) {
  20.             Log.d(TAG, "camera capture exception");
  21.         }
  22.     }
  24.     // Callback handling capture progress events
  25.     private final CameraCaptureSession.CaptureCallback mCaptureCallback =
  26.         new CameraCaptureSession.CaptureCallback() {
  27.             ...
  29.             @Override
  30.             public void onCaptureCompleted(CameraCaptureSession session,
  31.                                            CaptureRequest request,
  32.                                            TotalCaptureResult result) {
  33.                 if (session != null) {
  34.                     session.close();
  35.                     mCaptureSession = null;
  36.                     Log.d(TAG, "CaptureSession closed");
  37.                 }
  38.             }
  39.         };
  40. }

Handle the image result

处理图像结果

From your activity, initialize the camera and invoke the takePicture() method when the doorbell button is pressed. During capture, the camera hardware streams the image data to the provided ImageReader surface and invokes the OnImageAvailableListener with the result.

在您的界面，初始化相机并且当门铃按钮被按下时调用 takePicture() 方法。在拍照时，使用 ImageReader 对象预览相机图像数据并在 OnImageAvailableListener 监听方法中获取图片。

To obtain the image after capture completes:

为了在拍摄结束后获取图片，您需要：

  1. Obtain the latest Image from the ImageReader provided to the onImageAvailable() method.

  2. 使用 ImageReader 对象提供的 onImageAvailable() 方法获取最新的 Image。

  3. Retrieve the JPEG-encoded image as a byte[] from the buffer returned by the getBuffer() method:

  4. 使用 getBuffer() 方法将缓冲区中返回的 JPEG 编码的图像转为 byte[] 格式。

  1. public class DoorbellActivity extends Activity {
  3.     /**
  4.      *  Camera capture device wrapper
  5.      */
  6.     private DoorbellCamera mCamera;
  8.     @Override
  9.     protected void onCreate(Bundle savedInstanceState) {
  10.         super.onCreate(savedInstanceState);
  11.         ...
  13.         mCamera = DoorbellCamera.getInstance();
  14.         mCamera.initializeCamera(this, mCameraHandler, mOnImageAvailableListener);
  15.     }
  17.     @Override
  18.     protected void onDestroy() {
  19.         super.onDestroy();
  20.         ...
  22.         mCamera.shutDown();
  23.     }
  25.     @Override
  26.     public boolean onKeyUp(int keyCode, KeyEvent event) {
  27.         if (keyCode == KeyEvent.KEYCODE_ENTER) {
  28.             // Doorbell rang!
  29.             Log.d(TAG, "button pressed");
  30.             mCamera.takePicture();
  31.             return true;
  32.         }
  33.         return super.onKeyUp(keyCode, event);
  34.     }
  36.     // Callback to receive captured camera image data
  37.     private ImageReader.OnImageAvailableListener mOnImageAvailableListener =
  38.             new ImageReader.OnImageAvailableListener() {
  39.         @Override
  40.         public void onImageAvailable(ImageReader reader) {
  41.             // Get the raw image bytes
  42.             Image image = reader.acquireLatestImage();
  43.             ByteBuffer imageBuf = image.getPlanes()[0].getBuffer();
  44.             final byte[] imageBytes = new byte[imageBuf.remaining()];
  45.             imageBuf.get(imageBytes);
  46.             image.close();
  48.             onPictureTaken(imageBytes);
  49.         }
  50.     };
  52.     private void onPictureTaken(final byte[] imageBytes) {
  53.         if (imageBytes != null) {
  54.             // ...process the captured image...
  55.         }
  56.     }
  57. }