ImageCapture API example

ImageCapture API example. grabFrame(), drawImage(), and createImageBitmap() from the same camera. Six grabs, timed.

The problem

A live <video> is not a still. Preview pixels, a VideoFrame, and the hardware shutter are three different pictures.

The method

Canvas paths encode the preview. grabFrame() is a bitmap from the track. takePhoto() fires the shutter and can be much larger. Format and quality only touch the canvas encode. takePhoto() returns whatever the camera encoded.

What to do

  1. Start. The browser asks for the camera.
  2. Run one grab, or Capture all to compare.

canvas.toBlob()

Yes

Draw the video to a canvas, then encode. Works everywhere. Resolution is the canvas, not the sensor.

ctx.drawImage(video, 0, 0)
const blob = await new Promise((resolve) => {
  canvas.toBlob(resolve, type, quality)
})

OffscreenCanvas.convertToBlob()

Yes

Same draw path. The canvas can leave the main thread. convertToBlob() instead of toBlob().

const offscreen = new OffscreenCanvas(video.videoWidth, video.videoHeight)
offscreen.getContext("2d").drawImage(video, 0, 0)
const blob = await offscreen.convertToBlob({ type, quality })

createImageBitmap()

Yes

A GPU bitmap from the video element. You still need a canvas if you want a Blob.

const bitmap = await createImageBitmap(video)
ctx.drawImage(bitmap, 0, 0)
bitmap.close()

ImageCapture.grabFrame()

Yes

A frame from the live track. Chromium. Same resolution as the stream.

const imageCapture = new ImageCapture(track)
const bitmap = await imageCapture.grabFrame()

ImageCapture.takePhoto()

Yes

Fires the hardware shutter. The still can be much larger than the preview. The first shot is discarded on Start so the timed run is steady-state. Ignores the format slider.

const imageCapture = new ImageCapture(track)
const blob = await imageCapture.takePhoto()

VideoFrame

Yes

Read a VideoFrame from the track with MediaStreamTrackProcessor.

const processor = new MediaStreamTrackProcessor({ track })
const { value: frame } = await processor.readable.getReader().read()
ctx.drawImage(frame, 0, 0)
frame.close()

All Web API labs

  • HTML usermedia element example
  • MediaRecorder API example
  • Web Audio API example