Flutter Raw Image Provider

The Image Widget in Flutter has built-in support for files in the form of file, network, and memory. However, these only support regular compressed image files or binary data, such as jpg, png, webp files, etc. There is no support for raw rgba binary data. Raw binary data here refers to the array of bytes consisting of the color values of each pixel of an image. A picture has width x height pixel points, the color value of a pixel point is stored with 32bit, divided into 4 channels, each channel occupies 8bit, respectively, red, green, blue, transparency (RGBA), this array is the collection of color values of each pixel point, dart in general use Uint8List.

In general, considering the efficiency of network transmission, algorithms are used to compress this data, so you will see that there are various image compression algorithms and file formats.

You may ask when there is a need to load the raw rgba data of a picture directly?

Here’s a simple example: loading an image in chunks. After decoding the image, split it into rectangular areas, each rectangle will have a raw rgba data, and give it to Image to render, which can reduce a certain GPU memory pressure and reduce the probability of GPU OOM or black screen.

To support raw rgba, it is actually very simple, there is a method decodeImageFromPixels under dart:ui package that can be used directly, provided that the raw binary data, width and height are available.

import 'dart:ui';

Future<Image> decodeRawRgba(ByteData bytes, int width, int height) {
  final Completer<Image> completer = Completer<Image>();
  decodeImageFromPixels(
    bytes.buffer.asUint8List(),
    width,
    height,
    PixelFormat.rgba8888,
    completer.complete,
  );
  return completer.future;
}

With this Image (dart:ui) object you can leave it to the RawImage widget to load. But RawImage is too low-level, can we just use the Image widget? Because we need to reuse the LoadingBuilder logic.

Of course you can. A quick look at the constructor of the Image widget shows that we need an ImageProvider, so the problem is further reduced to how to write an ImageProvider to support raw rgba data.

To implement an ImageProvider, we need to implement the key method load. Let’s take MemoryImage as an example.

class MemoryImage extends ImageProvider<MemoryImage> {
  @override
  ImageStreamCompleter load(MemoryImage key, DecoderCallback decode) {
    return MultiFrameImageStreamCompleter(
      codec: _loadAsync(key, decode),
      scale: key.scale,
      debugLabel: 'MemoryImage(${describeIdentity(key.bytes)})',
    );
  }

  Future<ui.Codec> _loadAsync(MemoryImage key, DecoderCallback decode) {
    return decode(bytes);
  }
}

Obviously, we need to think of a way to construct a Codec for raw rgba data.

The secret is in the implementation of the decodeImageFromPixels method.

void decodeImageFromPixels(
  Uint8List pixels,
  int width,
  int height,
  PixelFormat format,
  ImageDecoderCallback callback, {
  int? rowBytes,
  int? targetWidth,
  int? targetHeight,
  bool allowUpscaling = true,
}) {
  if (targetWidth != null) {
    assert(allowUpscaling || targetWidth <= width);
  }
  if (targetHeight != null) {
    assert(allowUpscaling || targetHeight <= height);
  }

  ImmutableBuffer.fromUint8List(pixels)
    .then((ImmutableBuffer buffer) {
      final ImageDescriptor descriptor = ImageDescriptor.raw(
        buffer,
        width: width,
        height: height,
        rowBytes: rowBytes,
        pixelFormat: format,
      );

      if (!allowUpscaling) {
        if (targetWidth != null && targetWidth! > descriptor.width) {
          targetWidth = descriptor.width;
        }
        if (targetHeight != null && targetHeight! > descriptor.height) {
          targetHeight = descriptor.height;
        }
      }

      descriptor
        .instantiateCodec(
          targetWidth: targetWidth,
          targetHeight: targetHeight,
        )
        .then((Codec codec) => codec.getNextFrame())
        .then((FrameInfo frameInfo) => callback(frameInfo.image));
  });
}

First construct ImageDescriptor from the data, and then extract the step descriptor.installCodec() to get the Codec of the raw rgba data, and then implement a RawImageProvider of your own.

For example.

class RawImageProvider extends ImageProvider<_RawImageKey> {
  final RawImageData image;
  /// see [ui.decodeImageFromPixels]
  Future<ui.Codec> _loadAsync(_RawImageKey key) async {
    var buffer = await ui.ImmutableBuffer.fromUint8List(image.pixels);

    final descriptor = ui.ImageDescriptor.raw(
      buffer,
      width: image.width,
      height: image.height,
      pixelFormat: image.pixelFormat,
    );
    return descriptor.instantiateCodec(
        targetWidth: targetWidth, targetHeight: targetHeight);
  }

Detailed code is available at: https://github.com/yrom/flutter_raw_image_provider/blob/master/lib/raw_image_provider.dart

If you also happen to have this need, you can add the pub dependency directly.

1
2

dependencies:
    raw_image_provider: ^0.1.0

Table of Contents