"Google said that this milestone, announced January 28, solidifies LiteRT as a universal on-device framework and represents a significant leap over its predecessor, TFLite. LiteRT delivers 1.4x faster GPU performance than TFLite, provides a unified workflow for GPU and NPU acceleration across edge platforms, supports superior cross-platform deployment for generative AI models, and offers first-class PyTorch/JAX support through seamless model conversion, Google said."
"Via the new ML Drift GPU engine, LiteRT supports OpenCL, OpenGL, Metal, and WebGPU, allowing developers to deploy models across, mobile, desktop, and web. For Android, LiteRT automatically prioritizes when available for peak performance, while falling back to OpenGL for broader device coverage. In addition, LiteRT provides a unified, simplified NPU deployment workflow that abstracts away low-level, vendor-specific SDKs and handles fragmentation across numerous SoC (system on chip) variants, according to Google."
LiteRT is a modern on-device inference framework evolved from TensorFlow Lite with a next-generation GPU engine called ML Drift that enables advanced acceleration. It delivers 1.4x faster GPU performance than TFLite and provides a unified workflow for GPU and NPU acceleration across edge platforms. The framework supports OpenCL, OpenGL, Metal, and WebGPU for mobile, desktop, and web deployment; on Android LiteRT prioritizes available accelerators and falls back to OpenGL for broader coverage. LiteRT abstracts vendor-specific NPU SDKs to handle SoC fragmentation, supports generative AI cross-platform deployment, and offers PyTorch/JAX model conversion. The project is available on GitHub.
#on-device-inference #gpu-acceleration-ml-drift #npu-deployment #cross-platform-deployment #pytorchjax-support
Read at www.infoworld.com
Unable to calculate read time
Collection
[
|
...
]