ONNX Standard And Its Significance For Data Scientists

PyTorch, TensorFlow, Caffe2, MXNet and so forth are the most well-liked frameworks for deep learning fashions. Each framework has its personal execs and cons. But, what if we are able to mix the benefits of all these frameworks to optimise DL fashions? Open Neural Network Exchange (ONNX) was the results of this lightbulb concept.

What is the ONNX commonplace?

In September 2017, Microsoft and Facebook launched the ONNX format — a standard for deep studying that allows fashions to be transferred between completely different frameworks. ONNX breaks the dependence between frameworks and {hardware} architectures. It has in a short time emerged because the default commonplace for portability and interoperability between deep studying frameworks.

Before ONNX, knowledge scientists discovered it troublesome to select from a variety of AI frameworks accessible. Developers might choose a sure framework on the outset of the venture, through the analysis and growth stage, however might require a very completely different set of options for manufacturing. With no concrete resolution to those issues, corporations have been pressured to resort to inventive and sometimes cumbersome workarounds, together with translating fashions by hand.

ONNX commonplace goals to bridge the hole and allow AI builders to modify between frameworks based mostly on the venture’s present stage. Currently, the fashions supported by ONNX are Caffe, Caffe2, Microsoft Cognitive toolkit, MXNET, PyTorch. ONNX additionally provides connectors for different commonplace libraries and frameworks.

“ONNX is the first step toward an open ecosystem where AI developers can easily move between state-of-the-art tools and choose the combination that is best for them,” Facebook had mentioned in an earlier weblog. It was particularly designed for the event of machine studying and deep studying fashions. It features a definition for an extensible computation graph mannequin together with built-in operators and commonplace knowledge sorts.

ONNX is a regular format for each DNN and conventional ML fashions. The interoperability format of the ONNX offers knowledge scientists with the pliability to selected their framework and instruments to speed up the method, from the analysis stage to the manufacturing stage. It additionally permits {hardware} builders to optimise deep learning-focused {hardware} based mostly on a regular specification appropriate with completely different frameworks.

See Also

Two use circumstances the place ONNX has been efficiently adopted embody:

  • TensorRT: NVIDIA’s platform for top efficiency deep studying inference. It utilises ONNX to assist a variety of deep studying frameworks. 
  • Qualcomm Snapdragon NPE: The Qualcomm neural processing engine (NPE) SDK provides assist for neural community analysis to cell gadgets. While NPE immediately helps solely Caffe, Caffe 2 and TensorFlow frameworks, ONNX format helps in not directly supporting a wider vary of frameworks.

ONNX Runtime

Optimising machine studying fashions for inference is troublesome because it requires tuning the mannequin and inference library to profit from the {hardware} capabilities. It is a much bigger problem once we are attempting to attain optimum efficiency throughout completely different platforms — cloud, CPU, GPU — as every platform has its capabilities and traits. The complexity will increase when fashions from a wide range of frameworks are required to be run on completely different platforms. Optimising completely different combos of frameworks and {hardware} is a time-consuming job. The ONNX commonplace helps by permitting the mannequin to be educated in the popular framework after which run it wherever on the cloud. Models from frameworks, together with TensorFlow, PyTorch, Keras, MATLAB, SparkML might be exported and transformed to straightforward ONNX format. Once the model is within the ONNX format, it may well run on completely different platforms and gadgets.

ONNX Runtime is the inference engine for deploying ONNX fashions to manufacturing. The options embody:

  •  It is written in C++ and has C, Python, C#, and Java APIs for use in numerous environments.
  • It can be utilized on each cloud and edge and works equally nicely on Linux, Windows, and Mac.
  • ONNX Runtime helps DNN and conventional machine studying. It can combine with accelerators on completely different {hardware} platforms similar to NVIDIA GPUs, Intel processors, and DirectML on Windows.
  • ONNX Runtime provides intensive production-grade optimisation, testing, and different enhancements


  • ONNX format is comparatively new. Lack of use circumstances might increase doubts on its reliability and ease of use.
  • For straightforward utilization, two circumstances have to be mandatorily met — use of solely supported knowledge sorts and operations; no customisation by way of particular layers/operations is carried out.
  • Since its launch, the ONNX venture has seen fast growth. On the one hand, the brand new variations improve compatibility between frameworks; nonetheless, in circumstances the place preliminary circumstances are usually not met, the developer would wish to make customized implementations within the backend, which is a really time-consuming and laborious course of.

Subscribe to our Newsletter

Get the most recent updates and related provides by sharing your e-mail.

Join Our Telegram Group. Be a part of an interesting on-line neighborhood. Join Here.


Shraddha Goled

Shraddha Goled

I’m a journalist with a postgraduate diploma in pc community engineering. When not studying or writing, one can discover me doodling away to my coronary heart’s content material.


Please enter your comment!
Please enter your name here