ONNX MLIR应用示例（含源码链接）

ONNX MLIR应用示例（含源码链接）

开放式神经网络交换在MLIR中的实现 (http://onnx.ai/onnx-mlir/)。

 

 

 Prebuilt Containers

开始使用ONNX-MLIR的一个简单方法是使用预构建的docker映像。这些映像是在主干上成功合并生成的结果。最新的图像代表主干的顶端。目前，Docker Hub中保存的amd64、ppc64le和s390x的发布和调试模式映像分别为onnxmlirczar/onnx-mlir和onnxmlirczar/onnx-mlir-dev。要使用其中一个映像，可以直接从Docker Hub中取出，启动一个容器，运行一个交互式bash shell，或者用作dockerfile中的基础映像。onnx mlir映像只包含构建的编译器，可以立即使用编译模型，无需任何安装。提供了一个python便利脚本，在docker容器中运行ONNX-MLIR，就像直接在主机上运行ONNX-MLIR编译器一样。例如，

# docker/onnx-mlir.py --EmitLib mnist/model.onnx

505a5a6fb7d0: Pulling fs layer

505a5a6fb7d0: Verifying Checksum

505a5a6fb7d0: Download complete

505a5a6fb7d0: Pull complete

Shared library model.so has been compiled.

如果onnx mlir映像在本地不可用，脚本将提取该映像，将包含model.onnx的目录load到容器中，在同一目录中编译和生成model.so。

onnx mlir dev映像包含完整的构建树，其中包括先决条件和源代码的克隆。可以在容器中修改源代码，重建onnx-mlir，因此可以用作开发环境。还可以将vscode附加到正在运行的容器。docs文件夹中，可以看到一个对开发和vscode配置文件有用的Dockerfile示例。如果运行Docker build的目录中，不存在工作区目录和vscode文件，应注释掉或删除引用的行。

本文参考链接：

https://github.com/clang-ykt/llvm-project

Dockerfile显示在这里。

FROM onnxmlirczar/onnx-mlir-dev

WORKDIR /workdir

ENV HOME=/workdir

# 1) Install packages.

ENV PATH=$PATH:/workdir/bin

RUN apt-get update

RUN apt-get install -y python-numpy

RUN apt-get install -y python3-pip

RUN python -m pip install --upgrade pip

RUN apt-get install -y gdb

RUN apt-get install -y lldb

RUN apt-get install -y emacs

RUN apt-get install -y vim

# 2) Instal optional packages, uncomment/add as you see fit.

# RUN apt-get install -y valgrind

# RUN apt-get install -y libeigen3-dev

# RUN apt-get install -y clang-format

# RUN python -m pip install wheel

# RUN python -m pip install numpy

# RUN python -m pip install torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html

# RUN git clone https://github.com/onnx/tutorials.git

# Install clang-12.

# RUN apt-get install -y lsb-release wget software-properties-common

# RUN bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"

# 3) When using vscode, copy your .vscode in the Dockerfile dir and

#    uncomment the two lines below.

# WORKDIR /workdir/.vscode

# ADD .vscode /workdir/.vscode

# 4) When using a personal workspace folder, set your workspace sub-directory

#    in the Dockerfile dir and uncomment the two lines below.

# WORKDIR /workdir/workspace

# ADD workspace /workdir/workspace

# 5) Fix git by reattaching head and making git see other branches than main.

WORKDIR /workdir/onnx-mlir

RUN git checkout main

RUN git fetch --unshallow

# 6) Set the PATH environment vars for make/debug mode. Replace Debug

#    with Release in the PATH below when using Release mode.

WORKDIR /workdir

ENV MLIR_DIR=/workdir/llvm-project/build/lib/cmake/mlir

ENV NPROC=4

ENV PATH=$PATH:/workdir/onnx-mlir/build/Debug/bin/:/workdir/onnx-mlir/build/Debug/lib:/workdir/llvm-project/build/bin

Prerequisites

gcc >= 6.4

libprotoc >= 3.11.0

cmake >= 3.15.4

ninja >= 1.10.2

GCC可以在这里找到https://gcc.gnu.org/install/，或者如果有自制软件https://docs.brew.sh/Installation，可以使用brew安装GCC。要检查已安装的gcc版本，运行gcc--version。

安装libprotoc，或者，如果有自制软件，可以运行brew，安装protobuf。要检查已安装的版本，运行protoc--version。

可以在这里https://cmake.org/download/找到Cmake。但是，要使用Cmake，您需要遵循“如何安装以供命令行使用”教程，该教程可以在Cmake中的“工具”>“如何安装以供命令行使用”下找到。要检查您拥有的版本，您可以在桌面版本中的CMake>About下查看，或者运行CMake--version。

安装忍者的说明可以在这里找到https://ninja-build.org/。或者，使用自制软件，可以运行brew安装。要检查版本，运行ninja--version。

在任何时候，ONNX-MLIR都取决于LLVM项目的特定提交，该提交已证明与该项目一起工作。维护人员需要定期迁移到更新的LLVM级别。需要更新utils/clone-mlir.sh中的提交字符串。进行更改的结果是TravisCI构建将失败，直到重建包含prereqs的Docker映像。有一个GitHub工作流，可以为amd64体系结构重建映像，目前必须手动重建ppc64le和s390x映像。要完成的Dockerfiles在repo中。

Installation on UNIX

MLIR

Firstly, install MLIR (as a part of LLVM-Project):

git clone https://github.com/llvm/llvm-project.git

# Check out a specific branch that is known to work with ONNX MLIR.

cd llvm-project && git checkout 0bf230d4220660af8b2667506f8905df2f716bdf && cd ..

mkdir llvm-project/build

cd llvm-project/build

cmake -G Ninja ../llvm \

   -DLLVM_ENABLE_PROJECTS=mlir \

   -DLLVM_TARGETS_TO_BUILD="host" \

   -DCMAKE_BUILD_TYPE=Release \

   -DLLVM_ENABLE_ASSERTIONS=ON \

   -DLLVM_ENABLE_RTTI=ON

cmake --build . -- ${MAKEFLAGS}

cmake --build . --target check-mlir

ONNX-MLIR (this project)

The following environment variables can be set before building onnx-mlir (or alternatively, they need to be passed as CMake variables):

• MLIR_DIR should point to the mlir cmake module inside an llvm-project build or install directory (e.g., llvm-project/build/lib/cmake/mlir).

This project uses lit (LLVM's Integrated Tester) for unit tests. When running CMake, we can also specify the path to the lit tool from LLVM using the LLVM_EXTERNAL_LIT define but it is not required as long as MLIR_DIR points to a build directory of llvm-project. If MLIR_DIR points to an install directory of llvm-project, LLVM_EXTERNAL_LIT is required.

To build ONNX-MLIR, use the following commands:

git clone --recursive https://github.com/onnx/onnx-mlir.git

# Export environment variables pointing to LLVM-Projects.

export MLIR_DIR=$(pwd)/llvm-project/build/lib/cmake/mlir

mkdir onnx-mlir/build && cd onnx-mlir/build

cmake -G Ninja ..

cmake --build .

# Run lit tests:

export LIT_OPTS=-v

cmake --build . --target check-onnx-lit

If you are running on OSX Big Sur, you need to add -DCMAKE_CXX_COMPILER=/usr/bin/c++ to the cmake .. command due to changes in the compilers. After the above commands succeed, an onnx-mlir executable should appear in the bin directory.

LLVM and ONNX-MLIR CMake variables

The following CMake variables from LLVM and ONNX MLIR can be used when compiling ONNX MLIR.

MLIR_DIR:PATH Path to to the mlir cmake module inside an llvm-project build or install directory (e.g., c:/repos/llvm-project/build/lib/cmake/mlir). This is required if MLIR_DIR is not specified as an environment variable.

LLVM_EXTERNAL_LIT:PATH Path to the lit tool. Defaults to an empty string and LLVM will find the tool based on MLIR_DIR if possible. This is required when MLIR_DIR points to an install directory.

Installation on Windows

Building onnx-mlir on Windows requires building some additional prerequisites that are not available by default.

Note that the instructions in this file assume you are using Visual Studio 2019 Community Edition with ninja. It is recommended that you have the Desktop development with C++ and Linux development with C++ workloads installed. This ensures you have all toolchains and libraries needed to compile this project and its dependencies on Windows.

Run all the commands from a shell started from "Developer Command Prompt for VS 2019".

Protobuf

Build protobuf as a static library.

git clone --recurse-submodules https://github.com/protocolbuffers/protobuf.git

REM Check out a specific branch that is known to work with ONNX MLIR.

REM This corresponds to the v3.11.4 tag

cd protobuf && git checkout d0bfd5221182da1a7cc280f3337b5e41a89539cf && cd ..

set root_dir=%cd%

md protobuf_build

cd protobuf_build

call cmake %root_dir%\protobuf\cmake -G "Ninja" ^

   -DCMAKE_INSTALL_PREFIX="%root_dir%\protobuf_install" ^

   -DCMAKE_BUILD_TYPE=Release ^

   -Dprotobuf_BUILD_EXAMPLES=OFF ^

   -Dprotobuf_BUILD_SHARED_LIBS=OFF ^

   -Dprotobuf_BUILD_TESTS=OFF ^

   -Dprotobuf_MSVC_STATIC_RUNTIME=OFF ^

   -Dprotobuf_WITH_ZLIB=OFF

call cmake --build . --config Release

call cmake --build . --config Release --target install

Before running CMake for onnx-mlir, ensure that the bin directory to this protobuf is before any others in your PATH:

set PATH=%root_dir%\protobuf_install\bin;%PATH%

MLIR

Install MLIR (as a part of LLVM-Project):

git clone https://github.com/llvm/llvm-project.git

# Check out a specific branch that is known to work with ONNX MLIR.

cd llvm-project && git checkout 0bf230d4220660af8b2667506f8905df2f716bdf && cd ..

set root_dir=%cd%

md llvm-project\build

cd llvm-project\build

call cmake %root_dir%\llvm-project\llvm -G "Ninja" ^

   -DCMAKE_INSTALL_PREFIX="%root_dir%\llvm-project\build\install" ^

   -DLLVM_ENABLE_PROJECTS=mlir ^

   -DLLVM_TARGETS_TO_BUILD="host" ^

   -DCMAKE_BUILD_TYPE=Release ^

   -DLLVM_ENABLE_ASSERTIONS=ON ^

   -DLLVM_ENABLE_RTTI=ON ^

   -DLLVM_ENABLE_ZLIB=OFF

call cmake --build . --config Release

call cmake --build . --config Release --target install

call cmake --build . --config Release --target check-mlir

ONNX-MLIR (this project)

The following environment variables can be set before building onnx-mlir (or alternatively, they need to be passed as CMake variables):

• MLIR_DIR should point to the mlir cmake module inside an llvm-project build or install directory (e.g., c:/repos/llvm-project/build/lib/cmake/mlir).

This project uses lit (LLVM's Integrated Tester) for unit tests. When running CMake, we can also specify the path to the lit tool from LLVM using the LLVM_EXTERNAL_LIT define but it is not required as long as MLIR_DIR points to a build directory of llvm-project. If MLIR_DIR points to an install directory of llvm-project, LLVM_EXTERNAL_LIT is required.

To build ONNX MLIR, use the following commands:

git clone --recursive https://github.com/onnx/onnx-mlir.git

set root_dir=%cd%

md onnx-mlir\build

cd onnx-mlir\build

call cmake %root_dir%\onnx-mlir -G "Ninja" ^

   -DCMAKE_BUILD_TYPE=Release ^

   -DCMAKE_PREFIX_PATH=%root_dir%\protobuf_install ^

   -DLLVM_LIT_ARGS=-v ^

   -DMLIR_DIR=%root_dir%\llvm-project\build\lib\cmake\mlir

call cmake --build . --config Release --target onnx-mlir

To run the lit ONNX MLIR tests, use the following command:

call cmake --build . --config Release --target check-onnx-lit

To run the numerical ONNX MLIR tests, use the following command:

call cmake --build . --config Release --target check-onnx-numerical

To run the doc ONNX MLIR tests, use the following command after installing third_party ONNX:

call cmake --build . --config Release --target check-docs

After the above commands succeed, an onnx-mlir executable should appear in the bin directory.

LLVM and ONNX-MLIR CMake variables

The following CMake variables from LLVM and ONNX MLIR can be used when compiling ONNX MLIR.

MLIR_DIR:PATH Path to to the mlir cmake module inside an llvm-project build or install directory (e.g., c:/repos/llvm-project/build/lib/cmake/mlir). This is required if MLIR_DIR is not specified as an environment variable.

LLVM_EXTERNAL_LIT:PATH Path to the lit tool. Defaults to an empty string and LLVM will find the tool based on MLIR_DIR if possible. This is required when MLIR_DIR points to an install directory.

Using ONNX-MLIR

The usage of onnx-mlir is as such:

OVERVIEW: ONNX MLIR modular optimizer driver

USAGE: onnx-mlir [options] <input file>

OPTIONS:

Generic Options:

  --help        - Display available options (--help-hidden for more)

  --help-list   - Display list of available options (--help-list-hidden for more)

  --version     - Display the version of this program

ONNX MLIR Options:

These are frontend options.

  Choose target to emit:

      --EmitONNXBasic - Ingest ONNX and emit the basic ONNX operations without inferred shapes.

      --EmitONNXIR    - Ingest ONNX and emit corresponding ONNX dialect.

      --EmitMLIR      - Lower model to MLIR built-in transformation dialect.

      --EmitLLVMIR    - Lower model to LLVM IR (LLVM dialect).

      --EmitLib       - Lower model to LLVM IR, emit (to file) LLVM bitcode for model, compile and link it to a shared library.

Simple Example

For example, to lower an ONNX model (e.g., add.onnx) to ONNX dialect, use the following command:

./onnx-mlir --EmitONNXIR add.onnx

The output should look like:

module {

  func @main_graph(%arg0: tensor<10x10x10xf32>, %arg1: tensor<10x10x10xf32>) -> tensor<10x10x10xf32> {

    %0 = "onnx.Add"(%arg0, %arg1) : (tensor<10x10x10xf32>, tensor<10x10x10xf32>) -> tensor<10x10x10xf32>

    return %0 : tensor<10x10x10xf32>

  }

}

An example based on the add operation is found here, which build an ONNX model using a python script, and then provide a main program to load the model's value, compute, and print the models output.

End to end example

An end to end example is provided here, which train, compile, and execute a simple MNINST example.

Troubleshooting

If the latest LLVM project fails to work due to the latest changes to the MLIR subproject please consider using a slightly older version of LLVM. One such version, which we use, can be found here.

Installing third_party ONNX for Backend Tests or Rebuilding ONNX Operations

Backend tests are triggered by make check-onnx-backend in the build directory and require a few preliminary steps to run successfully. Similarily, rebuilding the ONNX operations in ONNX-MLIR from their ONNX descriptions is triggered by make OMONNXOpsIncTranslation.

You will need to install python 3.x if its not default in your environment, and possibly set the cmake PYTHON_EXECUTABLE varialbe in your top cmake file.

You will also need pybind11 which may need to be installed (mac: brew install pybind11 for example) and you may need to indicate where to find the software (Mac, POWER, possibly other platforms: export pybind11_DIR=<your path to pybind>). Then install the third_party/onnx software (Mac: pip install -e third_party/onnx) typed in the top directory.

On Macs/POWER and possibly other platforms, there is currently an issue that arises when installing ONNX. If you get an error during the build, try a fix where you edit the top CMakefile as reported in this PR: https://github.com/onnx/onnx/pull/2482/files.

Slack channel

We have a slack channel established under the Linux Foundation AI and Data Workspace, named #onnx-mlir-discussion. This channel can be used for asking quick questions related to this project.

 

参考链接：

https://github.com/onnx/onnx-mlir