4.1 python中调用rust程序

概述

使用rust-cpython将rust程序做为python模块调用；

通常为了提高python的性能；

 

参考-github

https://github.com/dgrunwald/rust-cpython

 

环境

系统：本次示例为ubantu20.04,等效于centos7

python:

python3,默认的版本，

root用户下默认的python通常是2版本，这是系统依赖的python版本，不要动root默认安装的东西；

新建一个用户，安装一个python3

 

创建rust lib库

cargo new rust2py --lib

或者使用IDE创建一个rust lib库项目

 

Cargo.toml

[package]
name = "rust2py"
version = "0.1.0"
edition = "2018"


[lib]
name = "rust2py"
crate-type = ["cdylib"]

[dependencies.cpython]
version = "0.7"
features = ["extension-module"]

 

lib.rs

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        assert_eq!(2 + 2, 4);
    }
}

#[macro_use]
extern crate cpython;

use cpython::{PyResult, Python, py_module_initializer, py_fn};


pub fn print_str(a: String) -> String {
    print!("{:#?}",a);
    a
}

pub fn print_str_py(_: Python, a: String) -> PyResult<String>{
    let mm = print_str(a);
    Ok(mm)
}


// logic implemented as a normal rust function
fn sum_as_str(a:i64, b:i64) -> String {
    format!("{}", a + b).to_string()
}

// rust-cpython aware function. All of our python interface could be
// declared in a separate module.
// Note that the py_fn!() macro automatically converts the arguments from
// Python objects to Rust values; and the Rust return value back into a Python object.
fn sum_as_str_py(_: Python, a:i64, b:i64) -> PyResult<String> {
    let out = sum_as_str(a, b);
    Ok(out)
}

py_module_initializer!(rust2py, init_rust2py, PyInit_rust2py, |py, m| {
    m.add(py, "__doc__", "This module is implemented in Rust.")?;
    m.add(py, "print_str", py_fn!(py, print_str_py(a: String)))?;
    m.add(py, "sum_as_str", py_fn!(py, sum_as_str_py(a: i64, b:i64)))?;
    Ok(())
});

 注意：py_module_initializer方法的参数的中rust2py一定要与模块的名称一致，这个不是随便写的字符串名称，比如PyInit_rust2py就表示将来在python中调用的模块名称是rust2py

 

编译并复制到python的模块

cargo build --release
cp target/debug/librust2py.so /opt/app/anaconda3/lib/python3.8/site-packages/rust2py.so

注意：复制到python模块的so没有lib前缀

可以换一个正规的python模块名称, 效果是一样的, 但这样的名字看起来更"专业"一点 ^_^

ai@aisty:/opt/app/anaconda3/envs/py37/lib/python3.7/site-packages$ mv rust2py.so rust2py.cpython-37m-x86_64-linux-gnu.so
ai@aisty:/opt/app/anaconda3/envs/py37/lib/python3.7/site-packages$ ll rust2py.cpython-37m-x86_64-linux-gnu.so 
-rwxrwxr-x 1 ai ai 5101552 12月  2 10:52 rust2py.cpython-37m-x86_64-linux-gnu.so*

 

封装一个自动安装的脚本install.sh

#!/bin/bash

cd /opt/wks/rust/rfil/rust2py/
/home/ai/.cargo/bin/cargo build --release
cp target/debug/librust2py.so /opt/app/anaconda3/envs/py37/lib/python3.7/site-packages/rust2py.cpython-37m-x86_64-linux-gnu.so

每次修改,执行一下脚本就会覆盖上一次的结果

(py37) ai@aisty:/opt/wks/rust/rfil/rust2py$ chmod +x install.sh 
(py37) ai@aisty:/opt/wks/rust/rfil/rust2py$ ./install.sh 

 

其他安装参考,setuptools-rust可以安装pyo3，也可以安装rust-cpython

https://github.com/PyO3/setuptools-rust

 

 

python调用模块

ai@aisty:/opt/app/anaconda3/lib/python3.8/site-packages$ python3.8
Python 3.8.5 (default, Sep  4 2020, 07:30:14) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rust2py
>>> rust2py.sum_as_str(2,5)
'7'
>>> rust2py.print_str("from rust")
'from rust'
>>> 

直接copy到site-packages目录下，有时IDE，比如vscode能加载模块名称，但无法访问其属性，可以将这个.so目录直接放到项目的python目录下

(py36_tf1.8_keras2.0) qisan@ai:~/73_code/aisty/test1/rust2py_t$ ls
rust2py.so  tt.py

 

输出：

(py36_tf1.8_keras2.0) qisan@ai:~/73_code$ /home/qisan/anaconda3/envs/py36_tf1.8_keras2.0/bin/python /home/qisan/73_code/aisty/test1/rust2py_t/tt.py
Hello qisan, I'm Python 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59) [GCC 7.5.0]
111

这样，.so包随python代码走，项目复制到哪.so文件就跟到哪

 

更好一点的做法是把.so文件放到一个模块中

(base) qisan@ai:~$ cd /home/qisan/73_code/mij
(base) qisan@ai:~/73_code/mij$ ls
__init__.py  __pycache__  rust2py.so
(base) qisan@ai:~/73_code/mij$ cat __init__.py 
from .rust2py import *

在__init__.py中引入这些模块的方法，后续有其他.so文件进来，也在这里统一引入，外部程序统一调用这个mij模块

import mij 

vsn = mij.hello()
print(vsn)

 提到python模块，就可以为这些外部方法包装一层python方法，补充上注释，可以方便阅读与调用。

 

接下来添加一个稍微复杂的方法:统计列表中元素的个数,输入Python列表,返回Python字典

fn elem_count(py: Python, pl: PyList) -> PyResult<PyDict> {
    let dt = PyDict::new(py);
    for e in pl.iter(py) {
        let el = &e;
        let ct = dt.contains(py,el).unwrap();

        if ct {
            let a = dt.get_item(py,el).unwrap().extract::<i32>(py).unwrap() + 1 ;
            dt.set_item(py,el,a)?;
        }else {
            dt.set_item(py,el,1)?;
        }

    }
    Ok(dt)
}

// https://dgrunwald.github.io/rust-cpython/doc/src/cpython/objects/dict.rs.html#129-143


py_module_initializer!(rust2py, init_rust2py, PyInit_rust2py, |py, m| {
    m.add(py, "__doc__", "This module is implemented in Rust.")?;
    m.add(py, "print_str", py_fn!(py, print_str_py(a: String)))?;
    m.add(py, "sum_as_str", py_fn!(py, sum_as_str_py(a: i64, b:i64)))?;
    m.add(py, "hello", py_fn!(py, hello_py()))?;
     m.add(py, "elem_count", py_fn!(py, elem_count(pl: PyList)))?;
    Ok(())
});

rust中的python方法通常返回一个PyResult,这是一个Python对象或Python异常的枚举,使用.unwrap()将之解析为一个Python对象,然后就可以调用Python对象的方法了,这些方法可以从后面介绍的文档上查看

>>> import rust2py
>>> rust2py.elem_count([1,2,3])
{1: 1, 2: 1, 3: 1}
>>> rust2py.elem_count([1,2,3,3])
{1: 1, 2: 1, 3: 2}

 

更多数据类型方法请参考

http://dgrunwald.github.io/rust-cpython/doc/cpython/

 

如果想知道更多的关于如何使用一个Py对象的细节,请看上面文件源码

每个py对象,后面都有一个[src]的标记,这是个超链接,点开之后会转向源码,比如PyDict,源码中有测试代码,对用法学习很有帮助

https://dgrunwald.github.io/rust-cpython/doc/src/cpython/objects/dict.rs.html#129-143

    #[test]
    fn test_items_list() {
        let gil = Python::acquire_gil();
        let py = gil.python();
        let mut v = HashMap::new();
        v.insert(7, 32);
        v.insert(8, 42);
        v.insert(9, 123);
        let dict = v.to_py_object(py);
        // Can't just compare against a vector of tuples since we don't have a guaranteed ordering.
        let mut key_sum = 0;
        let mut value_sum = 0;
        for el in dict.items_list(py).iter(py) {
            let tuple = el.cast_into::<PyTuple>(py).unwrap();
            key_sum += tuple.get_item(py, 0).extract::<i32>(py).unwrap();
            value_sum += tuple.get_item(py, 1).extract::<i32>(py).unwrap();
        }
        assert_eq!(7 + 8 + 9, key_sum);
        assert_eq!(32 + 42 + 123, value_sum);
    }

看源码,是最直接,直达本质的快捷学习通道!

 

全代码

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        assert_eq!(2 + 2, 4);
    }
}

#[macro_use]
extern crate cpython;

use cpython::{PyResult, Python, PyDict, py_module_initializer, py_fn, PyList};


pub fn print_str(a: String) -> String {
    print!("{:#?}",a);
    a
}

pub fn print_str_py(_: Python, a: String) -> PyResult<String>{
    let mm = print_str(a);
    Ok(mm)
}


// logic implemented as a normal rust function
fn sum_as_str(a:i64, b:i64) -> String {
    format!("{}", a + b).to_string()
}

// rust-cpython aware function. All of our python interface could be
// declared in a separate module.
// Note that the py_fn!() macro automatically converts the arguments from
// Python objects to Rust values; and the Rust return value back into a Python object.
fn sum_as_str_py(_: Python, a:i64, b:i64) -> PyResult<String> {
    let out = sum_as_str(a, b);
    Ok(out)
}

fn hello_py(py: Python) -> PyResult<String> {
    let sys = py.import("sys")?;
    let version: String = sys.get(py, "version")?.extract(py)?;


    let locals = PyDict::new(py);
    locals.set_item(py, "os", py.import("os")?)?;
    let user: String = py.eval("os.getenv('USER') or os.getenv('USERNAME')", None, Some(&locals))?.extract(py)?;

    let res = format!("Hello {}, I'm Python {}", user, version).to_string();
    let res = res.replace("\n","");
    Ok(res)
}

fn elem_count(py: Python, pl: PyList) -> PyResult<PyDict> {
    let dt = PyDict::new(py);
    for e in pl.iter(py) {
        let el = &e;
        let ct = dt.contains(py,el).unwrap();
        // let ct2 = match ct {
        //     Ok(b) => b,
        //     Err(e) => return Err(e),
        // };

        if ct {
            let a = dt.get_item(py,el).unwrap().extract::<i32>(py).unwrap() + 1 ;
            dt.set_item(py,el,a)?;
        }else {
            dt.set_item(py,el,1)?;
        }

    }
    Ok(dt)
}

// https://dgrunwald.github.io/rust-cpython/doc/src/cpython/objects/dict.rs.html#129-143


py_module_initializer!(rust2py, init_rust2py, PyInit_rust2py, |py, m| {
    m.add(py, "__doc__", "This module is implemented in Rust.")?;
    m.add(py, "print_str", py_fn!(py, print_str_py(a: String)))?;
    m.add(py, "sum_as_str", py_fn!(py, sum_as_str_py(a: i64, b:i64)))?;
    m.add(py, "hello", py_fn!(py, hello_py()))?;
     m.add(py, "elem_count", py_fn!(py, elem_count(pl: PyList)))?;
    Ok(())
});

 

 

 

pyo3与之类似

https://github.com/PyO3/pyo3

 

查看pyo3与python类型的对应

https://docs.rs/crate/pyo3/latest/source/src/types/

 

pyo3的一些测试用例

cargo new ij --lib

按官方文档的示例操作，然后执行

maturin develop

maturin develop会将项目 编译，然后安装到python