kedro package 命令内部处理简单说明

kedro package 是一个比较方便的能力，可以将我们开发的data pipeline 打包为python 包，方便分发以及使用，以下简单说明下内部实现

参考内部处理

• 代码
  可以看到内部并没有特殊的地方，对于标准python whl 文件基于了build 模块进行构建，对于配置部分使用了tar 命令进行处理

@project_group.command()

@click.pass_obj  # this will pass the metadata as first argument

def package(metadata: ProjectMetadata) -> None:

    """Package the project as a Python wheel."""

    # Even if the user decides for the older setup.py on purpose,

    # pyproject.toml is needed for Kedro metadata

    if (metadata.project_path / "pyproject.toml").is_file():

        metadata_dir = metadata.project_path

        destination_dir = "dist"

    else:

        # Assume it's an old Kedro project, packaging metadata was under src

        # (could be pyproject.toml or setup.py, it's not important)

        metadata_dir = metadata.source_dir

        destination_dir = "../dist"

 

    call(

        [

            sys.executable,

            "-m",

            "build",

            "--wheel",

            "--outdir",

            destination_dir,

        ],

        cwd=str(metadata_dir),

    )

 

    directory = (

        str(Path(settings.CONF_SOURCE).parent)

        if settings.CONF_SOURCE != "conf"

        else metadata.project_path

    )

    call(

        [

            "tar",

            "--exclude=local/*.yml",

            "-czf",

            f"dist/conf-{metadata.package_name}.tar.gz",

            f"--directory={directory}",

            str(Path(settings.CONF_SOURCE).stem),

        ]

    )

参考资料

https://docs.kedro.org/en/stable/kedro_project_setup/settings.html
https://docs.kedro.org/en/stable/tutorial/package_a_project.html