完全离线环境下安装Docker,Docker-compose,Airflow

准备工作#

下载docker:https://download.docker.com/linux/static/stable/x86_64

我下载的是docker-20.10.6.tgz

下载docker-compose:https://github.com/docker/compose/releases

我下载的是docker-compose-linux-x86_64

构建airflow镜像:#

Dockerfile

Copy
FROM python:3.8.12-buster ENV TZ="Asia/Shanghai" RUN sed -i s@/deb.debian.org/@/mirrors.aliyun.com/@g /etc/apt/sources.list RUN sed -i s@/security.debian.org/@/mirrors.aliyun.com/@g /etc/apt/sources.list RUN apt-get clean RUN apt-get -y update RUN apt-get -y install vim RUN pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple RUN pip install --upgrade pip COPY requirements.txt /requirements.txt RUN pip install -r /requirements.txt RUN rm /requirements.txt

requirements

Copy
apache_airflow==2.2.2 apache-airflow-providers-docker==2.3.0 psycopg2==2.9.2 pandas==1.3.5

build和save

Copy
docker build -t airflow:1.0 . docker save -o ./airflow.tar airflow:1.0

下载postgres数据库#

Copy
docker pull postgres:13 docker save -o ./ postgres:13

开始安装#

安装docker#

创建docker.service

Copy
[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target firewalld.service Wants=network-online.target [Service] Type=notify # the default is not to use systemd for cgroups because the delegate issues still # exists and systemd currently does not support the cgroup feature set required # for containers run by docker ExecStart=/usr/bin/dockerd ExecReload=/bin/kill -s HUP $MAINPID # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity # Uncomment TasksMax if your systemd version supports it. # Only systemd 226 and above support this version. #TasksMax=infinity TimeoutStartSec=0 # set delegate yes so that systemd does not reset the cgroups of docker containers Delegate=yes # kill only the docker process, not all processes in the cgroup KillMode=process # restart the docker process if it exits prematurely Restart=on-failure StartLimitBurst=3 StartLimitInterval=60s [Install] WantedBy=multi-user.target

依次执行以下命令

Copy
# 解压文件 tar -xvf docker-20.10.6.tgz # 移动docker到bin下 cp docker/* /usr/bin/ # 移动docker.service cp docker.service /etc/systemd/system/ # 设置权限 chmod +x /etc/systemd/system/docker.service # 重新加载配置文件 systemctl daemon-reload # 启动docker systemctl start docker # 开机自启 systemctl enable docker.service # 完成 docker -v

安装docker-compose#

Copy
mv docker-compose-linux-x86_64 /usr/local/bin/docker-compose

添加权限

Copy
chmod +x /usr/local/bin/docker-compose

测试

Copy
docker-compose -v

安装airflow#

创建docker-compose.yml

Copy
version: '3.8' x-airflow-common: &airflow-common image: airflow:1.0 environment: - AIRFLOW__CORE__EXECUTOR=LocalExecutor - AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql+psycopg2://airflow:airflow@postgres:5432/airflow - AIRFLOW__CORE__LOAD_EXAMPLES=False - AIRFLOW__CORE__DEFAULT_TIMEZONE=Asia/Shanghai - AIRFLOW__WEBSERVER__DEFAULT_UI_TIMEZONE=Asia/Shanghai - TZ=Asia/Shanghai - POSTGRES_ENGINE_STRING=postgresql+psycopg2://postgres:BBxy73i4YUGyHQtsDuQH2qe11ID624HETNXsdCbq@192.168.31.214:5432/stock - DOCKER_URL=tcp://192.168.31.214:2375 volumes: - /root/airflow/dags:/root/airflow/dags - /root/airflow/logs:/root/airflow/logs depends_on: - postgres services: postgres: container_name: airflow_postgres image: postgres:13 environment: - POSTGRES_USER=airflow - POSTGRES_PASSWORD=airflow - POSTGRES_DB=airflow - POSTGRES_PORT=5432 volumes: - /root/airflow/postgres/data:/var/lib/postgresql/data ports: - 54321:5432 airflow-init: <<: *airflow-common container_name: airflow_init entrypoint: /bin/bash command: - -c - airflow db check-migrations || ( airflow db init && airflow users create --role Admin --username airflow --password airflow --email airflow@airflow.com --firstname airflow --lastname airflow ) restart: on-failure airflow-webserver: <<: *airflow-common command: airflow webserver ports: - 8080:8080 container_name: airflow_webserver restart: always airflow-scheduler: <<: *airflow-common command: airflow scheduler container_name: airflow_scheduler restart: always depends_on: airflow-init: condition: service_completed_successfully

导入docker镜像

Copy
docker load --input airflow.tar docker load --input postgres.tar

启动docker

Copy
docker-compose up -d

启动完成后,打开http://192.168.31.214:8080就可以看到airflow了。

posted @   harrylyx  阅读(721)  评论(0编辑  收藏  举报
相关博文:
阅读排行:
· 震惊!C++程序真的从main开始吗?99%的程序员都答错了
· winform 绘制太阳,地球,月球 运作规律
· 【硬核科普】Trae如何「偷看」你的代码?零基础破解AI编程运行原理
· 上周热点回顾(3.3-3.9)
· 超详细:普通电脑也行Windows部署deepseek R1训练数据并当服务器共享给他人
点击右上角即可分享
微信分享提示
CONTENTS