使用CI/CD在Gitlab上自动化部署python项目

服务器权限申请，涉及（SSH_PRIVATE_KEY、SSH_SERVER_HOSTKEYS）
编辑Dockerfile
编辑docker-compose.yml
编辑 update.sh
编辑.gitlab-ci.yml

服务器权限申请

本地生成密钥和公钥

ssh-keygen
# 输入密钥和公钥的存储路径
# > C:\Users\[用户名]\.ssh\[文件名]
# eg：C:Users\Evan\.ssh\ubuntu_id_rsa

公钥为该目录下的ubuntu_id_rsa.pub

密钥为该目录下的ubuntu_id_rsa

将生成的公钥发送给管理服务器的mentor
mentor将公钥添加到authorized_keys文件中
使用mentor分配的vps信息登录服务器
```
ssh -i C:\Users\Evan\.ssh\ubuntu_id_rsa [vps_user]:[vps_ip]
```
出现服务器的欢迎登录的界面，说明公钥已正确配置
获取SSH_PRIVATE_KEY
```
cat C:\Users\Evan\.ssh\ubuntu_id_rsa
```
密钥需要注意以-----BEGIN OPENSSH PRIVATE KEY-----开头-----END OPENSSH PRIVATE KEY-----结尾
获取SSH_SERVER_HOSTKEYS
```
ssh-keyscan [vps_ip]
```
需要本地机器已经授权，未知机器无法获取内容

Dockerfile文件配置

该文件用于构建项目的docker的镜像，尽量使用分层构建的方式，避免镜像的大小过于庞大

示例

FROM python:3.10-slim-buster AS builder

ENV PIP_NO_CACHE_DIR=1

RUN echo "deb http://mirrors.aliyun.com/debian bullseye main" > /etc/apt/sources.list && \
    echo "deb http://mirrors.aliyun.com/debian-security bullseye-security main" >> /etc/apt/sources.list && \
    apt-get update && \
    apt-get install -y --no-install-recommends build-essential && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/

FROM python:3.10-slim-buster
ENV TZ=Asia/Shanghai

RUN echo "deb http://mirrors.aliyun.com/debian bullseye main" > /etc/apt/sources.list && \
    echo "deb http://mirrors.aliyun.com/debian-security bullseye-security main" >> /etc/apt/sources.list && \
    apt-get update && \
    apt-get install -y --no-install-recommends cifs-utils tzdata && \
    ln -sf /usr/share/zoneinfo/$TZ /etc/localtime && \
    echo $TZ > /etc/timezone && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

RUN mkdir -p /mnt/remote_share

WORKDIR /app

COPY --from=builder /usr/local/lib/python3.10/site-packages /usr/local/lib/python3.10/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin

COPY . .

EXPOSE 8077

CMD mount -t cifs //[ip]/shares /mnt/remote_share -o username=xxx,password=xxxx,iocharset=utf8 && uvicorn src.main:app --host 0.0.0.0 --port 8077  --workers 8

注意：
这里选择了 python:3.10-slim-buster 作为基础镜像，还可以选择 python:3.10-alpine, 该基础镜像的大小更小；
cifs-utils用于挂载公盘，账号登陆时应选择永久账号，否则每月需要修改公盘密码

Docker Compose 文件编写

该文件用于启动镜像

示例

version: '3'
services:
  ptcg:
    image: [域名]/xxx/ptcg:latest
    restart: always
    container_name: ptcg
    ports:
      - '8077:8077'
    shm_size: '256m'
    privileged: true
    volumes:
      - /data/app/ptcg/template:/app/template
      - /data/app/ptcg/logs:/app/logs
      - /data/app/ptcg/config:/app/config
      - /data/app/ptcg/backup:/app/backup
      - /data/app/ptcg/output:/app/output
      - /data/app/ptcg/data:/app/data

注意:
versions字段在新版的docker中已经废弃
image是远程镜像名称
volumes是需要挂载出来的路径，由于app和docker-compose文件在服务器上的位置不一致，所以挂载路径需要填写宿主机的绝对路径

update.sh文件编写

#!/bin/bash

REGISTRY=
REGISTRY_USER=
REGISTRY_PASSWORD=
# login
echo "$REGISTRY_PASSWORD" | docker login -u "$REGISTRY_USER" --password-stdin $REGISTRY

cd /data/docker-compose/ptcg
docker compose pull
docker compose down && docker compose up -d

docker logout

文件存放在服务器的/data/docker-compose/[项目名称]/update.sh

registry的相关配置信息参照confluence

sh文件需要注意\r符号问题，The character '\r' is carriage return. It returns the cursor to the start of the line.

GitLab CI/CD文件编写

gitlab仓库需要将CI/CD控制打开

示例

image: docker:20.10.20

services:
  - name: docker:20.10.20-dind
    command: ["--tls=false"]

stages:
  - build
  - deploy

variables:
  DOCKER_IMAGE: registry.xxx.com/xxx/ptcg:latest
  SSH_PRIVATE_KEY: |
    -----BEGIN OPENSSH PRIVATE KEY-----
    ...
    -----END OPENSSH PRIVATE KEY-----
  SSH_SERVER_HOSTKEYS: 
  CI_REGISTRY_PWD:
  CI_REGISTRY_USER:
  CI_REGISTRY:

before_script:
  - echo $CI_REGISTRY_PWD | docker login -u ${CI_REGISTRY_USER} --password-stdin ${CI_REGISTRY}

build:
  stage: build
  script:
    - docker build -t $DOCKER_IMAGE .
    - docker push $DOCKER_IMAGE
  only:
    - fastapi-release

deploy:
  stage: deploy
  script:
    - eval $(ssh-agent -s)
    - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -v -
    - mkdir -p ~/.ssh && chmod 700 ~/.ssh
    - known_hosts_key=$SSH_SERVER_HOSTKEYS
    - echo "$known_hosts_key" > ~/.ssh/known_hosts && chmod 644 ~/.ssh/known_hosts
    - ssh -v -tty itadmin@[ip] "/data/docker-compose/ptcg/update.sh; docker image prune -f; exit;"
  only:
    - fastapi-release

注意：
only字段需要和CI/CD当前的分支相同