Jenkins file一行代码部署.NET程序到K8S

  • A+
所属分类：.NET技术
摘要

随着微服务的增多，每个项目的都需要pipline文件，这样的话Pipeline代码冗余度高，并且pipeline的功能越来越复杂。
便宜好用的国外VPS推荐

什么是Jenkins共享库

随着微服务的增多，每个项目的都需要pipline文件，这样的话Pipeline代码冗余度高，并且pipeline的功能越来越复杂。

jenkins可以使用Shared Lib，将一些公共的pipeline抽象做成模块代码，在各种项目pipeline之间复用，以减少冗余。

共享库目录结构

共享库根目录 |-- vars     |-- test1.groovy |-- src     |-- test2.groovy |-- resources

vars: 依赖于Jenkins运行环境的Groovy脚本。其中的Groovy脚本被称之为全局变量。
src: 标准的Java源码目录结构,其中的Groovy脚本被称为类库(Library class)。
resources: 目录允许从外部库中使用 libraryResource 步骤来加载有关的非 Groovy 文件。

引用共享库的方式

#!/usr/bin/env groovy  // 引用默认配置的共享库 @Library('demo-shared-library') _  // 引用指定分支、tag的共享库代码 @Library('demo-shared-library@1.0') _  // 引用多个指定分支tag的共享库 @Library('demo-shared-library@$Branch/Tag','demo-shared-library-test@$Branch/Tag') _  @Library('utils') import org.foo.Utilities  @Library('utils') import static org.foo.Utilities.*

vars下的全局变量

/vars下的全局变量必须以全小写或驼峰（camelCased）
/vars/*.groovy若实现call()方法，直接引用时默认执行其中的方法

实现一行代码部署.NET程序到K8S

安装Jenkins Master

# master docker run --name jenkins-blueocean -u root --privileged -d -p 8080:8080 -p 50000:50000 -v D:/architecture/jenkins/data:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock jenkinsci/blueocean   # 访问：http://ip:8080/ # jenkins密码，查看容器日志: 7285ced325a24483bfdaab227415fdac # 安装推荐插件

安装Jenkins Agent

 
Manage Jenkins -> Manage Nodes and Clouds -> New Node
  • Name: agent2/agent3
  • Labels: agentnode
  • Launch method: Launch agent by connecting it to the master

Build Agent Docker Image

# slave # Dockerfile FROM jenkins/inbound-agent:latest USER root  RUN apt-get update RUN apt-get -y install ca-certificates curl gnupg lsb-release RUN curl -fsSL https://get.docker.com -o get-docker.sh RUN sh get-docker.sh   RUN curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" RUN install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl RUN kubectl version --client  ENV JMETER_VERSION=5.4.1 ENV JMETER_HOME=/jmeter/apache-jmeter-${JMETER_VERSION} ENV JMETER_PATH=${JMETER_HOME}/bin:${PATH} ENV PATH=${JMETER_HOME}/bin:${PATH}  RUN mkdir /jmeter COPY apache-jmeter-${JMETER_VERSION}.tgz /jmeter RUN cd /jmeter && tar xvf apache-jmeter-${JMETER_VERSION}.tgz #其中tgz文件请去官网下载 RUN sed -i 's/#jmeter.save.saveservice.output_format=csv/jmeter.save.saveservice.output_format=xml/g'  /jmeter/apache-jmeter-5.4.1/bin/jmeter.properties

 

docker build -t chesterjenkinsagent . docker tag chesterjenkinsagent:latest 192.168.43.95/jenkins/chesterjenkinsagent:v1 docker login --username=admin --password=123456  192.168.43.95 docker push 192.168.43.95/jenkins/chesterjenkinsagent:v1

运行Agent，其中的密钥通过jenkins的agent信息获取

Jenkins file一行代码部署.NET程序到K8S

# agent4 docker run  -v /var/run/docker.sock:/var/run/docker.sock --name agent4  -d --init 192.168.43.95/jenkins/chesterjenkinsagent:v1 -url http://192.168.43.94:8080   1e84c896dbffc0c325587eedb6301ab0ae66d3f4b49c4628dbb05714e382d7a2 agent4

 

新增K8S凭据

  • 将k8s集群的配置文件导出 ~/.kube/config
  •  Mange Jenkins -> Manage Credentials -> Add Credentials -> Secret File
  • 选择导出的kubeconfig，设置id为kubeconfig

新增Harbor凭据

  • Mange Jenkins -> Manage Credentials -> Add Credentials -> Username with password
  • 输入Harbor的用户名密码

新增Gitee凭据

  • Mange Jenkins -> Manage Credentials -> Add Credentials -> Username with password
  • 输入Gitee的用户名密码

安装插件

Manage Jenkins -> Manage Plugins -> Search "Performance" -> install

管理共享库

Manage Jenkins -> Configure System -> Global Pipeline Libraries，其中的git地址是共享库的地址
Jenkins file一行代码部署.NET程序到K8S

共享库中新增以下代码

vars/run.groovy

#!/usr/bin/env groovy  def call(String nameSpaceName, String serviceName, String servicePath, String servicePort, String nodePort, Map envInfo) {     def devBranch = envInfo['dev']     def prodBranch = envInfo['prod']      pipeline {         agent {             label 'agentnode'         }          environment {             DEV_MY_KUBECONFIG = credentials('kubeconfig')             PROD_MY_KUBECONFIG = credentials('kubeconfig')             HARBOR = credentials('harbor')         }          stages {             stage('Dev - GitPull') {                 steps {                     deleteDir()                     gitCheckOut devBranch, env.GIT_URL                 }                 post {                     success {                         script {                             echo 'pull done'                         }                     }                 }             }             stage('Dev - DockerBuild') {                 steps {                     dockerImageBuild serviceName, "${servicePath}Dockerfile"                 }             }             stage('Dev - DockerTag') {                 steps {                     dockerTag serviceName, 'dev'                 }             }             stage('Dev - DockerLogin') {                 steps {                     dockerLogin 'dev'                 }             }             stage('Dev - DockerPush') {                 steps {                     dockerPush serviceName, 'dev'                 }             }             stage('Dev - GenerateHarborSecretYAML') {                 steps {                     harborSecret nameSpaceName, serviceName, 'dev'                 }             }             stage('Dev - GenerateK8SYAML') {                 steps {                     k8sGenerateYaml nameSpaceName, serviceName, servicePath, 'dev', servicePort, nodePort                 }             }             stage('Dev - DeployToK8S') {                 steps {                     k8sDeploy servicePath, 'dev'                 }             }             stage('Dev - CheckDeployStatus') {                 steps {                     k8sCheckDeployStatus nameSpaceName, serviceName, 'dev'                 }             }             stage('Dev - Jmeter Test') {                 steps {                     jmeterTest servicePath                 }             }              stage('DeployToProd?') {                 steps {                     input '部署生产？'                 }             }              stage('Prod - GitPull') {                 steps {                     gitCheckOut prodBranch, env.GIT_URL                 }             }             stage('Prod - DockerBuild') {                 steps {                     dockerImageBuild serviceName, "${servicePath}Dockerfile"                 }             }             stage('Prod - DockerTag') {                 steps {                     dockerTag serviceName, 'prod'                 }             }             stage('Prod - DockerLogin') {                 steps {                     dockerLogin 'prod'                 }             }             stage('Prod - DockerPush') {                 steps {                     dockerPush serviceName, 'prod'                 }             }             stage('Prod - GenerateHarborSecretYAML') {                 steps {                     harborSecret nameSpaceName, serviceName, 'prod'                 }             }             stage('Prod - GenerateK8SYAML') {                 steps {                     k8sGenerateYaml nameSpaceName, serviceName, servicePath, 'prod', servicePort, nodePort                 }             }             stage('Prod - DeployToK8S') {                 steps {                     k8sDeploy servicePath, 'prod'                 }             }             stage('Prod - CheckDeployStatus') {                 steps {                     k8sCheckDeployStatus nameSpaceName, serviceName, 'prod'                 }             }         }     } }

 

vars/dockerImageBuild.groovy

#!/usr/bin/env groovy  def call(String serviceName, String dockerfilePath) {     echo "serviceName:${serviceName} dockerfilePath:${dockerfilePath}"     sh "docker build -t ${serviceName} -f  ${dockerfilePath} ." }

 

vars/dockerLogin.groovy

#!/usr/bin/env groovy  def call(String envName) {     sh 'docker login --username=$HARBOR_USR --password=$HARBOR_PSW  192.168.43.95' }

 

vars/dockerPush.groovy

#!/usr/bin/env groovy  def call(String serviceName,String envName) {     sh "docker push 192.168.43.95/dapr/${serviceName}:${envName}-${BUILD_NUMBER}" }

 

vars/dockerTag.groovy

#!/usr/bin/env groovy  def call(String serviceName, String envName) {     sh "docker tag ${serviceName}:latest 192.168.43.95/dapr/${serviceName}:${envName}-${BUILD_NUMBER}" }

 

vars/gitCheckOut.groovy

#!/usr/bin/env groovy  def call(String branchName, String gitUrl) {     echo "branchName:${branchName} gitUrl:${gitUrl}"     checkout([$class: 'GitSCM', branches: [[name: branchName]], extensions: [], userRemoteConfigs: [[credentialsId: 'gitee', url: gitUrl]]]) }

 

vars/harborSecret.groovy

def call(String namespaceName, String serviceName, String envName) {     dir('harborsecret') {         checkout([$class: 'GitSCM', branches: [[name: '*/master']], extensions: [], userRemoteConfigs: [[credentialsId: 'gitee', url: 'https://gitee.com/chesterdotchen/jenkins-demo-secrets.git']]])         sh """sed -i 's/{{ServiceName}}/${serviceName}/g'  secrets.yaml"""         sh """sed -i 's/{{NameSpaceName}}/${namespaceName}/g'  secrets.yaml"""          if (envName == 'dev') {             sh("kubectl --kubeconfig  ${DEV_MY_KUBECONFIG} apply -f secrets.yaml")         }         if (envName == 'prod') {             sh("kubectl --kubeconfig  ${PROD_MY_KUBECONFIG} apply -f secrets.yaml")         }     } }

 

vars/jmeterTest.groovy

#!/usr/bin/env groovy  def call(String servicePath) {     sh "jmeter -j jmeter.save.saveservice.output_format=xml -n -t ${servicePath}jmeter.jmx -l ${servicePath}jmeter.report.jtl"     sh "cp ${servicePath}jmeter.report.jtl ${servicePath}jmeter.report.${BUILD_NUMBER}.jtl"     perfReport errorFailedThreshold:5, sourceDataFiles:"${servicePath}jmeter.report.jtl"     sh "cat ${servicePath}jmeter.report.${BUILD_NUMBER}.jtl"      sh """#!/bin/sh                             grep '<failure>true</failure>' ${servicePath}jmeter.report.${BUILD_NUMBER}.jtl                             if [ $? = 0 ]                             then                                 exit 1                             else                                 exit 0                             fi                             """ }

 

vars/k8sCheckDeployStatus.groovy

#!/usr/bin/env groovy  def call(String nameSpaceName, String serviceName, String envName) {     if (envName == 'dev') {         sh("""                     ATTEMPTS=0                     ROLLOUT_STATUS_CMD='kubectl --kubeconfig  ${DEV_MY_KUBECONFIG} rollout status deployment/${serviceName}  -n ${nameSpaceName}-ns'                     until $ROLLOUT_STATUS_CMD || [ $ATTEMPTS -eq 60 ]; do                         $ROLLOUT_STATUS_CMD                         ATTEMPTS=$((attempts + 1))                         sleep 10                     done                  """)     }     if (envName == 'prod') {         sh("""                     ATTEMPTS=0                     ROLLOUT_STATUS_CMD='kubectl --kubeconfig  ${PROD_MY_KUBECONFIG} rollout status deployment/${serviceName}  -n ${nameSpaceName}-ns'                     until $ROLLOUT_STATUS_CMD || [ $ATTEMPTS -eq 60 ]; do                         $ROLLOUT_STATUS_CMD                         ATTEMPTS=$((attempts + 1))                         sleep 10                     done                  """)     } }

 

vars/k8sDeploy.groovy

#!/usr/bin/env groovy  def call(String servicePath, String envName) {     if (envName == 'dev') {         sh("kubectl --kubeconfig  ${DEV_MY_KUBECONFIG} apply -f ${servicePath}deployment.yaml")     }     if (envName == 'prod') {         sh("kubectl --kubeconfig  ${PROD_MY_KUBECONFIG} apply -f ${servicePath}deployment.yaml")     } }

 

vars/k8sGenerateYaml.groovy

#!/usr/bin/env groovy  def call(String namespaceName, String serviceName, String servicePath, String envName, String servicePort, String nodePort) {     sh """sed "s/{{tagversion}}/${envName}-${BUILD_NUMBER}/g"  ${servicePath}deployment.yaml.tpl > ${servicePath}deployment.yaml """     sh """sed -i 's/{{ServiceName}}/${serviceName}/g'  ${servicePath}deployment.yaml"""     sh """sed -i 's/{{ServicePort}}/${servicePort}/g'  ${servicePath}deployment.yaml"""     sh """sed -i 's/{{NodePort}}/${nodePort}/g'  ${servicePath}deployment.yaml"""     sh """sed -i 's/{{NameSpaceName}}/${namespaceName}/g'  ${servicePath}deployment.yaml""" }

 

jenkins-demo-secrets中代码如下

apiVersion: v1 kind: Namespace metadata:   name: {{NameSpaceName}}-ns  --- apiVersion: v1 kind: Secret metadata:  name: harbor-key  namespace: {{NameSpaceName}}-ns type: kubernetes.io/dockerconfigjson data:  .dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjQzLjk1IjogewoJCQkiYXV0aCI6ICJZV1J0YVc0Nk1USXpORFUyIgoJCX0sCgkJInJlZ2lzdHJ5LmNuLWJlaWppbmcuYWxpeXVuY3MuY29tIjogewoJCQkiYXV0aCI6ICI2Wm1JNUxpQTU0dXVPbU5vWlc1NWFYTm9hVEV5TXc9PSIKCQl9Cgl9Cn0=

 dockerconfigjson可通过以下方式获取

docker login --username=admin --password=123456 192.168.43.95 cat ~/.docker/config.json | base64

Jenkinsfile中引用共享库

项目库中需要提前编写好Dockerfile，Jenkinsfile，deployment.yaml.tpl，jmeter.jmx

Jenkins file一行代码部署.NET程序到K8S

 

Dockerfile

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.  FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base WORKDIR /app EXPOSE 5001  FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build WORKDIR /src COPY ["FrontEnd/FrontEnd.csproj", "FrontEnd/"] COPY ["Common/Common.csproj", "Common/"] RUN dotnet restore "FrontEnd/FrontEnd.csproj" COPY . . WORKDIR "/src/FrontEnd" RUN dotnet build "FrontEnd.csproj" -c Release -o /app/build  FROM build AS publish RUN dotnet publish "FrontEnd.csproj" -c Release -o /app/publish  FROM base AS final WORKDIR /app COPY --from=publish /app/publish . ENTRYPOINT ["dotnet", "FrontEnd.dll"]

Jenkinsfile

#!/usr/bin/env groovy  @Library('share@master') _  run 'daprtest', 'frontend', './FrontEnd/', '5001', '31111', ['dev':'*/master', 'prod':'*/master']

deployment.yaml.tpl

apiVersion: v1 kind: Namespace metadata:   name: {{NameSpaceName}}-ns   --- apiVersion: dapr.io/v1alpha1 kind: Component metadata:   name: statestore   namespace: {{NameSpaceName}}-ns spec:   type: state.redis   version: v1   metadata:   - name: redisHost     value: 192.168.43.102:6379   - name: redisPassword     value: "123456"   - name: actorStateStore     value: "true"  --- apiVersion: apps/v1 kind: Deployment metadata:   name: {{ServiceName}}   namespace: {{NameSpaceName}}-ns   labels:     app: {{ServiceName}} spec:   replicas: 1   selector:     matchLabels:       app: {{ServiceName}}   template:     metadata:       namespace: {{NameSpaceName}}-ns       labels:         app: {{ServiceName}}       annotations:         dapr.io/enabled: "true"         dapr.io/app-id: "{{ServiceName}}"         dapr.io/app-port: "{{ServicePort}}"     spec:       imagePullSecrets:       - name: harbor-key       containers:       - name: {{ServiceName}}         image: 192.168.43.95/dapr/{{ServiceName}}:{{tagversion}}         ports:         - containerPort: {{ServicePort}}         imagePullPolicy: Always  --- apiVersion: v1 kind: Service metadata:   namespace: {{NameSpaceName}}-ns   name: {{ServiceName}}-svc spec:   type: NodePort   selector:     app: {{ServiceName}}   ports:     - port: {{ServicePort}}       targetPort: {{ServicePort}}       nodePort: {{NodePort}}

jmter.jmx则根据需要自行编写

Jenkins file一行代码部署.NET程序到K8S

新建pipline

其中的URL指向自己项目的git地址

Jenkins file一行代码部署.NET程序到K8S

构建pipline

Jenkins file一行代码部署.NET程序到K8S

至此，我们已经完成了Jenkinsfile中一行代码部署我们的项目，如果有新的service，在编写好Dockerfile，deployment.yaml.tpl（保持一致），jmeter.jmx的前提下，我们只需要Jenkinsfile中通过共享库引用run方法即可

 

便宜好用的国外VPS推荐