自定义Jenkins-slave镜像

CI/CD 评论

概述：

在前面章节，我们提到过通过Jenkins对接Kubernetes能实现对Jenkins slave的弹性部署和回收，能非常有效的利用资源，在实际使用的过程中，我们会根据不同的编程语言去构建一个对应的jnlp，或在一个jnlp镜像内包含全部编程语言。下面讲解一下如何定制自己的jnlp镜像。

软件版本：
os：ubuntu 16.04
Kubernetes：1.12.0
jenkins：2.89.4

制做方法：

基于标准操作系统base镜像构建，我这里使用debian，当然也可以使用centos，ubuntu，alpine等其他镜像（以java语言环境为例，构建maven编译环境，其他开发语言类似)

1、安装配置JAVA环境和maven编译环境，下载jdk，maven
https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
下载Linux x64 tar包，解压到Dockerfile目录

http://mirror.bit.edu.cn/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
解压tar包，到Dockerfile目录

2、安装配置Jenkins-slave
https://github.com/jenkinsci/docker-jnlp-slave/blob/master/jenkins-slave
下载jenkins-slave启动脚本，并放置到 /usr/local/bin/目录

3、安装slave.jar包
slave.jar包主要用于Jenkins-slave去连接Jenkins-server所用
https://repo.jenkins-ci.org/public/org/jenkins-ci/main/remoting/3.16/remoting-3.16.jar
下载slave.jar包并放置到/usr/share/jenkins/目录

4、安装配置docker-in-docker
下载docker二进制包
https://download.docker.com/linux/static/stable/x86_64/
docker-17.03.2-ce.tgz
解压后，放到Dockerfile目录

5、下载kubectl二进制包
https://dl.k8s.io/v1.11.3/kubernetes-client-linux-amd64.tar.gz

解压后，放到Dockerfile目录

6、放置连接Kubernetes集群的kubeconfig文件
拷贝连接集群的kubeconfig文件，重命名为config

1
mv kubeconfig config

基于以下Dockerfile构建镜像

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
FROM debian:9
USER root
ARG AGENT_WORKDIR=/home/jenkins/agent
RUN mkdir -p /home/jenkins/.jenkins && mkdir -p ${AGENT_WORKDIR}
COPY jdk1.8.0_181 /usr/local/jdk1.8.0_181
COPY maven3.3.9 /usr/local/maven3.3.9
COPY jenkins-slave /usr/local/bin/jenkins-slave
COPY slave.jar /usr/share/jenkins/
COPY docker /usr/local/docker
COPY kubectl /usr/bin/
COPY config /root
ENV PATH=/usr/local/jdk1.8.0_181/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/maven3.3.9/bin:/usr/local/docker
RUN chmod +x /usr/local/bin/jenkins-slave
ENV HOME /home/jenkins
RUN chmod 755 /usr/share/jenkins && chmod 644 /usr/share/jenkins/slave.jar
USER root
ENV AGENT_WORKDIR=${AGENT_WORKDIR}
WORKDIR /home/jenkins
ENTRYPOINT ["jenkins-slave"]

测试用例
https://github.com/wanshaoyuan/pipeline-exemple.git

测试的Jenkins的pipeline
注意：deployment.yaml里面应用部署在sock-shop这个命名空间内，需要提前创建好。
jenkins-pipeline

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
podTemplate(label: 'jnlp-slave', cloud: 'kubernetes') {
    node('jnlp-slave') {
    stage('git clone'){
        //clone CODE根据实际情况修改
        git credentialsId: 'ba80cc36-77b1-42d0-8838-f834de346fe1', url: 'https://github.com/wanshaoyuan/pipeline-exemple.git'

    }
    stage('code build'){
        sh '''/usr/local/maven3.3.9/bin/mvn -DskipTests package -f /home/jenkins/workspace/test-jnlp2'''
    }
    stage('image build'){
        sh '''echo $BUILD_ID
              GROUP=172.31.164.66/library COMMIT=$BUILD_ID /home/jenkins/workspace/test-jnlp2/scripts/build.sh'''
    }
    stage('upload'){
        //上传镜像
        sh '''docker login 172.31.164.66 -u=admin -p=123456
        docker push 172.31.164.66/library/shipping:$BUILD_ID'''
    }
    stage('deploy'){
        //执行部署脚本
       sh 'sed -i "s/172.31.164.66\\/library\\/shipping:.*/172.31.164.66\\/library\\/shipping:$BUILD_ID/g" /home/jenkins/workspace/test-jnlp2/deployment.yml'
       sh 'kubectl apply -f /home/jenkins/workspace/test-jnlp2/deployment.yml --kubeconfig /root/config'

  }

 }
}

配置jnlp映射宿主机的docker.sock文件,不然无法实现容器镜像build

执行完毕

最后访问集群30001

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
kubectl get svc -n sock-shop
NAME           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
carts          ClusterIP   10.43.23.102    <none>        80/TCP         1d
carts-db       ClusterIP   10.43.229.245   <none>        27017/TCP      1d
catalogue      ClusterIP   10.43.133.136   <none>        80/TCP         1d
catalogue-db   ClusterIP   10.43.156.93    <none>        3306/TCP       1d
front-end      NodePort    10.43.130.209   <none>        80:30001/TCP   1d
orders         ClusterIP   10.43.212.179   <none>        80/TCP         1d
orders-db      ClusterIP   10.43.178.3     <none>        27017/TCP      1d
payment        ClusterIP   10.43.205.54    <none>        80/TCP         1d
queue-master   ClusterIP   10.43.178.73    <none>        80/TCP         1d
rabbitmq       ClusterIP   10.43.233.8     <none>        5672/TCP       1d
shipping       ClusterIP   10.43.198.185   <none>        80/TCP         1d
user           ClusterIP   10.43.213.139   <none>        80/TCP         1d
user-db        ClusterIP   10.43.94.66     <none>        27017/TCP      1d

http://host_ip:30001