容器云搭建
2.1.1 部署Kubernetes容器云平台
使用OpenStack私有云平台创建两台云主机,云主机类型使用4vCPU/12G/100G类型,分别作为Kubernetes集群的Master节点和node节点,然后完成Kubernetes集群部署。
2.1.2 部署Harbor镜像堆栈
在Kubernetes集群中完成Harbor镜像堆栈部署。
2.1.3 部署Istio服务网格
在Kubernetes集群中完成Istio服务网格组件部署。
2.1.4 部署kubeVirt 虚拟化组件
在Kubernetes集群中完成kubeVirt虚拟化组件部署。
- mount -o loop chinaskills_cloud_paas_v2.1.iso /mnt/
- cp -rfv /mnt/* /opt/
- umount /mnt/
- ## 在master节点安装kubeeasy工具:
- mv /opt/kubeeasy-v2.0 /usr/bin/kubeeasy
- ## 在master节点安装依赖包:
- kubeeasy install dependencies \
- --host 10.28.0.205,10.28.0.221 \
- --user root \
- --password Abc@1234 \
- --offline-file /opt/dependencies/packages.tar.gz
- ## 配置SSH免密:
- kubeeasy check ssh \
- --host 10.28.0.205,10.28.0.221 \
- --user root \
- --password Abc@1234
- kubeeasy create ssh-keygen \
- --master 10.28.2.191 \
- --worker 10.28.0.198 \
- --user root \
- --password Abc@1234
- ## master节点部署kubernetes
- kubeeasy install kubernetes \
- --master 10.24.2.10 \
- --worker 10.24.2.20,10.24.2.30,10.24.2.40 \
- --user root \
- --password 000000 \
- --version 1.25.2 \
- --offline-file /opt/kubeeasy.tar.gz
复制代码 容器云服务运维:
2.2.1 容器化部署Node-Exporter
编写Dockerfile文件构建exporter镜像,要求基于centos完成Node-Exporter服务的安装与配置,并设置服务开机自启。
编写Dockerfile构建monitor-exporter:v1.0镜像,具体要求如下:(必要用到的软件包:Monitor.tar.gz)
(1)底子镜像:centos:centos7.9.2009;
(2)使用二进制包node_exporter-0.18.1.linux-amd64.tar.gz安装node-exporter服务;
(3)声明端口:9100;
(4)设置服务开机自启。
- tar -zxvf Monitor.tar.gz
- docker load -i Monitor/CentOS_7.9.2009.tar
- cd Monitor/
- ##编写Dockerfile文件
- vim Dockerfile-exporter
- FROM centos:centos7.9.2009
- RUN rm -rf /etc/yum.repos.d/*
- ADD node_exporter-0.18.1.linux-amd64.tar.gz /root/
- EXPOSE 9100
- ENTRYPOINT ["./root/node_exporter-0.18.1.linux-amd64/node_exporter"]
- ##运行脚本
- docker build -t monitor-exporter:v1.0 -f Dockerfile-exporter .
复制代码 2.2.2容器化部署Alertmanager
编写Dockerfile文件构建alert镜像,要求基于centos:latest完成Alertmanager服务的安装与配置,并设置服务开机自启。
编写Dockerfile构建monitor-alert:v1.0镜像,具体要求如下:(必要用到的软件包:Monitor.tar.gz)
(1)底子镜像:centos:centos7.9.2009;
(2)使用二进制包alertmanager-0.19.0.linux-amd64.tar.gz安装Alertmanager服务;
(3)声明端口:9093、9094;
(4)设置服务开机自启。
- tar -zxvf Monitor.tar.gz
- docker load -i Monitor/CentOS_7.9.2009.tar
- cd Monitor/
- ##编写Dockerfile文件
- vim Dockerfile-alert
- FROM centos:centos7.9.2009
- RUN rm -rf /etc/yum.repos.d/*
- ADD alertmanager-0.19.0.linux-amd64.tar.gz /root/
- EXPOSE 9093 9094
- ENTRYPOINT ["./root/alertmanager-0.19.0.linux-amd64/alertmanager","--config.file","/root/alertmanager-0.19.0.linux-amd64/alertmanager.yml"]
- ##运行脚本
- docker build -t monitor-alert:v1.0 -f Dockerfile-alert .
复制代码 2.2.3 容器化部署Grafana
编写Dockerfile文件构建grafana镜像,要求基于centos完成Grafana服务的安装与配置,并设置服务开机自启。
编写Dockerfile构建monitor-grafana:v1.0镜像,具体要求如下:(必要用到的软件包:Monitor.tar.gz)
(1)底子镜像:centos:centos7.9.2009;
(2)使用二进制包grafana-6.4.1.linux-amd64.tar.gz安装grafana服务;
(3)声明端口:3000;
(4)设置nacos服务开机自启。
- tar -zxvf Monitor.tar.gz
- docker load -i Monitor/CentOS_7.9.2009.tar
- cd Monitor/
- ##编写Dockerfile文件
- vim Dockerfile-grafana
- FROM centos:centos7.9.2009
- RUN rm -rf /etc/yum.repos.d/*
- ADD grafana-6.4.1.linux-amd64.tar.gz /root/
- EXPOSE 3000
- ENTRYPOINT ["./root/grafana-6.4.1/bin/grafana-server","-homepath","/root/grafana-6.4.1/"]
- ##运行脚本
- docker build -t monitor-grafana:v1.0 -f Dockerfile-grafana .
复制代码 2.2.4 容器化部署Prometheus
编写Dockerfile文件构建prometheus镜像,要求基于centos完成Promethues服务的安装与配置,并设置服务开机自启。
编写Dockerfile构建monitor-prometheus:v1.0镜像,具体要求如下:(必要用到的软件包:Monitor.tar.gz)
(1)底子镜像:centos:centos7.9.2009;
(2)使用二进制包prometheus-2.13.0.linux-amd64.tar.gz安装promethues服务;
(3)编辑/data/prometheus/prometheus.yml文件,创建3个任务模板:prometheus、node和alertmanager,并将该文件拷贝到/data/prometheus/目次下;
(4)声明端口:9090;
(5)设置服务开机自启。
编写Dockerfile文件
- FROM centos:centos7.9.2009
- RUN rm -rf /etc/yum.repos.d/*
- ADD prometheus-2.13.0.linux-amd64.tar.gz /root/
- RUN mkdir -p /data/prometheus
- EXPOSE 9090
- RUN cat <<EOF > /data/prometheus/prometheus.yml
- global:
- scrape_interval: 15s
- scrape_configs:
- - job_name: prometheus
- static_configs:
- - targets: ['localhost:9090']
- - job_name: node
- static_configs:
- - targets: ['localhost:9090']
- - job_name: alertmanager
- static_configs:
- - targets: ['localhost:9090']
- - job_name: grafana:
- static_configs:
- - targets: ['localhost:9090']
- EOF
- ENTRYPOINT ["./root/prometheus-2.13.0.linux-amd64/prometheus","--config.file","/data/prometheus/prometheus.yml"]
复制代码 上面cat写入了 下面的prometheus.yml就不用再写了
编写prometheus.yml (如果写了下面的文件 必要在Dockerfile中COPY文件到/data/prometheus/)
- [root@master Monitor]# vim prometheus.yml
- global:
- scrape_interval: 15s
- evaluation_interval: 15s
- alerting:
- alertmanagers:
- - static_configs:
- - targets:
- - alertmanager: 9093
- rule_files:
- scrape_configs:
- - job_name: 'prometheus'
- static_configs:
- - targets: ['localhost:9090']
- - job_name: 'node'
- static_configs:
- - targets: ['node:9100']
- - job_name: 'alertmanager'
- static_configs:
- - targets: ['alertmanager:9093']
- - job_name: 'node-exporter'
- static_configs:
- - targets: ['node:9100']
复制代码 跑脚本
- docker build -t monitor-prometheus:v1.0 -f Dockerfile-prometheus .
复制代码 2.2.5 编排部署监控系统
编写docker-compose.yaml文件,使用镜像exporter、alert、grafana和prometheus完成监控系统的编排部署。
编写docker-compose.yaml文件,具体要求如下:
(1)容器1名称:monitor-node;镜像:monitor-exporter:v1.0;端口映射:9100:9100;
(2)容器2名称:monitor- alertmanager;镜像:monitor-alert:v1.0;端口映射:9093:9093、9094:9094;
(3)容器3名称:monitor-grafana;镜像:monitor-grafana:v1.0;端口映射:3000:3000;
(4)容器4名称:monitor-prometheus;镜像:monitor-prometheus:v1.0;端口映射:9090:9090。
完成后编排部署监控系统,将Prometheus设置为Grafana的数据源,并命名为Prometheus。
(5)添加元数据 进入grafana的网页 添加prometheus为数据源
编写docker-compose.yaml文件
- version: '3'
- services:
- # 容器1:用于监控节点的exporter服务
- monitor-node:
- image: monitor-exporter:v1.0
- ports:
- - "9100:9100"
- # 容器2:alertmanager服务
- monitor-alertmanager:
- image: monitor-alert:v1.0
- ports:
- - "9093:9093"
- - "9094:9094"
- # 容器3:grafana服务
- monitor-grafana:
- image: monitor-grafana:v1.0
- ports:
- - "3000:3000"
- # 容器4:prometheus服务
- monitor-prometheus:
- image: monitor-prometheus:v1.0
- ports:
- - "9090:9090"
复制代码 有依靠关系的写法;
- version: '3'
- services:
- node:
- container_name: monitor-node
- image: monitor-exporter:v1.0
- restart: always
- hostname: node
- ports:
- - 9100:9100
- alertmanager:
- container_name: monitor-alertmanager
- image: monitor-alert:v1.0
- depends_on:
- - node
- restart: always
- hostname: alertmanager
- links:
- - node
- ports:
- - 9093:9093
- - 9094:9094
- grafana:
- container_name: monitor-grafana
- image: monitor-grafana:v1.0
- depends_on:
- - node
- - alertmanager
- hostname: grafana
- restart: always
- links:
- - node
- - alertmanager
- ports:
- - 3000:3000
- prometheus:
- container_name: monitor-prometheus
- image: monitor-prometheus:v1.0
- depends_on:
- - node
- - alertmanager
- - grafana
- hostname: prometheus
- restart: always
- links:
- - node
- - alertmanager
- - grafana
- ports:
- - 9090:9090
复制代码 查察pod状态
- [root@master Monitor]# docker ps -a
- CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
- e4a643469259 monitor-prometheus:v1.0 "./root/prometheus-2…" 2 minutes ago Up 2 minutes 0.0.0.0:9090->9090/tcp, :::9090->9090/tcp monitor-prometheus
- cd1eddaba0d3 monitor-grafana:v1.0 "./root/grafana-6.4.…" 2 minutes ago Up 2 minutes 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp monitor-grafana
- 9032755f8e18 monitor-alert:v1.0 "./root/alertmanager…" 2 minutes ago Up 2 minutes 0.0.0.0:9093-9094->9093-9094/tcp, :::9093-9094->9093-9094/tcp monitor-alertmanager
- e3ae4d3bf8f9 monitor-exporter:v1.0 "./root/node_exporte…" 2 minutes ago Up 2 minutes 0.0.0.0:9100->9100/tcp, :::9100->9100/tcp monitor-node
复制代码 登录grafana网页
http://10.28.0.244:3000 账号admin 暗码随便(admin)
登录后会提示修改暗码 可以跳过
添加prometheus为数据源
输入主节点的ip加端口号
http://10.28.0.244:9090(普罗米修斯的端口)
然后点击下面绿色的生存 再点back退出
2.2.6 部署GitLab
将GitLab部署到Kubernetes集群中,设置GitLab服务root用户的暗码,使用Service暴露服务,并将提供的项目包导入到GitLab中。
在Kubernetes集群中新建命名空间gitlab-ci,将GitLab部署到该命名空间下,Deployment和Service名称均为gitlab,以NodePort方式将80端口对外暴露为30880,设置GitLab服务root用户的暗码为admin@123,将项目包demo-2048.tar.gz导入到GitLab中并命名为demo-2048。必要用到的软件包:CICD-Runners-demo2048.tar.gz
解压软件包,导入镜像
- [root@master ~]# tar -zxvf CICD-Runners-demo2048.tar.gz
- [root@master ~]# ctr -n k8s.io image import gitlab-ci/images
- /images.tar
- [root@master ~]# docker load < gitlab-ci/images/images.tar
复制代码 部署GitLab服务
- [root@master ~]# kubectl create ns gitlab-ci ## 新建命名空间
- [root@master ~]# cd gitlab-ci
- [root@master gitlab-ci]# vi gitlab-deploy.yaml
- apiVersion: apps/v1
- kind: Deployment
- metadata:
- name: gitlab
- namespace: gitlab-ci
- labels:
- name: gitlab
- spec:
- selector:
- matchLabels:
- name: gitlab
- template:
- metadata:
- name: gitlab
- labels:
- name: gitlab
- spec:
- containers:
- - name: gitlab
- image: gitlab/gitlab-ce:latest
- imagePullPolicy: IfNotPresent
- env:
- - name: GITLAB_ROOT_PASSWORD
- value: admin@123
- - name: GITLAB_ROOT_EMAIL
- value: 123456@qq.com
- ports:
- - name: http
- containerPort: 80
- volumeMounts:
- - name: gitlab-config
- mountPath: /etc/gitlab
- - name: gitlab-logs
- mountPath: /var/log/gitlab
- - name: gitlab-data
- mountPath: /var/opt/gitlab
- volumes:
- - name: gitlab-config
- hostPath:
- path: /home/gitlab/conf
- - name: gitlab-logs
- hostPath:
- path: /home/gitlab/logs
- - name: gitlab-data
- hostPath:
- path: /home/gitlab/data
复制代码 删除deployment资源的下令
- kubectl -n gitlab-ci delete -f gitlab-deploy.yaml
复制代码- [root@master gitlab-ci]# vi gitlab-svc.yaml ## 创建service服务释放端口
- apiVersion: v1
- kind: Service
- metadata:
- name: gitlab
- namespace: gitlab-ci
- labels:
- name: gitlab
- spec:
- type: NodePort
- ports:
- - name: http
- port: 80
- targetPort: http
- nodePort: 30880
- selector:
- name: gitlab
- [root@master gitlab-ci]# kubectl apply -f gitlab-deploy.yaml
- [root@master gitlab-ci]# kubectl apply -f gitlab-svc.yaml
- ## 查看pod
- [root@master gitlab-ci]# kubectl -n gitlab-ci get pod
- NAME READY STATUS RESTARTS AGE
- gitlab-65c6b98f6b-q4dwq 1/1 Running 0 2m3s
- [root@master gitlab-ci]# kubectl -n gitlab-ci get pods -owide ## 查看pod详细信息
- NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
- gitlab-65c6b98f6b-q4dwq 1/1 Running 0 2m57s 192.244.0.21 master <none> <none>
复制代码 在集群中界说hosts添加gitlabPod的解析
- [root@master gitlab-ci]# kubectl edit configmap coredns -n kube-system
- ... ...
- 16 fallthrough in-addr.arpa ip6.arpa
- 17 ttl 30
- 18 }
- 19 hosts {
- 20 192.244.0.21 gitlab-65c6b98f6b-q4dwq ## 这里是Pod容器的ip
- 21 fallthrough
- 22 }
- 23 prometheus :9153
- 24 ## 这里有三行删除
- 25 cache 30
- ... ...
- 保存退出 需要保存两遍
- [root@master gitlab-ci]# kubectl -n kube-system rollout restart deploy coredns ## 保存刚才的设置
复制代码 进入gitlab Pod中
- [root@master gitlab-ci]# kubectl -n gitlab-ci get pods
- [root@master gitlab-ci]# kubectl exec -it -n gitlab-ci gitlab-65c6b98f6b-q4dwq bash
- root@gitlab-7b54df755-6ljtp:/# vi /etc/gitlab/gitlab.rb
- external_url 'http://192.244.0.21:80' ## 再首行添加 这里也是Pod的ip
- root@gitlab-7b54df755-6ljtp:/# reboot
- root@gitlab-7b54df755-6ljtp:/# exit
复制代码 查察service
- [root@master gitlab-ci]# kubectl -n gitlab-ci get svc
- NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
- gitlab NodePort 192.102.225.126 <none> 80:30880/TCP 18m
复制代码 访问主机IPhttp://10.28.3.102:30880
用户:123456@qq.com 暗码:admin@123
点击 “Create a project”
点击“Create biank project” 创建项目demo-2048 可见等级选Public
填好后 点“Create project” 进入项目
将代码推送到项目中
- [root@master gitlab-ci]# cd /root/gitlab-ci/demo-2048
- [root@master demo-2048]# git config --global user.name "Administrator" ## 这里的用户密码
- [root@master demo-2048]# git config --global user.email "123456@qq.com" ## 是用于下载时候登录的
- [root@master demo-2048]# git remote remove origin ## 删除原有库
- [root@master demo-2048]# git remote add origin http://10.28.0.95:30880/root/demo-2048.git ## 添加库主节点IP
- [root@master demo-2048]# git add .
- [root@master demo-2048]# git commit -m "initial commit"
- [root@master demo-2048]# git push -u origin drone
- Username for 'http://10.28.0.198:30880': root
- Password for 'http://root@10.28.0.198:30880': admin@123 ## 这是deployment资源文件中设置的
复制代码 推送完刷新 项目库
2.2.7 部署GitLab Runner
将GitLab Runner部署到Kubernetes集群中,为GitLab Runner创建长期化构建缓存目次以加速构建速度,并将其注册到GitLab中。
将GitLab Runner部署到gitlab-ci命名空间下,Release名称为gitlab-runner,为GitLab Runner创建长期化构建缓存目次/home/gitlab-runner/ci-build-cache以加速构建速度,并将其注册到GitLab中。
登录GitLab管理界面(http://10.24.2.14:30880/admin),然后点击左侧菜单栏中的CI/CD下的Runners
记住复制的token:DN3ZZDAGSGB-kWSb-qBT
创建Service服务
- [root@master ~]# cd /root/gitlab-ci/
- [root@master gitlab-ci]# cat runner-sa.yaml
- apiVersion: v1
- kind: ServiceAccount
- metadata:
- name: gitlab-ci
- namespace: gitlab-ci
复制代码 创建脚色
- [root@master gitlab-ci]# cat runner-role.yaml
- kind: Role
- apiVersion: rbac.authorization.k8s.io/v1
- metadata:
- name: gitlab-ci
- namespace: gitlab-ci
- rules:
- - apiGroups: [""]
- resources: ["*"]
- verbs: ["*"]
复制代码 创建脚色对接
- [root@master gitlab-ci]# cat runner-rb.yaml
- kind: RoleBinding
- apiVersion: rbac.authorization.k8s.io/v1
- metadata:
- name: gitlab-ci
- namespace: gitlab-ci
- subjects:
- - kind: ServiceAccount
- name: gitlab-ci
- namespace: gitlab-ci
- roleRef:
- kind: Role
- name: gitlab-ci
- apiGroup: rbac.authorization.k8s.io
复制代码 创建资源对象
- [root@master gitlab-ci]# kubectl apply -f runner-sa.yaml
- [root@master gitlab-ci]# kubectl apply -f runner-role.yaml
- [root@master gitlab-ci]# kubectl apply -f runner-rb.yaml
复制代码- apiVersion: rbac.authorization.k8s.io/v1
- kind: ClusterRoleBinding
- metadata:
- name: default
- labels:
- k8s-app: gitlab-default
- roleRef:
- apiGroup: rbac.authorization.k8s.io
- kind: ClusterRole
- name: cluster-admin
- subjects:
- - kind: ServiceAccount
- name: default
- namespace: gitlab-ci
复制代码
免责声明:如果侵犯了您的权益,请联系站长,我们会及时删除侵权内容,谢谢合作!更多信息从访问主页:qidao123.com:ToB企服之家,中国第一个企服评测及商务社交产业平台。 |