how to set a specific ip and MAC address for a pod with calico

28 9 月, 2022 1061点热度 0人点赞 0条评论

前言

客户有一个需求，应用需要固定 ip 和固定 mac地址，查了一下，固定 ip 直接直接通过添加注解实现，测试顺利通过。

"cni.projectcalico.org/ipAddrs": "<span class="error">[\"192.168.0.1\"]</span>"

1	"cni.projectcalico.org/ipAddrs": "<span class="error">[\"192.168.0.1\"]</span>"

固定mac

固定mac 这边试了很久，一直失败，官网的介绍只有一句话

https://projectcalico.docs.tigera.io/networking/pod-mac-address#how-to

  "cni.projectcalico.org/hwAddr": "1c:0c:0a:c0:ff:ee"

1	"cni.projectcalico.org/hwAddr": "1c:0c:0a:c0:ff:ee"

折腾许久，原来和 calico 版本有关，手上2套集群 calico 是3.16.10，另一套是 3.19.1，修复版本3.24 ，8月中旬修复的，不满足这个issus 提到的版本。
https://github.com/projectcalico/calico/pull/6249/files

解决方法1，绕行

需要通过prestart方式，获取网络权限后修改 mac 地址，解决容器网卡 mac 地址固定的问题，通过添加cap_net权限并使用poststart配置网卡，固定 mac 测试通过。

spec:
  replicas: 1
  selector:
    matchLabels:
      app: my2048-test
  template:
    metadata:
      name: my2048-test
      creationTimestamp: null
      labels:
        app: my2048-test
      annotations:
        cni.projectcalico.org/ipAddrs: '["10.252.35.42"]'
    spec:
      containers:
        - name: dao-2048
          image: '10.23.45.67/myapp/2048:latest'
          ports:
            - containerPort: 80
              protocol: TCP
          resources:
            limits:
              cpu: 50m
              memory: 50M
          lifecycle:
            postStart:
              exec:
                command:
                  - /bin/sh
                  - '-c'
                  - 'ifconfig eth0 hw ether 76:15:a2:d8:40:5e'
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: Always
          securityContext:
            capabilities:
              add:
                - NET_ADMIN
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      securityContext: {}
      schedulerName: default-scheduler

spec:

replicas: 1

selector:

matchLabels:

app: my2048-test

template:

metadata:

name: my2048-test

creationTimestamp: null

labels:

app: my2048-test

annotations:

cni.projectcalico.org/ipAddrs: '["10.252.35.42"]'

spec:

containers:

- name: dao-2048

image: '10.23.45.67/myapp/2048:latest'

ports:

- containerPort: 80

protocol: TCP

resources:

limits:

cpu: 50m

memory: 50M

lifecycle:

postStart:

exec:

command:

- /bin/sh

- '-c'

- 'ifconfig eth0 hw ether 76:15:a2:d8:40:5e'

terminationMessagePath: /dev/termination-log

terminationMessagePolicy: File

imagePullPolicy: Always

securityContext:

capabilities:

add:

- NET_ADMIN

restartPolicy: Always

terminationGracePeriodSeconds: 30

dnsPolicy: ClusterFirst

securityContext: {}

schedulerName: default-scheduler

重点部分截图如下

解决方法2，升级calico

尝试升级 calico 3.24.1

官方文档 https://projectcalico.docs.tigera.io/maintenance/kubernetes-upgrade

参考了这2篇

Upgrading an installation that uses the operator

https://projectcalico.docs.tigera.io/maintenance/operator-migration

Download the v3.24 operator manifest.

<span class="code-snippet__command-prefix">$ </span>curl https://raw.githubusercontent.com/projectcalico/calico/v3.24.1/manifests/tigera-operator.yaml <span class="nt">-O</span>

				1
2

						<span class="code-snippet__command-prefix">$ </span>curl https://raw.githubusercontent.com/projectcalico/calico/v3.24.1/manifests/tigera-operator.yaml <span class="nt">-O</span>

Use the following command to initiate an upgrade.

<span class="code-snippet__command-prefix">$ </span>kubectl apply <span class="nt">-f</span> tigera-operator.yaml

				1

						<span class="code-snippet__command-prefix">$ </span>kubectl apply <span class="nt">-f</span> tigera-operator.yaml

3. Monitor the migration status with the following command:

kubectl describe tigerastatus calico

				1
2

						kubectl describe tigerastatus calico

4. Now that the migration is complete, you will see Calico resources have moved to the calico-system namespace.

kubectl get pods -n calico-system

				1
2

						kubectl get pods -n calico-system

You should see output like this:

NAME                                       READY   STATUS    RESTARTS   AGE
calico-kube-controllers-7688765788-9rqht   1/1     Running   0          17m
calico-node-4ljs6                          1/1     Running   0          14m
calico-node-bd8mc                          1/1     Running   0          14m
calico-node-cpbd8                          1/1     Running   0          14m
calico-node-jl97q                          1/1     Running   0          14m
calico-node-xw2nj                          1/1     Running   0          14m
calico-typha-57bf79f96f-6sk8x              1/1     Running   0          14m
calico-typha-57bf79f96f-g99s9              1/1     Running   0          14m
calico-typha-57bf79f96f-qtchs              1/1     Running   0          14m

				
					
				1
2
3
4
5
6
7
8
9
10
11

						NAME                                       READY   STATUS    RESTARTS   AGE
calico-kube-controllers-7688765788-9rqht   1/1     Running   0          17m
calico-node-4ljs6                          1/1     Running   0          14m
calico-node-bd8mc                          1/1     Running   0          14m
calico-node-cpbd8                          1/1     Running   0          14m
calico-node-jl97q                          1/1     Running   0          14m
calico-node-xw2nj                          1/1     Running   0          14m
calico-typha-57bf79f96f-6sk8x              1/1     Running   0          14m
calico-typha-57bf79f96f-g99s9              1/1     Running   0          14m
calico-typha-57bf79f96f-qtchs              1/1     Running   0          14m
 

					

			

At this point, the operator will have automatically cleaned up any Calico resources in the kube-system namespace. No manual cleanup is required.

升级后 calico 在 calico-system 命名空间下的运行情况及 calico 版本截图

# calicoctl version
Client Version: v3.24.1
Git commit: 83493da01
Cluster Version: v3.24.1
Cluster Type: k8s,bgp,kubeadm,kdd,typha,operator

测试新版 calico 固定mac功能

折腾了一番升级calico，赶紧测试一下 2行注解搞定固定ip，固定mac吧。

注解添加

    annotations:  
      "cni.projectcalico.org/ipAddrs": '["100.113.137.134"]'
      "cni.projectcalico.org/hwAddr": "aa:af:43:e7:18:3e"

annotations:

"cni.projectcalico.org/ipAddrs": '["100.113.137.134"]'

"cni.projectcalico.org/hwAddr": "aa:af:43:e7:18:3e"

容器多次重启，测试有效，升级 calico 版本来固定mac的办法有效，收工。