Spiderpool: a new solution to fixed application IPs for Calico

Automatically create fixed IP pools for applications

Spiderpool allows to automatically create IP pools from the subnet SpiderSubnet(10.244.0.0/16) and binds them to Pods. Additionally, it supports features such as fixed Pod IP addresses and automatic scaling of IP pools based on the replica count.

Create an Nginx Deployment with two replicas with the help of Spiderpool:

 [root@master1 ~]# cat <<EOF | kubectl create -f -
 apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: nginx
 spec:
   replicas: 2
   selector:
     matchLabels:
       app: nginx
   template:
     metadata:
       annotations:
         ipam.spidernet.io/subnet: '{"ipv4":["nginx-subnet-v4"]}'
         ipam.spidernet.io/ippool-ip-number: '+3'
       labels:
         app: nginx
     spec:
       containers:
       - name: nginx
         image: nginx
         imagePullPolicy: IfNotPresent
         ports:
         - name: http
           containerPort: 80
           protocol: TCP

ipam.spidernet.io/subnet: Spiderpool randomly selects some IPs from the subnet “nginx-subnet-v4” to create a fixed IP pool which is then bound to the application.

ipam.spidernet.io/ippool-ip-number: ‘+3’ indicates that the IP pool has three more IP addresses than the replica count, which ensures the availability of temporary IP addresses for the application during rolling releases.

Upon creating a Deployment, Spiderpool automatically creates an IP pool named auto-nginx-v4-eth0-452e737e5e12, and then binds it to the Deployment. The IP pool contains 5 IPs, with an IP range of 10.244.100.90-10.244.100.95:

[root@master1 ~]# kubectl get po -o wide
NAME                     READY   STATUS        RESTARTS   AGE     IP              NODE      NOMINATED NODE   READINESS GATES
nginx-644659db67-6lsmm   1/1     Running       0          12s     10.244.100.93    worker5   <none>           <none>
nginx-644659db67-n7ttd   1/1     Running       0          12s     10.244.100.91    master1   <none>           <none>

[root@master1 ~]# kubectl get sp
NAME                              VERSION   SUBNET          ALLOCATED-IP-COUNT   TOTAL-IP-COUNT   DEFAULT   DISABLE
auto-nginx-v4-eth0-452e737e5e12   4         10.244.0.0/16   2                    5                false     false

[root@master ~]# kubectl get sp auto-nginx-v4-eth0-452e737e5e12 -o jsonpath='{.spec.ips}' 
["10.244.100.90-10.244.100.95"]

Pod IPs are fixed within the auto IP pool: auto-nginx-v4-eth0-452e737e5e12 (10.244.100.90-10.244.100.95). If a pod is restarted, its IP will also be fixed within this range:

 [root@master1 ~]# kubectl get po -o wide
 NAME                     READY   STATUS        RESTARTS   AGE     IP              NODE      NOMINATED NODE   READINESS GATES
 nginx-644659db67-szgcg   1/1     Running       0          23s     10.244.100.90    worker5   <none>           <none>
 nginx-644659db67-98rcg   1/1     Running       0          23s     10.244.100.92    master1   <none>           <none>

While scaling, the IP addresses of the new replicas are also automatically assigned from the auto pool auto-nginx-v4-eth0-452e737e5e12(10.244.100.90-10.244.100.95), and the size of the IP pool dynamically increases as per the replica count:

[root@master1 ~]# kubectl scale deploy nginx --replicas 3  # scale pods
deployment.apps/nginx scaled

[root@master1 ~]# kubectl get po -o wide
NAME                     READY   STATUS        RESTARTS   AGE     IP              NODE      NOMINATED NODE   READINESS GATES
nginx-644659db67-szgcg   1/1     Running       0          1m     10.244.100.90    worker5   <none>           <none>
nginx-644659db67-98rcg   1/1     Running       0          1m     10.244.100.92    master1   <none>           <none>
nginx-644659db67-brqdg   1/1     Running       0          10s    10.244.100.94    master1   <none>           <none>

[root@master1 ~]# kubectl get sp
NAME                              VERSION   SUBNET          ALLOCATED-IP-COUNT   TOTAL-IP-COUNT   DEFAULT   DISABLE
auto-nginx-v4-eth0-452e737e5e12   4         10.244.0.0/16   3                    6                false     false

Manually specify an IP pool

In some cases, users hope to allocate IP addresses directly from a fixed range instead of assigning them in random by Spiderpool. The following demonstrates how to manually specify an IP pool for this purpose:

Create an IP pool:

cat << EOF | kubectl apply -f -
 apiVersion: spiderpool.spidernet.io/v2beta1
 kind: SpiderIPPool
 metadata:
   name: nginx-v4-ippool
 spec:
   ipVersion: 4
   subnet: 10.244.0.0/16
   ips:
   - 10.244.120.10-10.244.120.20

spec.subnet indicates which subnet the IP pool belongs to

spec.ips represents a fixed range of IP addresses, such as 10.244.120.10-10.244.120.20, with a total of 10 IPs

Manually specify an IP pool nginx-v4-ippool using the annotation ipam.spidernet.io/ippool when creating an application named nginx-m ：

[root@master1 ~]# cat <<EOF | kubectl create -f -
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx-m
  spec:
    replicas: 2
    selector:
      matchLabels:
        app: nginx
    template:
      metadata:
        annotations:
         ipam.spidernet.io/ippool: '{"ipv4":["nginx-v4-ippool"]}'
        labels:
          app: nginx
      spec:
        containers:
        - name: nginx
          image: nginx
          imagePullPolicy: IfNotPresent
          ports:
          - name: http
            containerPort: 80
            protocol: TCP

Spiderpool allocates two IP addresses from nginx-v4-ippooland then assigns them to the application. This ensures that the same pool is used for IP allocation regardless of whether the pod is restarted or scaled:

[root@master1 ~]# kubectl get po -o wide | grep nginx-m
 NAME                       READY   STATUS        RESTARTS   AGE     IP               NODE      NOMINATED NODE   READINESS GATES
 nginx-m-7c879df6bc-26dcq   1/1     Running       0          23s     10.244.120.12    worker5   <none>           <none>
 nginx-m-7c879df6bc-nwdtp   1/1     Running       0          23s     10.244.120.14    master1   <none>           <none>

[root@master1 ~]# kubectl get sp
 NAME                              VERSION   SUBNET          ALLOCATED-IP-COUNT   TOTAL-IP-COUNT   DEFAULT   DISABLE
 auto-nginx-v4-eth0-452e737e5e12   4         10.244.0.0/16   3                    6                false     false
 nginx-v4-ippool                   4         10.244.0.0/16   2                    11               false     false

Conclusion

The test result shows that external clients can access Nginx Pod directly via its IP and east-west communication is normal. Adopting Spiderpool in the Calico BGP mode makes it effortless to provide fixed IP addresses for Deployments and other use cases, offering a fresh alternative in this context.

For more information on Spiderpool’s usages, please refer to the documentation: https://github.com/spidernet-io/spiderpool/blob/main/docs/usage/

Reference:

https://spidernet-io.github.io/spiderpool
https://github.com/spidernet-io/spiderpool/blob/main/docs/usage/
https://docs.tigera.io/calico/latest/networking/