Skip to main content
Version: v0.4

PodDecoration

PodDecoration works in conjunction with CollaSet to selectively inject specific configurations to Pods that meet certain criteria.

PodDecoration not only allows injecting sidecar containers to Pods but also enables modifying existing container configurations, metadata, and scheduling parameters etc. The PodDecoration controller does not control the upgrade of Pods. The actual upgrade process is fully controlled by the CollaSet controller. This means that the injection upgrade of PodDecoration can also be performed InPlaceIfPossible.

About CollaSet.

Example

1. Create CollaSet​

# collaset.yaml
apiVersion: apps.kusionstack.io/v1alpha1
kind: CollaSet
metadata:
name: foo
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: foo
template:
metadata:
labels:
app: foo
spec:
containers:
- image: nginx:1.25.2
name: foo

Use collaset.yaml to create three pods under CollaSet foo management.

$ kubectl apply -f collaset.yaml
collaset.apps.kusionstack.io/foo created

$ kubectl get cls
NAME DESIRED CURRENT AVAILABLE UPDATED UPDATED_READY UPDATED_AVAILABLE CURRENT_REVISION UPDATED_REVISION AGE
foo 3 3 3 3 3 3 foo-7bdb974bc7 foo-7bdb974bc7 7s

$ kubectl get pod
NAME READY STATUS RESTARTS AGE
foo-2wnnf 1/1 Running 0 41s
foo-hqpx7 1/1 Running 0 41s
foo-mqt48 1/1 Running 0 41s

2. Create PodDecoration​

The following poddecoration.yaml file describes a PodDecoration, which selects the pod under CollaSet foo and injects the content in template into the pod with instance-id=0.

# poddecoration.yaml
apiVersion: apps.kusionstack.io/v1alpha1
kind: PodDecoration
metadata:
name: poddecoration
spec:
selector: # selected pod range in which PodDecoration takes effect
matchLabels:
app: foo
updateStrategy:
rollingUpdate:
selector: # select pod to upgrade in effect range
matchLabels:
collaset.kusionstack.io/instance-id: "0"
template:
metadata:
- patchPolicy: Overwrite
labels:
custom.io/sidecar-version: "v1"
containers:
- injectPolicy: AfterPrimaryContainer
name: sidecar-a
image: ubuntu:22.04
command: ["sleep", "2h"]
volumeMounts:
- name: sample-volume
mountPath: /vol/sample
volumes:
- name: sample-volume
emptyDir: {}

Create PodDecoration sample-pd to upgrade selected pod

$ kubectl apply -f poddecoration.yaml
poddecoration.apps.kusionstack.io/sample-pd created

The status of PodDecoration is updated, and one pod is injected with sidecar through recreate.

$ kubectl get pd
NAME EFFECTIVE MATCHED INJECTED UPDATED UPDATED_READY CURRENT_REVISION UPDATED_REVISION AGE
sample-pd true 3 1 1 1 sample-pd-9465f4c84 20s

$ kubectl get pod
NAME READY STATUS RESTARTS AGE
foo-2gnnl 2/2 Running 0 15s
foo-2wnnf 1/1 Running 0 2m
foo-hqpx7 1/1 Running 0 2m

$ kubectl get pd sample-pd -o yaml | grep -A20 status
status:
details:
- affectedReplicas: 3
collaSet: foo
pods:
- name: foo-2gnnl
revision: sample-pd-9465f4c84
- name: foo-2wnnf
escaped: true
- name: foo-hqpx7
escaped: true
matchedPods: 3
injectedPods: 1
updatedPods: 1
updatedReadyPods: 1
updatedAvailablePods: 1
isEffective: true
updatedRevision: sample-pd-9465f4c84

3. Update PodDecoration​

3.1. Rolling update v1​

Edit sample-pd to expand the upgrade scope.

$ kubectl edit pd sample-pd
# poddecoration.yaml
# Edit updateStrategy to select instance-id in [0, 1, 2]
...
spec:
...
updateStrategy:
rollingUpdate:
selector:
matchExpressions:
- key: collaset.kusionstack.io/instance-id
operator: In
values:
- "0"
- "1" # add
- "2" # add
template:
...

All pods updated.

$ kubectl get pd
NAME EFFECTIVE MATCHED INJECTED UPDATED UPDATED_READY CURRENT_REVISION UPDATED_REVISION AGE
sample-pd true 3 3 3 3 sample-pd-9465f4c84 sample-pd-9465f4c84 3m

$ kubectl get pod
NAME READY STATUS RESTARTS AGE
foo-2gnnl 2/2 Running 0 3m
foo-lftw8 2/2 Running 0 8s
foo-n57rr 2/2 Running 0 8s

$ kubectl get pd sample-pd -o yaml | grep -A20 status
status:
currentRevision: sample-pd-9465f4c84
details:
- affectedReplicas: 3
collaSet: foo
pods:
- name: foo-2gnnl
revision: sample-pd-9465f4c84
- name: foo-lftw8
revision: sample-pd-9465f4c84
- name: foo-n57rr
revision: sample-pd-9465f4c84
matchedPods: 3
injectedPods: 3
updatedPods: 3
updatedReadyPods: 3
updatedAvailablePods: 3
isEffective: true
currentRevision: sample-pd-9465f4c84
updatedRevision: sample-pd-9465f4c84

3.2. Rolling update v1 -> v2​

Update sample-pd's sidecar container image and updateStrategy.

$ kubectl edit pd sample-pd
# poddecoration.yaml
# Update sidecar-a's image with ubuntu:22.10
# Edit updateStrategy to select instance-id in [0]
apiVersion: apps.kusionstack.io/v1alpha1
kind: PodDecoration
metadata:
name: poddecoration
spec:
...
updateStrategy:
rollingUpdate:
selector:
- key: collaset.kusionstack.io/instance-id
operator: In
values:
- "0"
template:
...
containers:
- injectPolicy: AfterPrimaryContainer
name: sidecar-a
image: ubuntu:22.10
...

Pod foo-2gnnl in-place upgrade sidecar container image.

$ kubectl get pd
NAME EFFECTIVE MATCHED INJECTED UPDATED UPDATED_READY CURRENT_REVISION UPDATED_REVISION AGE
sample-pd true 3 3 1 1 sample-pd-9465f4c84 sample-pd-8697d4bf8c 6min

$ kubectl get pod
NAME READY STATUS RESTARTS AGE
foo-2gnnl 2/2 Running 1 (12s ago) 6m
foo-lftw8 2/2 Running 0 3min
foo-n57rr 2/2 Running 0 3min

$ kubectl get pod foo-2gnnl -o yaml | grep "image: ubuntu"
image: ubuntu:22.10

$ kubectl get pd sample-pd -o yaml | grep -A20 status
status:
details:
- affectedReplicas: 3
collaSet: foo
pods:
- name: foo-2gnnl
revision: sample-pd-8697d4bf8c
- name: foo-lftw8
revision: sample-pd-9465f4c84
- name: foo-n57rr
revision: sample-pd-9465f4c84
matchedPods: 3
injectedPods: 3
updatedPods: 1
updatedReadyPods: 1
updatedAvailablePods: 1
isEffective: true
currentRevision: sample-pd-9465f4c84
updatedRevision: sample-pd-8697d4bf8c

Features

Injection​

Metadata​

apiVersion: apps.kusionstack.io/v1alpha1
kind: PodDecoration
metadata:
name: poddecoration
spec:
template:
metadata:
- patchPolicy: MergePatchJson
annotations:
cafe.sofastack.io/decoration-version: '[{"name":"sample-pd","version":"v2"}]'
- patchPolicy: Overwrite
labels:
custom.io/sidecar-version: "v2"
annotations:
cafe.sofastack.io/decoration-name: sample-pd

patchPolicy is the injected policy, as follows:

  • Retain: The original value of annotations and labels will be retained.
  • Overwrite: The value of annotations and labels corresponding to the existing key will be overwritten.
  • MergePatchJson: It only takes effect for annotation. If the key does not exist, the value will be written directly. Otherwise, the json value will be merged.

For example:

# Old pod metadata
metadata:
labels:
custom.io/sidecar-version: "v1"
annotations:
cafe.sofastack.io/decoration-version: '[{"name":"old-pd","version":"v1"}]'

# After metadata injected
metadata:
labels:
custom.io/sidecar-version: "v2"
annotations:
cafe.sofastack.io/decoration-type: sample-pd
cafe.sofastack.io/decoration-version: '[{"name":"old-pd","version":"v1"}, {"name":"sample-pd","version":"v2"}]'

Primary Container​

apiVersion: apps.kusionstack.io/v1alpha1
kind: PodDecoration
metadata:
name: poddecoration
spec:
# ...
template:
primaryContainers:
- targetPolicy: ByName
name: foo
image: foo:v2
env:
- name: APP_NAME
value: foo
volumeMounts:
- name: sample-volume
mountPath: /vol/sample
volumes:
- name: sample-volume
emptyDir: {}

Injection into the primary containers only supports limited fields: image, env and volumeMounts.

targetPolicy indicates which existed container these configuration should inject into, as follows:

  • ByName: Only inject containers matching name.
  • All: Inject all primary containers.
  • First: Inject into first primary container.
  • Last: Inject into last primary container.

Sidecar Container​

apiVersion: apps.kusionstack.io/v1alpha1
kind: PodDecoration
metadata:
name: poddecoration
spec:
# ...
template:
containers:
- injectPolicy: AfterPrimaryContainer # Container injected policy, AfterPrimaryContainer or BeforePrimaryContainer
name: sidecar-a
image: ubuntu:22.04
...

Inject a new sidecar container. Optional, it can be placed in front or behind the primary container.

InitContainer​

apiVersion: apps.kusionstack.io/v1alpha1
kind: PodDecoration
metadata:
name: poddecoration
spec:
# ...
template:
initContainers:
- name: init
image: custom-init-image:v1
...

Upgrade strategy​

selector​

You can use selector to select the pod. The CollaSet provides a unique instance-id for each pod. Of course, custom labels can also be used to label pods for triggering upgrades.

apiVersion: apps.kusionstack.io/v1alpha1
kind: PodDecoration
metadata:
name: poddecoration
spec:
# ...
updateStrategy:
rollingUpdate:
selector:
- key: collaset.kusionstack.io/instance-id
operator: In
values:
- "0"

partition​

Partition is the desired number or percent of Pods in old revisions, defaults to 0.

apiVersion: apps.kusionstack.io/v1alpha1
kind: PodDecoration
metadata:
name: poddecoration
spec:
# ...
updateStrategy:
rollingUpdate:
partition: 2 # int number