Experimental Feature: MachineSetPreflightChecks (beta)
The MachineSetPreflightChecks
feature can provide additional safety while creating new Machines and remediating existing unhealthy Machines of a MachineSet.
When a MachineSet creates machines under certain circumstances, the operation fails or leads to a new machine that will be deleted and recreated in a short timeframe, leading to unwanted Machine churn. Some of these circumstances include, but not limited to, creating a new Machine when Kubernetes version skew could be violated or joining a Machine when the Control Plane is upgrading leading to failure because of mixed kube-apiserver version or due to the cluster load balancer delays in adapting to the changes.
Enabling MachineSetPreflightChecks
provides safety in such circumstances by making sure that a Machine is only created when it is safe to do so.
Feature gate name: MachineSetPreflightChecks
Variable name to enable/disable the feature gate: EXP_MACHINE_SET_PREFLIGHT_CHECKS
Supported PreflightChecks
ControlPlaneIsStable
- This preflight check ensures that the ControlPlane is currently stable i.e. the ControlPlane is currently neither provisioning, upgrading.
- For Clusters with a managed topology it also checks if a control plane upgrade is pending.
- This preflight check is only performed if:
- The Cluster uses a ControlPlane provider.
- ControlPlane version is defined (
ControlPlane.spec.version
is set).
KubernetesVersionSkew
- This preflight check ensures that the MachineSet and the ControlPlane conform to the Kubernetes version skew.
- This preflight check is only performed if:
- The Cluster uses a ControlPlane provider.
- ControlPlane version is defined (
ControlPlane.spec.version
is set). - MachineSet version is defined (
MachineSet.spec.template.spec.version
is set).
KubeadmVersionSkew
- This preflight check ensures that the MachineSet and the ControlPlane conform to the kubeadm version skew.
- This preflight check is only performed if:
- The Cluster uses a ControlPlane provider.
- ControlPlane version is defined (
ControlPlane.spec.version
is set). - MachineSet version is defined (
MachineSet.spec.template.spec.version
is set). - MachineSet uses the
Kubeadm
Bootstrap provider.
ControlPlaneVersionSkew
- This preflight check ensures that the MachineSet and the ControlPlane have the same version. The idea behind this check is that it doesn’t make sense to create a Machine with an old version, if we already know based on the control plane version that the Machine has to be replaced soon.
- This preflight check is only performed if:
- The Cluster has a managed topology
- The Cluster uses a ControlPlane provider.
- ControlPlane version is defined (
ControlPlane.spec.version
is set). - MachineSet version is defined (
MachineSet.spec.template.spec.version
is set).
Configuring MachineSet PreflightChecks
Per default all preflight checks are enabled for all MachineSets including new and existing MachineSets.
The enabled preflight checks can be overwritten with the --machineset-preflight-checks
command-line flag.
It is also possible to opt-out of one or all of the preflight checks on a per MachineSet basis by specifying a
comma-separated list of the preflight checks via the machineset.cluster.x-k8s.io/skip-preflight-checks
annotation
on the MachineSet.
Examples:
- To opt out of all the preflight checks set the
machineset.cluster.x-k8s.io/skip-preflight-checks: All
annotation. - To opt out of the
ControlPlaneIsStable
preflight check set themachineset.cluster.x-k8s.io/skip-preflight-checks: ControlPlaneIsStable
annotation. - To opt out of multiple preflight checks set the
machineset.cluster.x-k8s.io/skip-preflight-checks: ControlPlaneIsStable,KubernetesVersionSkew
annotation.