Using Config Files¶
The config file feature is available as of k3d v4.0.0
Syntax & Semantics
The options defined in the config file are not 100% the same as the CLI flags.
This concerns naming and style/usage/structure, e.g.
--api-portis split up into a field named
kubeAPIthat has 3 different “child fields” (
- k3d options are bundled in a scope named
--no-rollbackis defined as
- repeatable flags (like
--port) are reflected as YAML lists
Using a config file is as easy as putting it in a well-known place in your file system and then referencing it via flag:
- All options in config file:
k3d cluster create --config /home/me/my-awesome-config.yaml(must be
- With CLI override (name):
k3d cluster create somename --config /home/me/my-awesome-config.yaml
- With CLI override (extra volume):
k3d cluster create --config /home/me/my-awesome-config.yaml --volume '/some/path:/some:[email protected]:0'
As of the time of writing this documentation, the config file only requires you to define two fields:
apiVersionto match the version of the config file that you want to use (at this time it would be
kindto define the kind of config file that you want to use (currently we only have the
So this would be the minimal config file, which configures absolutely nothing:
apiVersion: k3d.io/v1alpha3 kind: Simple
The configuration options for k3d are continuously evolving and so is the config file (syntax) itself.
Currently, the config file is still in an Alpha-State, meaning, that it is subject to change anytime (though we try to keep breaking changes low).
Validation via JSON-Schema
k3d uses a JSON-Schema to describe the expected format and fields of the configuration file.
This schema is also used to validate a user-given config file.
This JSON-Schema can be found in the specific config version sub-directory in the repository (e.g. here for
v1alpha3) and could be used to lookup supported fields or by linters to validate the config file, e.g. in your code editor.
All Options: Example¶
Since the config options and the config file are changing quite a bit, it’s hard to keep track of all the supported config file settings, so here’s an example showing all of them as of the time of writing:
# k3d configuration file, saved as e.g. /home/me/myk3dcluster.yaml apiVersion: k3d.io/v1alpha3 # this will change in the future as we make everything more stable kind: Simple # internally, we also have a Cluster config, which is not yet available externally name: mycluster # name that you want to give to your cluster (will still be prefixed with `k3d-`) servers: 1 # same as `--servers 1` agents: 2 # same as `--agents 2` kubeAPI: # same as `--api-port myhost.my.domain:6445` (where the name would resolve to 127.0.0.1) host: "myhost.my.domain" # important for the `server` setting in the kubeconfig hostIP: "127.0.0.1" # where the Kubernetes API will be listening on hostPort: "6445" # where the Kubernetes API listening port will be mapped to on your host system image: rancher/k3s:v1.20.4-k3s1 # same as `--image rancher/k3s:v1.20.4-k3s1` network: my-custom-net # same as `--network my-custom-net` subnet: "172.28.0.0/16" # same as `--subnet 172.28.0.0/16` token: superSecretToken # same as `--token superSecretToken` volumes: # repeatable flags are represented as YAML lists - volume: /my/host/path:/path/in/node # same as `--volume '/my/host/path:/path/in/[email protected]:0;agent:*'` nodeFilters: - server:0 - agent:* ports: - port: 8080:80 # same as `--port '8080:[email protected]'` nodeFilters: - loadbalancer env: - envVar: bar=baz # same as `--env '[email protected]:0'` nodeFilters: - server:0 registries: # define how registries should be created or used create: # creates a default registry to be used with the cluster; same as `--registry-create registry.localhost` name: registry.localhost host: "0.0.0.0" hostPort: "5000" use: - k3d-myotherregistry:5000 # some other k3d-managed registry; same as `--registry-use 'k3d-myotherregistry:5000'` config: | # define contents of the `registries.yaml` file (or reference a file); same as `--registry-config /path/to/config.yaml` mirrors: "my.company.registry": endpoint: - http://my.company.registry:5000 options: k3d: # k3d runtime settings wait: true # wait for cluster to be usable before returining; same as `--wait` (default: true) timeout: "60s" # wait timeout before aborting; same as `--timeout 60s` disableLoadbalancer: false # same as `--no-lb` disableImageVolume: false # same as `--no-image-volume` disableRollback: false # same as `--no-Rollback` loadbalancer: configOverrides: - settings.workerConnections=2048 k3s: # options passed on to K3s itself extraArgs: # additional arguments passed to the `k3s server|agent` command; same as `--k3s-arg` - arg: --tls-san=my.host.domain nodeFilters: - server:* nodeLabels: - label: foo=bar # same as `--k3s-node-label '[email protected]:1'` -> this results in a Kubernetes node label nodeFilters: - agent:1 kubeconfig: updateDefaultKubeconfig: true # add new cluster to your default Kubeconfig; same as `--kubeconfig-update-default` (default: true) switchCurrentContext: true # also set current-context to the new cluster's context; same as `--kubeconfig-switch-context` (default: true) runtime: # runtime (docker) specific options gpuRequest: all # same as `--gpus all` labels: - label: bar=baz # same as `--runtime-label '[email protected]:1'` -> this results in a runtime (docker) container label nodeFilters: - agent:1
Config File vs. CLI Flags¶
Config Precedence Order
Internal Setting > CLI Flag > Environment Variable > Config File > (k/v store >) Defaults
This means, that you can define e.g. a “base configuration file” with settings that you share across different clusters and override only the fields that differ between those clusters in your CLI flags/arguments.
For example, you use the same config file to create three clusters which only have different names and
- k3d demo repository: https://github.com/iwilltry42/k3d-demo/blob/main/README.md#config-file-support
- SUSE Blog: https://www.suse.com/c/introduction-k3d-run-k3s-docker-src/ (Search for
The “Configuration as Code” Way)