Flatcar Self-Paced Learning Series: Advanced Provisioning Configuration
Building on the first session, we’ll dive into Butane configuration and deploy a more elaborate webpage.
A note for infrastructure developers
All configuration included in this course can also be generated programmatically. Ignition provides Go bindings for all Butane options used.
Goals
In this session, you’ll learn:
- Getting familiar with Butane configuration and performing quick local test iterations to get production ready. Find the full specification here .
- Extending your configuration to provision more complex services.
- Securing services by running as unprivileged users.
- Splitting your Butane configuration into multiple files for readability and maintainability.
Prerequisites
The session builds on the first session, Basic Operation and Local Testing . It assumes you
- have created a local test environment.
- are able to start ephemeral Flatcar VMs.
- know how to transpile Butane YAML to Ignition JSON.
- pass Ignition JSON configuration to a VM at launch.
- redirect host ports to the VM.
A slightly more useful basic web server
In the previous session, we set up an NGINX web server. However, that server was unable to serve any actual content (apart from the default index page). In our first example, we’ll set up a server that serves content we’ll provide.
We’ll add an inline web page to serve as our index page. And because choice is important, we’ll use Caddy instead of NGINX this time.
Check out this Butane config
|
|
Copy and paste it into a local file webserver.yaml, then run
|
|
Now start the VM with this configuration, and redirect host port 12345 to VM port 80:
|
|
After a few seconds you should be able to connect to http://localhost:12345 on your local machine and marvel at the web page you’ve deployed.
Since we activated access logging in Caddy, you should be able to see your web browser requests in the Caddy log:
|
|
Then reload the localhost page in your browser to generate log entries.
To exit the log view, press Ctrl+C.
When you’re done testing, shut down the VM
|
|
Different ways to provision files to Flatcar VMs
There are a number of ways to add files to your VM at provisioning time.
inlinewe’ve used in the above example: the file contents are added inside the provisioning configuration. Nice for small text based files.localwill include local files - i.e. files in the same directory as the Butane config file, or in a subdirectory. Suitable for small binary files.- there’s also
treeto include whole local directory trees. Use with care as this tends to impact the JSON file size.
- there’s also
sourceprovisions a remote file. This file will be downloaded at provisioning time; the file will not be included in the JSON config. Recommended for large files since the JSON config is passed as user data / custom data, which has size limits on most vendors / clouds.
Inline
We already use inline - it’s the simplest and most straightforward way to include content, by embedding it straight into the YAML file. Suitable for small text files like our HTML index page.
|
|
Local
Embeds a local file into the JSON configuration at transpile time. Suitable for small binary or text files, e.g. configuration files under source control. We’ll use this in our next example to embed an SVG logo into our deployment.
|
|
Source
Instructs Ignition to download a file at provisioning time, on early first boot. Useful for fetching larger files. We’ll use it to provision a video to our website.
|
|
Putting it all together
Let’s add a few more files!
First, download the Flatcar logo to the same local directory the Butane config resides in,so we can include it via local:
|
|
Now update your Butane config to
- Include the logo file in the deployment configuration.
- Reference a video file in the configuration. The file will be downloaded by the VM on first boot.
- Add both to the index web page HTML.
Check out this Butane config
|
|
Let’s transpile!
|
|
Whoops? What was that? Why doesn’t it work? Any idea?
What do you think went wrong? How do we resolve it? Extend this section only after trying to resolve it yourself.
Ah right! We added a reference to a file to our YAML, to be merged into the JSON configuration.
And we run Butane in a container - it gets our YAML via stdin and emits the JSON to stdout since we use docker’s -i option.
But since butane itself runs isolated from the host file system, it cannot access the host file logo.svg from inside the container - so it has no way of embedding the logo into the JSON config!
So we need to
- Tell docker to volume-mount the current directory (where the logo file resides) into the container, to a well-known place (we use
/files):-v "$(pwd):/files". - Tell Butane where it finds that directory inside the container so it can access all files referenced by
source: locallines:--files-dir /files
|
|
Did you fix it? Let’s fire it up!
|
|
Securing our service: Users and Groups
Our web server container runs as root, so any container escape will automatically gain superuser privileges.
That’s not healthy.
But we can use Linux user account isolation to mitigate this issue.
In the VM, verify that everything indeed runs as root:
|
|
Also check the container:
|
|
And lastly, check that the container can happily create new files in the bind-mounted volume:
|
|
When you’re done testing, shut down the VM:
|
|
Scary!
Let’s create an unprivileged user webby and run the web server with that account instead of root.
Note that by default, creating a user will also create a group of the same name, to be used as the user’s primary group.
We don’t need a home directory for that user so we’ll tell Butane to not create one.
Also, webby will need access to the docker socket to launch the caddy container, so we’ll add the user to the docker group.
The following snippet will create user webby.
|
|
Note that we explicitly set a uid for use with the docker command line later.
If we don’t set a uid ignition would pick one automatically, starting with 1000 for the first user created.
Now we can set the ownership of all files we ship, by adding to each file’s definition, respectively:
|
|
Lastly, we want systemd to start the docker container as that user, and docker to run services inside the container with the user’s uid, so we add
|
|
in the [Service] section of the web service unit definition and
|
|
to the docker command line in ExecStart= ....
Docker needs the uid instead of the user’s name because this applies inside the caddy container, and there is no user webby defined in that container.
The full Butane config:
|
|
Don’t forget to transpile!
|
|
Let’s run it:
|
|
In the VM, verify that the user exists, and check access rights to the files we ship:
|
|
We see that while individual files in the directory are owned by user webby, the /srv/www/html/ directory itself is still owned by root.
So the caddy will not be able to create new files on the host.
We can also check the user ID caddy has been started with inside the container:
|
|
And lastly, can we create files from the container?
Let’s try to create a file in /usr/share/caddy/ by writing to it from inside the running container. /usr/share/caddy/ in the container is volume-mounted from the host’s /srv/www/html:
|
|
If you see a “Permission denied” error, then everything worked - the web server is not supposed to write to host.
This works because the container is running as user 1234 which we created earlier, and that user doesn’t have the same privileges as root in the host’s /srv/www/html.
When you’re done, shut down the VM:
|
|
Splitting the configuration file
Over time, a single Butane config file can grow quite large. Looking at ours, we might be at that point already! Fortunately, Ignition offers an option to merge multiple configuration files.
For the purpose of this example, we’ll split up our config into:
main.yaml- the main configuration file that pulls everything togetherfiles.yaml- a files configuration that covers all content we want to provisionsystemd.yaml- systemd configuration that sets up our web server systemd unit.
We’ll keep the few lines of user generation in the main YAML file as it is quite small.
Consider our neat and concise new main:
|
|
With concerns separated into individual sub-configurations:
files.yaml
|
|
and
systemd.yaml
|
|
You probably noticed that the above snippets are for .yaml files while we are referencing .json files in the main Butane configuration.
That’s because the merge happens on the node, at first provisioning.
It’s done by Ignition, not by Butane.
So we have to transpile ALL the yaml.
The order is important - we need to transpile the snippets first, and the main YAML last - otherwise the JSON files referenced in main.yaml do not exist (or worse, are outdated!).
|
|
And then of course start Flatcar with main.json instead of our earlier webserver.json
|
|
Done!
In this session, you learned to:
- Customise your service and to provision arbitrary files from various sources.
- Secure your service by running as unprivileged user.
- Manage complex configuration by splitting it into separate files.