Cloud configuration in linux

Ubuntu Documentation

Summary

cloud-init is the Ubuntu package that handles early initialization of a cloud instance. It is installed in the official Ubuntu live server images since the release of 18.04, Ubuntu Cloud Images and also in the official Ubuntu images available on EC2.

Some of the things it configures are:

• setting a default locale
• setting hostname
• generate ssh private keys
• adding ssh keys to user’s .ssh/authorized_keys so they can log in
• setting up ephemeral mount points

cloud-init’s behavior can be configured via user-data. User-data is passed to cloud-init by the user to the cloud provider at launch time, usually as a parameter in the CLI, template, or cloud portal used to launch an instance.

User Data Input Formats

User data that will be acted upon by cloud-init must be in one of the following types:

Gzip Compressed Content

content found to be gzip compressed will be uncompressed. The uncompressed data will then be used as if it were not compressed. Compression of data is useful because user-data is limited to 16384 bytes 1

Mime Multi Part archive

• This list of rules is applied to each part of this multi-part file. Using a mime-multi part file, the user can specify more than one type of data. For example, both a user data script and a cloud-config type could be specified.

User-Data Script

begins with: » #!» or » Content-Type: text/x-shellscript»
script will be executed at «rc.local-like» level during first boot. rc.local-like means «very late in the boot sequence»

Include File

begins with » #include» or » Content-Type: text/x-include-url»
This content is a «include» file. The file contains a list of urls, one per line. Each of the URLs will be read, and their content will be passed through this same set of rules. Ie, the content read from the URL can be gzipped, mime-multi-part, or plain text

Cloud Config Data

begins with » #cloud-config» or » Content-Type: text/cloud-config»
This content is «cloud-config» data. See the examples for a commented example of supported config formats.

Upstart Job

begins with » #upstart-job» or » Content-Type: text/upstart-job»
Content is placed into a file in /etc/init, and will be consumed by upstart as any other upstart job.

Cloud Boothook

begins with » #cloud-boothook» or » Content-Type: text/cloud-boothook»

This content is «boothook» data. It is stored in a file under /var/lib/cloud and then executed immediately.
This is the earliest «hook» available. Note, that there is no mechanism provided for running only once. The boothook must take care of this itself. It is provided with the instance id in the environment variable «INSTANCE_ID». This could be made use of to provide a ‘once-per-instance’
Only available in 10.10 or later (cloud-init 0.5.12 and later)

Part Handler

begins with » #part-handler» or » Content-Type: text/part-handler»

This is a ‘part-handler’. It will be written to a file in /var/lib/cloud/data based on its filename. This must be python code that contains a list_types method and a handle_type method. Once the section is read the ‘list_types’ method will be called. It must return a list of mime-types that this part-handler handlers.
The ‘handle_type’ method must be like:

Cloud-init will then call the ‘handle_type’ method once at begin, once per part received, and once at end. The ‘ begin ‘ and ‘ end ‘ calls are to allow the part handler to do initialization or teardown.
There is an example at doc/examples/part-handler.txt. Also this blog post offers another example

User-Data Scripts

As popularized by alestic.com, user-data scripts are a convenient way to do something on first boot of a launched instance. This input format is accepted to cloud-init and handled as you would expect. The script will be invoked at an «rc.local» like point in the boot sequence.

After running the above, you can expect that /root/output.txt will contain the desired text.

Cloud Config Syntax

Cloud Config is the simplest way to accomplish some things via user-data. Using cloud-config syntax, the user can specify certain things in a human friendly format. These things include:

• apt upgrade should be run on first boot
• a different apt mirror should be used
• additional apt sources should be added
• certain ssh keys should be imported

The file must be valid yaml syntax.

Here are some simple examples of what can be done with cloud-config syntax:

run ‘apt-get upgrade’ on first boot

enable byobu by default for all system users

import ssh keys for launchpad user ‘smoser’ and add his ppa

run a few commands on first boot
The output of these commands will appear in console output

Multipart Input

A single format of user data might not be enough to accomplish what you want. For example, you may want to insert an upstart job and also run a user-data script.

There is a tool in cloud-utils’s bin/ directory called ‘write-mime-multipart’ which can aid creating mime multipart content.

Consider the following example:

Now, given the files above in the current directory, you can do:

Now, when your instance is booted, you will have

smoser’s keys imported into the ubuntu users

/.ssh/authorized_keys . Notice that I did not specify ‘—key’ on the command line as it wasn’t needed. Instead, my launchpad keys were imported.

a root owned file in /var/lib/cloud/data/user-data.txt that contains the gzip compressed user data.

a root owned file in /var/lib/cloud/data/user-data.txt.i that contains the post-processed user data (uncompressed, with includes resolved).

a file ‘/var/lib/cloud/data/user-data.txt.i’

Also, notice

because ‘my-cloudconfig.txt’ begins with ‘#cloud-config’ we did not need to specify the mime-type.

600799 makes boothook get read as a user script

More information

Examples of user-data for cloud-init can be seen in the Github examples directory.

Using cloud-init with example clouds:

Footnotes

The following was output of ‘ec2-run-instances —user-data-file=ud.txt’ where ud.txt was sufficiently large on 2010-06-28: ‘Client.InvalidParameterValue: User data is limited to 16384 bytes’ (1)

CloudInit (последним исправлял пользователь danielbowers 2021-07-25 16:34:51)

The material on this wiki is available under a free license, see Copyright / License for details
You can contribute to this wiki, see Wiki Guide for details

Источник

How to customize cloud-init autoinstall (user-data) Ubuntu 20.04

Table of Contents

In this article I will cover different scenarios which requires modification of cloud-init autoinstall configuration file such as installing GUI (GNOME), changing partition layout, installing additional packages, configuring network etc. I will share multiple examples of user-data configuration file to cover these scenarios.

Normally when we try to bring up a Ubuntu server using cloud-init then we don’t get too many prompts (this is by design) as by default cloud-init considers default value unless you mark those parameters are interactive in the configuration file.

So I will try to share some sample cloud-init autoinstall configuration file (user-data) which you can use based or atleast you get a fair idea of what kind of modification is required for your requirement.

1. Generating encrypted password for user-data config file

In the cloud-init autoinstall configuration file (user-data), the password is expected to be encrypted format. For example:

Here we have to replace after encrypting the password which you intend to use for your user. You can use the following command to generate an encrypted password with SHA512 algorithm. Replace PASSWORD with your password:

Or alternatively you can also use following command, here replace ‘ ubuntu ‘ with your password:

Or alternatively (the recommended method) you can use mkpasswd command:

Now you can copy this password and place it in your user-data file. It would be better if you place it inside quotes to avoid conflicts with special characters present in the encrypted password.

2. Install Ubuntu with GUI (GNOME Desktop)

By default cloud-init will perform the autoinstall with basic Ubuntu packages i.e. only CLI but most of us are used to GNOME desktop for ease of usage. So if you want to deploy Ubuntu with GUI desktop then add packages section as shown below:

This will install the ubuntu desktop for your server.

3. Execute commands during installation

You can add custom commands to be executed at different stage of the installation. These are defined using following parameters:

• early-commands: A list of shell commands to invoke as soon as the installer starts, in particular before probing for block and network devices. The autoinstall config is available at /autoinstall.yaml (irrespective of how it was provided) and the file will be re-read after the early-commands have run to allow them to alter the config if necessary.
• late-commands: Shell commands to run after the install has completed successfully and any updates and packages installed, just before the system reboots. They are run in the installer environment with the installed system mounted at /target
• error-commands: Shell commands to run after the install has failed. They are run in the installer environment, and the target system (or as much of it as the installer managed to configure) will be mounted at /target . Logs will be available at /var/log/installer in the live session.

Here is a sample user-data file covering all 3 types of commands:

1. early-commands: We are checking the connectivity towards our host server
2. late-commands: We have created a script to update the sshd_config file. Now since at this stage the filesystem is mounted on /target so we have added our config file path prefixed with /target
3. error-commands: In case of failure we will automatically transfer our logs to host server using netcat command.

To be able to use the nc command, you already should have nc command running on 10.43.138.8 server listening on port 5050 to which our client will transfer the files: For example in my case:

4. Use proxy in cloud-init autoinstall configuration file

If your client is running behind some proxy then you may have to specify the proxy sever details to be able to download and install packages. You can define proxy using the following supported directives:

The syntax to be used is:

5. Customize SSH Configuration

We have a couple of options to choose from for SSHD usecase such as:

• install-server: Whether to install OpenSSH server in the target system.
• authorized-keys: A list of SSH public keys to install in the initial user’s account.
• allow-pw: true if authorized_keys is empty, false otherwise

Let’s generate a private public key pair using ssh-keygen

Next we add the content of test.pub in authorized-keys section of cloud-init file. Here is a sample user-data config file:

Now, we trigger the installation and post installation of our target node we can verify if we are able to SSH to the default system user using our private key:

So, we were able to successfully connect to our target server which we recently brought up using the above cloud-init autoinstall configuration file.

6. Customize network configuration

We also have a couple of options to customize our network configuration. The default is to interpret the config for the install media, which runs DHCPv4 on any interface with a name matching “ eth* ” or “ en* ” but then disables any interface that does not receive an address.

6.1 Configure static IPv4 network

To configure a static IPv4 network you can use below template. Here the interface name can be a problem in such case so unless you are sure of the network name mapping or you can just disable consistent network device naming by using biosdevname=0 net.ifnames=0 as the kernel menu while performing PXE. As with this option, the kernel will use the old interface naming convention i.e. ethX

6.2 Configure static IPv6 network

To configure a static IPv6 network you can use the following template (Here we have enabled DHCP for IPv4 network) . Here also the same rule applies as discussed in previous section. You may have to disable consistent network device naming to be able to assign static address.

6.3 Create bond

We can also create a bond using cloud-init autoinstall configuration file. Here is a sample template where I am creating a bond using eno49 and eno50 interface with a static IPv4 address.

You can get the list of supported bonding modes, I have used active-backup for my environment.

7. Create EFI Boot Partition

If you are trying to bring up Ubuntu using autoinstall configuration file on UEFI BIOS Environment then you will also need an EFI boot partition or else you will get below error during installation:

I stumbled upon the same problem when I was trying to install Ubuntu using kickstart and luckily this article on stackoverflow has the fix. But those steps will not work with cloud-init autoinstall. Here you can consider the following sample template to create an EFI boot partition:

You can check the complete sample autoinstall configuration file at Prepare cloud-init autoinstall file

8. Perform a complete interactive installation

With cloud-init autoinstall, you can explicitly specify directives which you wish to configure manually rather than automated. In such case, you can specify those directives under interactive-sections as shown below:

So, here we have specified that we wish to configure storage and network manually so don’t take the default values. Once we trigger an installation using this, the autoinstall will pause for user input to configure network and storage.

But if you wish to perform complete interactive installation (considering no defaults) then you can use * with interactive as shown below:

Summary

In this tutorial I covered different options with cloud-init autoinstall (user-data) which you can customize based on your requirement. I will try to update this article as and when I find more interesting which we can customize in user-data configuration file. If you have something which you feel will be a great add-on to this article, drop me a message via the comment section.

Further Reading

Didn’t find what you were looking for? Perform a quick search across GoLinuxCloud

If my articles on GoLinuxCloud has helped you, kindly consider buying me a coffee as a token of appreciation.

For any other feedbacks or questions you can either use the comments section or contact me form.

Thank You for your support!!

to stay connected and get the latest updates

Источник