CMS is back! Use it in LibreSSL > 3.0.2 or with USE_LIBRESSL_CMS=1

Allow to specify the probed cloud stacks with -c cloud[,cloud...]

CHANGELOG.md

@@ -0,0 +1,69 @@
+# Changelog
+
+## v1.0 (2019-11-29)
+
+* Append `/etc/resolv.conf.tail` to `/etc/resolv.conf` if it exists.
+* Fixed usage.
+
+## v0.9 (2019-06-26)
+
+* Added support for `-c` to specify the probing order of different cloud stacks.
+* Added support for OpenNebula's START_SCRIPT methods (user-data alike).
+* Added support for OpenNebula's USERNAME method.
+* Added support for generating a default user password and writing it into
+  `~/.ssh/authorized_keys`.
+* Fixed handling of OpenNebula SSH_PUBLIC_KEY entries with multiple keys.
+* Improved documentation, added `CHANGELOG.md` file.
+
+## v0.8 (2019-06-02)
+
+* Added support for growing the root disk and its last partition (optional).
+* Fixed OpenStack support.
+* Fixed compilation with LibreSSL on OpenBSD 6.5 or newer.
+* Fixed probing order and OpenStack with `169.254.169.254` as the endpoint IP.
+* Improved OpenNebula support.
+
+## v0.7 (2018-08-15)
+
+* Added initial support for OpenNebula contextualization.
+* Added support for setting a custom login user or "root" with `-U`.
+* Added support for writing `resolv.conf` and static network configuration.
+* Fixed the generated pf rule that is loaded during cloud-agent operation.
+
+## v0.6 (2018-05-15)
+
+* Fixed compilation with (old) OpenSSL releases.
+
+---
+
+## v0.5 (2018-05-08)
+
+* Fixed the user-data script by loading it from /etc/rc.user-data.
+
+## v0.4 (2018-05-08)
+
+* Added support for user-data that is not base64-encoded.
+
+## v0.3 (2018-05-08)
+
+* Added support for user-data scripts.
+* Make the public key optional for stacks that supply a password (e.g. Azure).
+
+## v0.2.2 (2018-05-07)
+
+* Fixed issues in the v0.2.1 release.
+
+## v0.2.2 (2018-05-07)
+
+* Fixed issues in the v0.2 release.
+
+## v0.2 (2018-01-10)
+
+* Added support for OpenStack and its JSON-based meta data.
+* Added support for Apache CloudStack.
+* Try to get meta data from `dhcp-server-identifier` instead of
+  `169.254.169.254`.
+
+## v0.1 (2017-07-03)
+
+* Initial release with support for Microsoft Azure and Amazon AWS EC2.

LICENSE.md

@@ -6,10 +6,10 @@ License
 
 * `cloud-agent` is free software under OpenBSD's ISC-style license.
 * Most of the code has been written by Reyk Floeter <reyk@openbsd.org>
-* The http.[ch] files have been written by Kristaps Dzonsons <kristaps@bsd.lv>
+* The {http,json}.[ch] files were written by Kristaps Dzonsons <kristaps@bsd.lv>
 * Please refer to the individual source files for other copyright holders!
 
-> Copyright (c) 2017 Reyk Floeter <reyk@openbsd.org>
+> Copyright (c) 2017, 2018, 2019 Reyk Floeter <reyk@openbsd.org>
 > 
 > Permission to use, copy, modify, and distribute this software for any
 > purpose with or without fee is hereby granted, provided that the above
@@ -26,5 +26,5 @@ License
 `cms/`
 ------
 
-* The CMS code is from the OpenSSL and/or LibreSSL.
+* The CMS code is from OpenSSL and/or LibreSSL.
 * Please refer to the individual source files for other copyright holders!

Makefile

@@ -1,10 +1,13 @@
 #
 # The Azure agents needs CMS to obtain the SSH public keys.
 # LibreSSL has removed CMS, so either use OpenSSL to decrypt CMS
-# messages or compile the old CMS code for LibreSSL.
+# messages or compile the old CMS code for LibreSSL.  Or use
+# CMS that has returned to newer versions of LibreSSL.
 #
 .ifdef USE_OPENSSL
 MAKE_FLAGS+=	USE_OPENSSL=1
+.elifdef USE_LIBRESSL_CMS
+MAKE_FLAGS+=	USE_LIBRESSL_CMS=1
 .else
 SUBDIR=		cms
 .endif

README.md

@@ -23,26 +23,40 @@ has removed CMS which is required by Azure.
 Usage
 -----
 
-Installation is easy, `cloud-agent` detects the cloud type automatically.
+See the [cloud-agent(8)](cloud-agent.md) documentation for more
+information about the usage.
+
+Basic installation is easy, `cloud-agent` detects the cloud type
+automatically.
 
 * On Microsoft Azure, create a file `/etc/hostname.hvn0`
 
 * On Amazon AWS, create a file `/etc/hostname.xnf0`
 
-* On Exoscale, create a file `/etc/hostname.vio0`
+* On CloudStack, such as Exoscale, create a file `/etc/hostname.vio0`
 
 * On OpenBSD VMM (with meta-data), create a file `/etc/hostname.vio0`
 
 * On OpenStack/VMware, create a file `/etc/hostname.vmx0`
 
-* On OpenNebula, create a file `/etc/hostname.if`
+* The content of the file is identical for all of the above:
-  where _if_ is the name of your primary interface.
-
-* The content of the file is identical for all of them:
 
 		dhcp
 		!/usr/local/libexec/cloud-agent "\$if"
 
+* On OpenNebula, such as Data Center Light, create a file `/etc/hostname.if`
+  where _if_ is the name of your primary interface.
+  The `dhcp` line should be ommitted in the file:
+
+		!/usr/local/libexec/cloud-agent "\$if"
+
+Releases
+--------
+
+See the [Changelog](CHANGELOG.md) for a summary of changes and
+download the releases from the
+[release page](https://github.com/reyk/cloud-agent/releases).
+
 Author
 ------
 

agent/azure.c

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2017 Reyk Floeter <reyk@openbsd.org>
+ * Copyright (c) 2017, 2018, 2019 Reyk Floeter <reyk@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -23,8 +23,11 @@
 #include <stdlib.h>
 #include <unistd.h>
 #include <fcntl.h>
+#include <pwd.h>
 #include <err.h>
 
+#include <openssl/opensslv.h>
+
 #include "main.h"
 #include "http.h"
 #include "xml.h"
@@ -62,16 +65,22 @@ azure(struct system_config *sc)
 {
 	int	 ret = -1;
 
-	sc->sc_stack = "azure";
+	if (sc->sc_state == STATE_INIT) {
+		free(sc->sc_username);
+		if ((sc->sc_username = strdup("azure-user")) == NULL) {
+			log_warnx("failed to set default user");
+			return (-1);
+		}
 
-	/* Apply defaults */
+		/* Apply defaults */
-	free(sc->sc_username);
+		sc->sc_ovfenv = "/var/db/azure-ovf-env.xml";
-	if ((sc->sc_username = strdup("azure-user")) == NULL) {
+		sc->sc_priv = &az_config;
-		log_warnx("failed to set default user");
+		sc->sc_state = STATE_DHCP;
-		goto fail;
+		return (-1);
 	}
-	sc->sc_ovfenv = "/var/db/azure-ovf-env.xml";
+
-	sc->sc_priv = &az_config;
+	/* Don't try other endpoints */
+	sc->sc_state = STATE_DONE;
 
 	if (azure_getovfenv(sc) != 0) {
 		log_warnx("failed to get ovf-env.xml");
@@ -434,13 +443,16 @@ azure_certificates(struct system_config *sc)
 
 	fd = disable_output(sc, STDERR_FILENO);
 
-#ifdef USE_OPENSSL
+#if defined(USE_OPENSSL)
 	/*
 	 * XXX Now comes the part that needs CMS which is only
 	 * XXX present in OpenSSL but got removed from LibreSSL.
 	 */
 	log_debug("%s: running openssl cms", __func__);
 	if (shell("/usr/local/bin/eopenssl", "cms", /* )) */
+#elif defined(USE_LIBRESSL_CMS) || LIBRESSL_VERSION_NUMBER > 0x3000200fL
+	/* And CMS returned to LibreSSL! */
+	if (shell("/usr/bin/openssl", "cms", /* )) */
 #else
 	if (shell("/usr/local/bin/cms",
 #endif
@@ -719,7 +731,7 @@ azure_getovfenv(struct system_config *sc)
 	}
 
 	if ((xe = xml_findl(&xp->xe_head, "UserPassword", NULL)) != NULL) {
-		if ((sc->sc_password = calloc(1, 128)) == NULL) {
+		if ((sc->sc_password_hash = calloc(1, _PASSWORD_LEN)) == NULL) {
 			log_debug("%s: password failed", __func__);
 			goto done;
 		}
@@ -727,13 +739,14 @@ azure_getovfenv(struct system_config *sc)
 		str = strndup(xe->xe_data, xe->xe_datalen);
 		if (str == NULL ||
 		    crypt_newhash(str, "bcrypt,a",
-		    sc->sc_password, 128) != 0) {
+		    sc->sc_password_hash, _PASSWORD_LEN) != 0) {
 			log_debug("%s: password hashing failed", __func__);
-			free(sc->sc_password);
+			free(sc->sc_password_hash);
-			sc->sc_password = NULL;
+			sc->sc_password_hash = NULL;
 			free(str);
 			goto done;
 		}
+		explicit_bzero(str, xe->xe_datalen);
 		free(str);
 
 		/* Replace unencrypted password with hash */
@@ -743,11 +756,11 @@ azure_getovfenv(struct system_config *sc)
 		/* Update element for xml_print() below */
 		explicit_bzero(xe->xe_data, xe->xe_datalen);
 		free(xe->xe_data);
-		xe->xe_data = strdup(sc->sc_password);
+		xe->xe_data = strdup(sc->sc_password_hash);
-		xe->xe_datalen = strlen(sc->sc_password);
+		xe->xe_datalen = strlen(sc->sc_password_hash);
 	} else if ((xe = xml_findl(&xp->xe_head,
 	    "UserPasswordHash", NULL)) != NULL) {
-		if ((sc->sc_password =
+		if ((sc->sc_password_hash =
 		    get_word(xe->xe_data, xe->xe_datalen)) != NULL) {
 			log_debug("%s: password hash failed", __func__);
 			goto done;

agent/cloud-agent.8

@@ -1,6 +1,6 @@
 .\"	$OpenBSD: mdoc.template,v 1.15 2014/03/31 00:09:54 dlg Exp $
 .\"
-.\" Copyright (c) 2017 Reyk Floeter <reyk@openbsd.org>
+.\" Copyright (c) 2017, 2018, 2019 Reyk Floeter <reyk@openbsd.org>
 .\"
 .\" Permission to use, copy, modify, and distribute this software for any
 .\" purpose with or without fee is hereby granted, provided that the above
@@ -23,6 +23,8 @@
 .Sh SYNOPSIS
 .Nm cloud-agent
 .Op Fl nuv
+.Op Fl c Ar cloud Ns Op , Ns Ar cloud Ns ...
+.Op Fl p Ar length
 .Op Fl r Ar rootdisk
 .Op Fl t Ar timeout
 .Op Fl U Ar username
@@ -31,10 +33,42 @@
 The
 .Nm
 program manages the OpenBSD provisioning and VM interaction in cloud
-environments, including Microsoft Azure and Amazon AWS.
+environments.
 .Pp
 The options are as follows:
 .Bl -tag -width Ds
+.It Fl c Ar cloud Ns Op , Ns Ar cloud Ns ...
+Probe a list of cloud stacks for provisioning in the specified order.
+If this option is not specified,
+.Nm
+tries to detect the environment and possible cloud stacks automatically.
+Supported
+.Ar cloud
+stacks are:
+.Pp
+.Bl -tag -width opennebula -offset indent -compact
+.It Ic azure
+Microsoft Azure
+.It Ic cloudinit
+Generic cloud-init
+.It Ic ec2
+Amazon AWS EC2
+.It Ic opennebula
+OpenNebula
+.It Ic openstack
+OpenStack
+.El
+.It Fl p Ar length
+Generate and set a random password for the default user.
+The password will be written in its plain form into the
+.Pa ~/.ssh/authorized_keys
+file.
+This allows to use the
+.Xr doas 1
+command to gain root privileges.
+The minimum
+.Ar length
+is 8 characters and the default is an empty password.
 .It Fl n
 Do not configure the system and skip the provisioning step.
 .It Fl t Ar timeout
@@ -78,6 +112,8 @@ dhcp
 .Ed
 .Sh FILES
 .Bl -tag -width "/usr/local/libexec/cloud-agentX" -compact
+.It Pa ~/.ssh/authorized_keys
+The location of the agent-configured SSH public keys and optional password.
 .It Pa /usr/local/libexec/cloud-agent
 The agent itself.
 .It Pa /usr/local/bin/cms

agent/cloudinit.c

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2017 Reyk Floeter <reyk@openbsd.org>
+ * Copyright (c) 2017, 2018, 2019 Reyk Floeter <reyk@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -31,65 +31,29 @@
 
 static int	 cloudinit_fetch(struct system_config *);
 
-int
-tryendpoint(struct system_config *sc,
-    int (fetch)(struct system_config *),
-    int (next)(struct system_config *))
-{
-	int	 errfd = -1, ret;
-
-	free(sc->sc_endpoint);
-	sc->sc_endpoint = NULL;
-
-	switch (sc->sc_dhcpendpoint) {
-	case 0:
-		sc->sc_dhcpendpoint = 1;
-		if (dhcp_getendpoint(sc) == -1)
-			return tryendpoint(sc, fetch, next);
-		break;
-	case 1:
-		sc->sc_dhcpendpoint = 2;
-		if ((sc->sc_endpoint = strdup(DEFAULT_ENDPOINT)) == NULL) {
-			log_warnx("failed to set defaults");
-			return (-1);
-		}
-		break;
-	default:
-		if (next == NULL)
-			return (-1);
-		sc->sc_dhcpendpoint = 0;
-		return (*next)(sc);
-	}
-
-	errfd = disable_output(sc, STDERR_FILENO);
-	ret = (*fetch)(sc);
-	enable_output(sc, STDERR_FILENO, errfd);
-	if (ret != 0)
-		return tryendpoint(sc, fetch, next);
-
-	return (0);
-}
-
 int
 ec2(struct system_config *sc)
 {
-	free(sc->sc_username);
+	if (sc->sc_state == STATE_INIT) {
-	if ((sc->sc_username = strdup("ec2-user")) == NULL) {
+		free(sc->sc_username);
-		log_warnx("failed to set default user");
+		if ((sc->sc_username = strdup("ec2-user")) == NULL) {
+			log_warnx("failed to set default user");
+			return (-1);
+		}
+		sc->sc_state = STATE_169;
 		return (-1);
 	}
-
+	return cloudinit_fetch(sc);
-	sc->sc_stack = "ec2";
-	sc->sc_dhcpendpoint = 1;
-	return tryendpoint(sc, cloudinit_fetch, NULL);
 }
 
 int
 cloudinit(struct system_config *sc)
 {
-	sc->sc_stack = "cloudinit";
+	if (sc->sc_state == STATE_INIT) {
-	sc->sc_dhcpendpoint = 0;
+		sc->sc_state = STATE_DHCP;
-	return tryendpoint(sc, cloudinit_fetch, NULL);
+		return (-1);
+	}
+	return cloudinit_fetch(sc);
 }
 
 static int

agent/main.c

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2017, 2018 Reyk Floeter <reyk@openbsd.org>
+ * Copyright (c) 2017, 2018, 2019 Reyk Floeter <reyk@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -37,6 +37,7 @@
 #include <fcntl.h>
 #include <errno.h>
 #include <poll.h>
+#include <pwd.h>
 #include <err.h>
 
 #include "main.h"
@@ -49,11 +50,13 @@ static int	 agent_configure(struct system_config *);
 static int	 agent_network(struct system_config *);
 static void	 agent_free(struct system_config *);
 static int	 agent_pf(struct system_config *, int);
-static int	 agent_userdata(const unsigned char *, size_t);
+static int	 agent_userdata(struct system_config *,
+		    const unsigned char *, size_t);
 static void	 agent_unconfigure(void);
 static char	*metadata_parse(char *, size_t, enum strtype);
 
 static int	 agent_timeout;
+static char	*cloudnames[] = CLOUDNAMES;
 
 int
 shell(const char *arg, ...)
@@ -359,13 +362,19 @@ agent_free(struct system_config *sc)
 		log_debug("%s: unmounted %s", __func__, sc->sc_cdrom);
 	}
 
-	free(sc->sc_args);
+	if (sc->sc_password_plain != NULL) {
+		/* XXX can be removed with calloc_conceal() post-6.6 */
+		explicit_bzero(sc->sc_password_plain,
+		    strlen(sc->sc_password_plain));
+		free(sc->sc_password_plain);
+	}
+	free(sc->sc_password_hash);
 	free(sc->sc_hostname);
 	free(sc->sc_username);
-	free(sc->sc_password);
 	free(sc->sc_userdata);
 	free(sc->sc_endpoint);
 	free(sc->sc_instance);
+	free(sc->sc_args);
 	close(sc->sc_nullfd);
 
 	while ((ssh = TAILQ_FIRST(&sc->sc_pubkeys))) {
@@ -627,6 +636,7 @@ agent_pf(struct system_config *sc, int open)
 static int
 agent_configure(struct system_config *sc)
 {
+	char			 pwbuf[_PASSWORD_LEN + 2];
 	struct ssh_pubkey	*ssh;
 	char			*str1, *str2;
 
@@ -668,18 +678,30 @@ agent_configure(struct system_config *sc)
 	}
 
 	/* password */
-	if (sc->sc_password == NULL) {
+	if (sc->sc_password_hash == NULL) {
 		if (asprintf(&str2, "permit keepenv nopass %s as root\n"
 		    "permit keepenv nopass root\n", sc->sc_username) == -1)
 			str2 = NULL;
 	} else {
-		if (shell("usermod", "-p", sc->sc_password,
+		if (shell("usermod", "-p", sc->sc_password_hash,
 		    sc->sc_username, NULL) != 0)
 			log_warnx("password failed");
 
 		if (asprintf(&str2, "permit keepenv persist %s as root\n"
 		    "permit keepenv nopass root\n", sc->sc_username) == -1)
 			str2 = NULL;
+
+		/* write generated password as comment to authorized_keys */
+		if (sc->sc_password_plain != NULL) {
+			snprintf(pwbuf, sizeof(pwbuf), "# %s",
+			    sc->sc_password_plain);
+			if (fileout(pwbuf, "w",
+			    "%s/%s/.ssh/authorized_keys",
+			    strcmp("root", sc->sc_username) == 0 ? "" : "/home",
+			    sc->sc_username) != 0)
+				log_warnx("password comment failed");
+			explicit_bzero(pwbuf, sizeof(pwbuf));
+		}
 	}
 
 	/* doas */
@@ -689,7 +711,7 @@ agent_configure(struct system_config *sc)
 	free(str2);
 
 	/* ssh configuration */
-	if (sc->sc_password == NULL && !TAILQ_EMPTY(&sc->sc_pubkeys))
+	if (sc->sc_password_hash == NULL && !TAILQ_EMPTY(&sc->sc_pubkeys))
 		str1 = "/PasswordAuthentication/"
 		    "s/.*/PasswordAuthentication no/";
 	else
@@ -713,7 +735,7 @@ agent_configure(struct system_config *sc)
 	}
 
 	if (sc->sc_userdata) {
-		if (agent_userdata(sc->sc_userdata,
+		if (agent_userdata(sc, sc->sc_userdata,
 		    strlen(sc->sc_userdata)) != 0)
 			log_warnx("user-data failed");
 	}
@@ -737,7 +759,8 @@ agent_configure(struct system_config *sc)
 }
 
 static int
-agent_userdata(const unsigned char *userdata, size_t len)
+agent_userdata(struct system_config *sc,
+    const unsigned char *userdata, size_t len)
 {
 	char		*shebang = NULL, *str = NULL, *line = NULL;
 	const char	*file;
@@ -760,7 +783,7 @@ agent_userdata(const unsigned char *userdata, size_t len)
 		/* Decode user-data and call the function again */
 		if ((str = calloc(1, len + 1)) == NULL ||
 		    (len = b64_pton(userdata, str, len)) < 1 ||
-		    agent_userdata(str, len) != 0) {
+		    agent_userdata(sc, str, len) != 0) {
 			log_warnx("failed to decode user-data");
 			goto fail;
 		}
@@ -780,6 +803,9 @@ agent_userdata(const unsigned char *userdata, size_t len)
 		goto fail;
 	}
 
+	if (sc->sc_dryrun)
+		goto done;
+
 	/* write user-data script into file */
 	file = "/etc/rc.user-data";
 	if (fileout(str, "w", file) != 0) {
@@ -812,7 +838,7 @@ agent_network(struct system_config *sc)
 	const char	*family;
 	char		 domain[(NI_MAXHOST + 1) * 6 + 8]; /* up to 6 domains */
 	int		 has_domain = 0;
-	char		 ifidx[UINT16_MAX];
+	char		 ifidx[UINT16_MAX], *str;
 	const char	*comment = "# Generated by cloud-agent";
 
 	if (!sc->sc_network)
@@ -881,6 +907,12 @@ agent_network(struct system_config *sc)
 	if (has_domain)
 		fileout(domain, "a", "/etc/resolv.conf");
 
+	/* append resolv.conf.tail if it exists */
+	if ((str = filein("r", "/etc/resolv.conf.tail")) != NULL) {
+		fileout(str, "a", "/etc/resolv.conf");
+		free(str);
+	}
+
 	return (0);
 }
 
@@ -910,8 +942,10 @@ agent_unconfigure(void)
 
 	(void)fileout("permit keepenv persist :wheel as root\n"
 	    "permit keepenv nopass root\n", "w", "/etc/doas.conf");
-}
 
+	/* Remove cloud-instance file */
+	(void)unlink("/var/db/cloud-instance");
+}
 
 static char *
 metadata_parse(char *s, size_t sz, enum strtype type)
@@ -943,7 +977,8 @@ metadata(struct system_config *sc, const char *path, enum strtype type)
 	g = http_get(&sc->sc_addr, 1,
 	    sc->sc_endpoint, 80, path, NULL, 0, NULL);
 	if (g != NULL)
-		log_debug("%s: HTTP %d %s", __func__, g->code, path);
+		log_debug("%s: HTTP %d http://%s%s", __func__, g->code,
+		    sc->sc_endpoint, path);
 
 	if (g != NULL && g->code == 200 && g->bodypartsz > 0)
 		str = metadata_parse(g->bodypart, g->bodypartsz, type);
@@ -1081,7 +1116,8 @@ dhcp_getendpoint(struct system_config *sc)
 	sc->sc_addr.ip = sc->sc_endpoint;
 	sc->sc_addr.family = 4;
 
-	log_debug("%s: %s", __func__, ep);
+	if (log_getverbose() > 2)
+		log_debug("%s: %s", __func__, ep);
 
 	return (0);
 }
@@ -1091,8 +1127,9 @@ usage(void)
 {
 	extern char	*__progname;
 
-	fprintf(stderr, "usage: %s [-nuv] [-r rootdisk] [-t 3] [-U puffy] "
+	fprintf(stderr, "usage: %s [-nuv] "
-	    "interface\n", __progname);
+	    "[-c cloud[,...]] [-p length] [-r rootdisk]\n\t"
+	    "[-t 3] [-U puffy] interface\n", __progname);
 	exit(1);
 }
 
@@ -1126,23 +1163,163 @@ get_args(int argc, char *const *argv)
 	return (args);
 }
 
+static char *
+pwgen(size_t len, char **hash)
+{
+	char		*password;
+	size_t		 i, alphabet_len;
+	const char	*alphabet =
+	    "0123456789_"
+	    "abcdefghijklmnopqrstuvwxyz"
+	    "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+
+	alphabet_len = strlen(alphabet);
+	*hash = NULL;
+
+	/* XXX use calloc_conceal() post-6.6 */
+	if ((password = calloc(1, len + 1)) == NULL)
+		return (NULL);
+
+	/* Simple password generator */
+	for (i = 0; i < len; i++)
+		password[i] = alphabet[arc4random_uniform(alphabet_len)];
+
+	if ((*hash = calloc(1, _PASSWORD_LEN)) == NULL) {
+		freezero(password, len + 1);
+		return (NULL);
+	}
+
+	if (crypt_newhash(password, "bcrypt,a",
+	    *hash, _PASSWORD_LEN) != 0) {
+		freezero(password, len + 1);
+		freezero(*hash, _PASSWORD_LEN);
+		password = NULL;
+		*hash = NULL;
+	}
+
+	return (password);
+}
+
+static void
+cloud_add(enum cloudname name, struct clouds *clouds)
+{
+	struct cloud	*cloud;
+
+	TAILQ_FOREACH(cloud, clouds, cloud_entry) {
+		if (cloud->cloud_name == name)
+			fatalx("cloud %s defined twice",
+			    cloudnames[name]);
+	}
+
+	if ((cloud = calloc(1, sizeof(*cloud))) == NULL)
+		fatal("%s: calloc", __func__);
+
+	cloud->cloud_name = name;
+	switch (name) {
+	case AZURE:
+		cloud->fetch = azure;
+		break;
+	case CLOUDINIT:
+		cloud->fetch = cloudinit;
+		break;
+	case EC2:
+		cloud->fetch = ec2;
+		break;
+	case OPENNEBULA:
+		cloud->fetch = opennebula;
+		break;
+	case OPENSTACK:
+		cloud->fetch = openstack;
+		break;
+	}
+
+	TAILQ_INSERT_TAIL(clouds, cloud, cloud_entry);
+}
+
+static int
+trycloud(struct system_config *sc, struct cloud *cloud)
+{
+	int		 errfd = -1, ret = -1;
+
+	free(sc->sc_endpoint);
+	sc->sc_endpoint = NULL;
+
+	switch (sc->sc_state) {
+	case STATE_INIT:
+		if ((cloud = TAILQ_FIRST(sc->sc_clouds)) == NULL)
+			return (0);
+		sc->sc_stack = cloudnames[cloud->cloud_name];
+		log_debug("%s: %s", __func__, sc->sc_stack);
+		TAILQ_REMOVE(sc->sc_clouds, cloud, cloud_entry);
+		break;
+	case STATE_DHCP:
+		sc->sc_state = STATE_169;
+		if (dhcp_getendpoint(sc) == -1)
+			return trycloud(sc, cloud);
+		break;
+	case STATE_169:
+		sc->sc_state = STATE_DONE;
+		if ((sc->sc_endpoint = strdup(DEFAULT_ENDPOINT)) == NULL) {
+			log_warnx("failed to set defaults");
+			goto done;
+		}
+		break;
+	case STATE_DONE:
+		sc->sc_state = STATE_INIT;
+		ret = trycloud(sc, NULL);
+		goto done;
+	}
+
+	errfd = disable_output(sc, STDERR_FILENO);
+	ret = (*cloud->fetch)(sc);
+	enable_output(sc, STDERR_FILENO, errfd);
+	if (ret != 0)
+		return trycloud(sc, cloud);
+
+ done:
+	free(cloud);
+	return (0);
+}
+
 int
 main(int argc, char *const *argv)
 {
 	struct system_config	*sc;
-	int			 verbose = 0, dryrun = 0, unconfigure = 0;
+	int			 verbose = 0, dryrun = 0, unconfigure = 0, sub;
-	int			 ch, ret, timeout = CONNECT_TIMEOUT;
+	int			 genpw = 0, ch, ret, timeout = CONNECT_TIMEOUT;
 	const char		*error = NULL, *rootdisk = NULL;
-	char			*args, *username = NULL;
+	char			*args, *username = NULL, *options, *value;
+	struct clouds		 clouds;
+
+	/* log to stderr */
+	log_init(1, LOG_DAEMON);
 
 	if ((args = get_args(argc, argv)) == NULL)
 		fatalx("failed to save args");
 
-	while ((ch = getopt(argc, argv, "nr:t:U:uv")) != -1) {
+	TAILQ_INIT(&clouds);
+
+	while ((ch = getopt(argc, argv, "c:np:r:t:U:uv")) != -1) {
 		switch (ch) {
+		case 'c':
+			options = optarg;
+			while (*options) {
+				if ((sub = getsubopt(&options,
+				    cloudnames, &value)) == -1)
+					fatalx("invalid cloud stack");
+				else if (value != NULL)
+					fatalx("unexpected value");
+				cloud_add(sub, &clouds);
+			}
+			break;
 		case 'n':
 			dryrun = 1;
 			break;
+		case 'p':
+			genpw = strtonum(optarg, 8, 8192, &error);
+			if (error != NULL)
+				fatalx("invalid password length: %s", error);
+			break;
 		case 'r':
 			rootdisk = optarg;
 			break;
@@ -1169,8 +1346,6 @@ main(int argc, char *const *argv)
 	argv += optind;
 	argc -= optind;
 
-	/* log to stderr */
-	log_init(1, LOG_DAEMON);
 	log_setverbose(verbose);
 
 	if (unconfigure) {
@@ -1193,24 +1368,50 @@ main(int argc, char *const *argv)
 	if (rootdisk != NULL && growdisk(sc) == -1)
 		fatalx("failed to grow %s", rootdisk);
 
+	sc->sc_clouds = &clouds;
 	sc->sc_args = args;
 	if (username != NULL) {
 		free(sc->sc_username);
 		sc->sc_username = username;
 	}
 
-	/*
+	if (TAILQ_EMPTY(&clouds)) {
-	 * XXX Detect cloud with help from hostctl and sysctl
+		/*
-	 * XXX in addition to the interface name.
+		 * XXX Auto-detect cloud with help from hostctl and
-	 */
+		 * XXX sysctl in addition to the interface name.
-	if (opennebula(sc) == 0)
+		 */
-		ret = 0;
+		if (strcmp("hvn0", sc->sc_interface) == 0) {
-	else if (strcmp("hvn0", sc->sc_interface) == 0)
+			cloud_add(AZURE, &clouds);
-		ret = azure(sc);
+		} else if (strcmp("xnf0", sc->sc_interface) == 0) {
-	else if (strcmp("xnf0", sc->sc_interface) == 0)
+			cloud_add(OPENNEBULA, &clouds);
-		ret = ec2(sc);
+			cloud_add(EC2, &clouds);
-	else
+		} else {
-		ret = openstack(sc);
+			cloud_add(OPENNEBULA, &clouds);
+			cloud_add(OPENSTACK, &clouds);
+			cloud_add(CLOUDINIT, &clouds);
+		}
+	}
+	ret = trycloud(sc, NULL);
+
+	if (genpw) {
+		if (sc->sc_password_hash != NULL)
+			log_debug("%s: user password hash: %s", __func__,
+			    sc->sc_password_hash);
+		else if ((sc->sc_password_plain = pwgen(genpw,
+		    &sc->sc_password_hash)) != NULL) {
+			if (log_getverbose() > 2)
+				log_debug("%s: generated password: %s",
+				    __func__, sc->sc_password_plain);
+		} else
+			log_warnx("failed to generate password");
+	}
+
+	/* Debug userdata */
+	if (sc->sc_dryrun && sc->sc_userdata) {
+		if (agent_userdata(sc, sc->sc_userdata,
+		    strlen(sc->sc_userdata)) != 0)
+			log_warnx("user-data failed");
+	}
 
 	if (sc->sc_stack != NULL)
 		log_debug("%s: %s", __func__, sc->sc_stack);

agent/main.h

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2017 Reyk Floeter <reyk@openbsd.org>
+ * Copyright (c) 2017, 2018, 2019 Reyk Floeter <reyk@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -29,12 +29,30 @@
 #define DEFAULT_ENDPOINT	"169.254.169.254"
 #define CONNECT_TIMEOUT		10 /* in seconds */
 
+enum cloudname {
+	AZURE,
+	CLOUDINIT,
+	EC2,
+	OPENNEBULA,
+	OPENSTACK
+};
+#define CLOUDNAMES	{						\
+	"azure", "cloudinit", "ec2", "opennebula", "openstack", NULL	\
+}
+
 enum strtype {
 	WORD,
 	LINE,
 	TEXT
 };
 
+enum state {
+	STATE_INIT,
+	STATE_DHCP,
+	STATE_169,
+	STATE_DONE
+};
+
 struct ssh_pubkey {
 	char			*ssh_keyval;
 	char			*ssh_keyfp;
@@ -66,17 +84,27 @@ struct net_addr {
 };
 TAILQ_HEAD(net_addrs, net_addr);
 
+struct system_config;
+struct cloud {
+	enum cloudname		 cloud_name;
+	int			(*fetch)(struct system_config *);
+	TAILQ_ENTRY(cloud)	 cloud_entry;
+};
+TAILQ_HEAD(clouds, cloud);
+
 struct system_config {
 	const char		*sc_stack;
 	char			*sc_args;
 
 	char			*sc_hostname;
 	char			*sc_username;
-	char			*sc_password;
+	char			*sc_password_plain;
+	char			*sc_password_hash;
 	char			*sc_pubkey;
 	char			*sc_userdata;
 	char			*sc_endpoint;
-	int			 sc_dhcpendpoint;
+	enum state		 sc_state;
+	struct clouds		*sc_clouds;
 	char			*sc_instance;
 	int			 sc_timeout;
 
@@ -124,9 +152,6 @@ int	 azure(struct system_config *);
 /* cloudinit.c */
 int	 ec2(struct system_config *);
 int	 cloudinit(struct system_config *);
-int	 tryendpoint(struct system_config *,
-	    int (fetch)(struct system_config *),
-	    int (next)(struct system_config *));
 
 /* opennebula.c */
 int	 opennebula(struct system_config *);

agent/opennebula.c

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2018 Reyk Floeter <reyk@openbsd.org>
+ * Copyright (c) 2018, 2019 Reyk Floeter <reyk@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -34,25 +34,29 @@ opennebula(struct system_config *sc)
 	FILE		*fp;
 	const char	*delim = "\\\\\0", *errstr;
 	char		*line = NULL, *k, *v, *p, q;
-	char		*hname = NULL;
+	char		*value = NULL, *next = NULL, *last;
+	char		*hname = NULL, *uname = NULL;
 	size_t		 len, lineno = 0, i;
 	int		 ret = -1;
 	unsigned short	 unit;
 
+	if (sc->sc_state == STATE_INIT) {
+		sc->sc_state = STATE_169;
+		return (-1);
+	}
+
 	/* Return silently without error */
 	if ((fp = fopen("/mnt/context.sh", "r")) == NULL)
 		goto done;
 
-	sc->sc_stack = "opennebula";
-
 	while ((line = fparseln(fp, &len, &lineno,
 	    delim, FPARSELN_UNESCALL)) != NULL) {
 		/* key */
-		k = line;
+		k = line + strspn(line, " \t\r");
 
 		/* a context always starts with this header */
 		if (lineno == 1) {
-			ret = strcmp(line,
+			ret = strcmp(k,
 			    "# Context variables generated by OpenNebula");
 			if (ret != 0) {
 				log_debug("%s: unsupported context", __func__);
@@ -61,7 +65,12 @@ opennebula(struct system_config *sc)
 			free(line);
 			continue;
 		}
-		line[strcspn(line, "#")] = '\0';
+
+		/* Strip comments that do not occur within a value */
+		if (*k == '#') {
+			free(line);
+			continue;
+		}
 
 		/* value */
 		if ((v = strchr(line, '=')) == NULL || *(v + 1) == '\0') {
@@ -72,20 +81,57 @@ opennebula(struct system_config *sc)
 
 		/* value is quoted */
 		q = *v;
-		if (strspn(v, "\"'") == 0 || (p = strrchr(v, q)) == v) {
+		if (strspn(v, "\"'") == 0) {
 			free(line);
 			continue;
 		}
 		*v++ = '\0';
-		*p = '\0';
+
+		/* quoted value can be continued on multiple lines */
+		if ((value = strdup("")) == NULL) {
+			log_debug("%s: strdup", __func__);
+			goto done;
+		}
+		next = v;
+		do {
+			if ((p = strrchr(next, q)) != NULL)
+				*p++ = '\0';
+			if (*next) {
+				last = value;
+				if (asprintf(&value, "%s%s\n",
+				    last, next) == -1) {
+					log_debug("%s: asprintf", __func__);
+					if (next != v)
+						free(next);
+					goto done;
+				}
+				free(last);
+			}
+			if (next != v)
+				free(next);
+		} while (p == NULL &&
+		    (next = fparseln(fp, &len, &lineno,
+		    delim, FPARSELN_UNESCALL)) != NULL);
+		next = NULL;
+		v = value;
+
+		/* strip trailing newline */
+		if ((p = strrchr(v, '\n')) != NULL)
+			*p = '\0';
 
 		/* continue if value is empty */
 		if (*v == '\0') {
 			free(line);
+			free(value);
+			value = NULL;
 			continue;
 		}
 
-		log_debug("%s: %s = %s", __func__, k, v);
+		/* print key/value unless it is a multi-line value */
+		if (strcasecmp("SSH_PUBLIC_KEY", k) != 0 &&
+		    strcasecmp("START_SCRIPT", k) != 0 &&
+		    strcasecmp("START_SCRIPT_BASE64", k) != 0)
+			log_debug("%s: %s = %s", __func__, k, v);
 
 		if (strcasecmp("NETWORK", k) == 0) {
 			if (strcasecmp("YES", v) == 0)
@@ -178,11 +224,35 @@ opennebula(struct system_config *sc)
 			if ((hname = strdup(v)) == NULL)
 				log_warnx("failed to set hostname");
 		} else if (strcasecmp("SSH_PUBLIC_KEY", k) == 0) {
-			if (agent_addpubkey(sc, v, NULL) != 0)
+			do {
-				log_warnx("failed to set ssh pubkey");
+				p = v + strcspn(v, "\n");
+				*p++ = '\0';
+				if (*v)
+					log_debug("%s: %s = %s",
+					    __func__, k, v);
+				if (*v && agent_addpubkey(sc, v, NULL) != 0)
+					log_warnx("failed to set ssh pubkey");
+				v = p + strspn(p, "\n");
+			} while (*v != '\0');
+		} else if (strcasecmp("START_SCRIPT", k) == 0 ||
+		    strcasecmp("START_SCRIPT_BASE64", k) == 0) {
+			log_debug("%s: %s = ...", __func__, k);
+
+			/* We will detect and decode base64 later */
+			if ((sc->sc_userdata = strdup(v)) == NULL)
+				log_warnx("failed to set userdata");
+		} else if (strcasecmp("USERNAME", k) == 0) {
+			if ((uname = strdup(v)) == NULL)
+				log_warnx("failed to set username");
+			else {
+				free(sc->sc_username);
+				sc->sc_username = uname;
+			}
 		}
 
 		free(line);
+		free(value);
+		value = NULL;
 	}
 
 	fclose(fp);
@@ -216,5 +286,6 @@ opennebula(struct system_config *sc)
 	if (fp != NULL)
 		fclose(fp);
 	free(line);
+	free(value);
 	return (ret);
 }

agent/openstack.c

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2018 Reyk Floeter <reyk@openbsd.org>
+ * Copyright (c) 2018, 2019 Reyk Floeter <reyk@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -31,11 +31,14 @@
 
 static int	 openstack_fetch(struct system_config *);
 
-
 int
 openstack(struct system_config *sc)
 {
-	return tryendpoint(sc, openstack_fetch, cloudinit);
+	if (sc->sc_state == STATE_INIT) {
+		sc->sc_state = STATE_DHCP;
+		return (-1);
+	}
+	return openstack_fetch(sc);
 }
 
 static int
@@ -53,7 +56,6 @@ openstack_fetch(struct system_config *sc)
 	if ((json = metadata(sc,
 	    "/openstack/latest/meta_data.json", TEXT)) == NULL)
 		goto fail;
-	sc->sc_stack = "openstack";
 
 	if ((j = json_parse(json, strlen(json))) == NULL)
 		goto fail;

cloud-agent.md

@@ -0,0 +1,150 @@
+CLOUD-AGENT(8) - System Manager's Manual
+
+# NAME
+
+**cloud-agent** - cloud provisioning for OpenBSD VMs
+
+# SYNOPSIS
+
+**cloud-agent**
+\[**-nuv**]
+\[**-c** *cloud*\[,*cloud*...]]
+\[**-p** *length*]
+\[**-r** *rootdisk*]
+\[**-t** *timeout*]
+\[**-U** *username*]
+*interface*
+
+# DESCRIPTION
+
+The
+**cloud-agent**
+program manages the OpenBSD provisioning and VM interaction in cloud
+environments.
+
+The options are as follows:
+
+**-c** *cloud*\[,*cloud*...]
+
+> Probe a list of cloud stacks for provisioning in the specified order.
+> If this option is not specified,
+> **cloud-agent**
+> tries to detect the environment and possible cloud stacks automatically.
+> Supported
+> *cloud*
+> stacks are:
+
+> **azure**
+
+> > Microsoft Azure
+
+> **cloudinit**
+
+> > Generic cloud-init
+
+> **ec2**
+
+> > Amazon AWS EC2
+
+> **opennebula**
+
+> > OpenNebula
+
+> **openstack**
+
+> > OpenStack
+
+**-p** *length*
+
+> Generate and set a random password for the default user.
+> The password will be written in its plain form into the
+> *~/.ssh/authorized\_keys*
+> file.
+> This allows to use the
+> doas(1)
+> command to gain root privileges.
+> The minimum
+> *length*
+> is 8 characters and the default is an empty password.
+
+**-n**
+
+> Do not configure the system and skip the provisioning step.
+
+**-t** *timeout*
+
+> Change the HTTP timeout.
+> The default is 3 seconds.
+
+**-U** *username*
+
+> Change the default user.
+> The default is
+> "ec2-user"
+> on AWS,
+> "azure-user"
+> on Azure, and
+> "puffy"
+> everywhere else.
+> The default user is used when it is not obtained from the cloud
+> configuration.
+> Using
+> "root"
+> is supported, but not recommended.
+
+**-r** *rootdisk*
+
+> Automatically grow the last
+> OpenBSD
+> FFS partition of the root disk to use all the available space.
+
+**-u**
+
+> Deprovision and unconfigure the system.
+> This deletes keys, passwords, and logs files without asking for permission.
+
+**-v**
+
+> Produce more verbose output.
+
+Enable
+**cloud-agent**
+in the
+hostname.if(5)
+of the VM's primary networking interface and automatically the last
+partition of the root disk:
+
+	# cat /etc/hostname.hvn0
+	dhcp
+	!/usr/local/libexec/cloud-agent -r sd0 "\$if"
+
+# FILES
+
+*~/.ssh/authorized\_keys*
+
+> The location of the agent-configured SSH public keys and optional password.
+
+*/usr/local/libexec/cloud-agent*
+
+> The agent itself.
+
+*/usr/local/bin/cms*
+
+> The CMS binary that is used to decrypt messages from the Azure fabric.
+
+*/var/db/cloud-instance*
+
+> The instance ID as reported by the cloud.
+> **cloud-agent**
+> reprovisions the system when the value has changed.
+
+# SEE ALSO
+
+meta-data(8),
+vmd(8)
+
+# AUTHORS
+
+Reyk Floeter <[reyk@openbsd.org](mailto:reyk@openbsd.org)>
+
+OpenBSD 6.5 - June 26, 2019