tabulon/cdist
1
0
Fork 0
forked from ungleich-public/cdist
Code Pull requests Activity

Merge remote-tracking branch 'cdist/install' into install

Browse source
This commit is contained in:
Steven Armstrong 2011-09-16 12:43:43 +02:00
commit a4ec6f93ce
110 changed files with 1921 additions and 1672 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
.gitignore vendored
View file

@ -13,3 +13,6 @@ doc/man/man*/docbook-xsl.css
# Ignore cache for version control
cache/
# Python
bin/__pycache__/

40
HACKERS_README
View file

@ -1,40 +0,0 @@
cat << eof
Hey hackers,
this README is for you, for those who want to dig into cdist, hack it or try
to get a deeper understanding. Please read doc/man/man7/cdist-hacker.text.
I hope you have a lot of fun with cdist, because it was also a lot of fun to
develop it!
-- Nico, 20110324
## Running cdist when developing
This file is suitable for execution and saving the objects and
explorers from cdist. I usually do it like this:
% ./HACKERS_README
################################################################################
eof
set -x
# Tell the user what we do, so this script makes sense during execution
# prepare use (only from top level directory)
export PATH="$(pwd -P)/bin:$PATH"
export __cdist_conf_dir="$(pwd -P)/conf"
# Allow user to supply hostname
target="${1:-localhost}"
# And use hostname as basedir (dangerous, but hackers know what they do)
export __cdist_local_base_dir="/tmp/$target"
# Run the real script
cdist-deploy-to "$target"
# Display results
find "${__cdist_local_base_dir}"

19
README
View file

@ -38,13 +38,13 @@ Design | Define target state, do not focus on methods or scripts
Design | Push architecture: Instantly apply your changes
Small core | cdist's core is very small - less code, less bugs
Fast development | Focus on straightforwardness of type creation is a main development objective
Modern Programming Language | cdist is written in Python
Requirements, Scalability | No central server needed, cdist operates in push mode and can be run from any computer
Requirements, Scalability, Upgrade | cdist only needs to be updated on the master, not on the target hosts
Requirements, Security | Uses well-know [SSH](http://www.openssh.com/) as transport protocol
Requirements, Simplicity | Requires only shell and SSH server on the target
UNIX | Reuse of existing tools like cat, find, mv, ...
UNIX, familar environment, documentation | Is available as manpages and HTML
UNIX, simplicity, familar environment | cdist is written in POSIX shell
UNIX, simplicity, familar environment | cdist is configured in POSIX shell
"""]]
@ -76,6 +76,7 @@ cdist was tested or is know to run on at least
### Server
* A posix like shell
* Python (>= 3.2 required)
* SSH-Client
### Client ("target host")
@ -109,11 +110,12 @@ how to use cdist.
There are at least the following branches available:
* master: the development branch
* 1.7: Bugfixes, cleanups, new type and explorer rename
* Development: master
* 2.0: Python rewrite of cdist core
Old versions:
* 1.7: Bugfixes, cleanups, new type and explorer rename
* 1.6: New types, cleaned up \_\_package* types, internal cleanup
* 1.5: Focus on object orientation instead of global stage orientation
* 1.4: Support for redefiniton of objects (if equal)
@ -129,13 +131,12 @@ may vanish at any point. To select a specific branch use
git checkout -b <name> origin/<name>
# Stay on a specific version
version=1.7
version=2.0
git checkout -b $version origin/$version
### Mirrors
* git://github.com/telmich/cdist.git ([github](https://github.com/telmich/cdist))
* git://git.sans.ethz.ch/cdist ([sans](http://git.sans.ethz.ch/?p=cdist;a=summary))
## Update
@ -151,6 +152,14 @@ If you stay on a version branche (i.e. 1.0, 1.1., ...), nothing should break.
The master branch on the other hand is the development branch and may not be
working, break your setup or eat the tree in your garden.
### Upgrading from 1.7 to 2.0
* Ensure python (>= 3.2) is installed on the server
* Use "cdist config host" instead of "cdist-deploy-to host"
* Use "cdist config -p host1 host2" instead of "cdist-mass-deploy"
* Use "cdist banner" for fun
* Use **\_\_object_fq** instead of **\_\_self** in manifests
### Upgrading from 1.6 to 1.7
* If you used the global explorer **hardware_type**, you need to change

684
bin/cdist Executable file
View file

@ -0,0 +1,684 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# 2010-2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
import argparse
import datetime
import logging
import multiprocessing
import os
import subprocess
import shutil
import stat
import sys
import tempfile
BANNER = """
.. . .x+=:. s
dF @88> z` ^% :8
'88bu. %8P . <k .88
. '*88888bu . .@8Ned8" :888ooo
.udR88N ^"*8888N .@88u .@^%8888" -*8888888
<888'888k beWE "888L ''888E` x88: `)8b. 8888
9888 'Y" 888E 888E 888E 8888N=*8888 8888
9888 888E 888E 888E %8" R88 8888
9888 888E 888F 888E @8Wou 9% .8888Lu=
?8888u../ .888N..888 888& .888888P` ^%888*
"8888P' `"888*"" R888" ` ^"F 'Y"
"P' "" ""
"""
# Given paths from installation
REMOTE_BASE_DIR = "/var/lib/cdist"
REMOTE_CONF_DIR = os.path.join(REMOTE_BASE_DIR, "conf")
REMOTE_OBJECT_DIR = os.path.join(REMOTE_BASE_DIR, "object")
REMOTE_TYPE_DIR = os.path.join(REMOTE_CONF_DIR, "type")
REMOTE_GLOBAL_EXPLORER_DIR = os.path.join(REMOTE_CONF_DIR, "explorer")
CODE_HEADER = "#!/bin/sh -e\n"
DOT_CDIST = ".cdist"
VERSION = "2.0.0"
logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s')
log = logging.getLogger()
class TypeEmulator:
def __init__(self, name):
self.name = name
self.type = os.path.basename(name)
def type_emulator(self):
type = basename(sys.argv[0])
type_is_singleton(type)
# Check object id
# Prevent double slash if id begins with /
# Record parameter: opt_file="${opt#--}"
# [ $# -ge 1 ] || __cdist_usage "Missing value for $opt"
# echo "${value}" > "${__cdist_parameter_dir}/${opt_file}"
# Record requirements
# echo $requirement >> "$(__cdist_object_require "$__cdist_object_self")"
# Ensure required parameters are given
# Ensure that only optional or required parameters are given
# [ "$is_valid" ] || __cdist_usage "Unknown parameter $parameter"
# Merge object (creating twice with the same parameter + requirements == allowed)
# diff -ru "${__cdist_new_object_dir}/${__cdist_name_parameter}
# # Add "I was here message"
# _cdist_object_source_add "${__cdist_object_dir}"
class Cdist:
"""Cdist main class to hold arbitrary data"""
def __init__(self, target_host,
initial_manifest=False, remote_user="root",
home=None, debug=False):
self.target_host = target_host
self.remote_prefix = ["ssh", "root@" + self.target_host]
# Setup directory paths
self.temp_dir = tempfile.mkdtemp()
self.debug = debug
if home:
self.base_dir = home
else:
self.base_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir))
self.conf_dir = os.path.join(self.base_dir, "conf")
self.cache_base_dir = os.path.join(self.base_dir, "cache")
self.cache_dir = os.path.join(self.cache_base_dir, self.target_host)
self.global_explorer_dir = os.path.join(self.conf_dir, "explorer")
self.lib_dir = os.path.join(self.base_dir, "lib")
self.manifest_dir = os.path.join(self.conf_dir, "manifest")
self.type_base_dir = os.path.join(self.conf_dir, "type")
self.out_dir = os.path.join(self.temp_dir, "out")
os.mkdir(self.out_dir)
self.global_explorer_out_dir = os.path.join(self.out_dir, "explorer")
os.mkdir(self.global_explorer_out_dir)
self.object_base_dir = os.path.join(self.out_dir, "object")
# Setup binary directory + contents
self.bin_dir = os.path.join(self.out_dir, "bin")
os.mkdir(self.bin_dir)
self.link_type_to_emulator()
# List of type explorers transferred
self.type_explorers_transferred = {}
# objects
self.objects_prepared = []
self.remote_user = remote_user
# Mostly static, but can be overwritten on user demand
if initial_manifest:
self.initial_manifest = initial_manifest
else:
self.initial_manifest = os.path.join(self.manifest_dir, "init")
def cleanup(self):
# Do not use in __del__:
# http://docs.python.org/reference/datamodel.html#customization
# "other globals referenced by the __del__() method may already have been deleted
# or in the process of being torn down (e.g. the import machinery shutting down)"
#
log.debug("Saving" + self.temp_dir + "to " + self.cache_dir)
# Remove previous cache
if os.path.exists(self.cache_dir):
shutil.rmtree(self.cache_dir)
shutil.move(self.temp_dir, self.cache_dir)
def exit_error(self, *args):
log.error(*args)
sys.exit(1)
def remote_mkdir(self, directory):
"""Create directory on remote side"""
self.run_or_fail(["mkdir", "-p", directory], remote=True)
def remote_cat(filename):
"""Use cat on the remote side for output"""
self.run_or_fail(["cat", filename], remote=True)
def shell_run_or_debug_fail(self, script, *args, **kargs):
# Manually execute /bin/sh, because sh -e does what we want
# and sh -c -e does not exit if /bin/false called
args[0][:0] = [ "/bin/sh", "-e" ]
remote = False
if "remote" in kargs:
if kargs["remote"]:
args[0][:0] = self.remote_prefix
remote = true
del kargs["remote"]
log.debug("Shell exec cmd: %s", args)
log.debug("Shell exec env: %s", kargs['env'])
try:
subprocess.check_call(*args, **kargs)
except subprocess.CalledProcessError:
log.error("Code that raised the error:\n")
if remote:
remote_cat(script)
else:
script_fd = open(script)
print(script_fd.read())
script_fd.close()
self.exit_error("Command failed (shell): " + " ".join(*args))
def run_or_fail(self, *args, **kargs):
if "remote" in kargs:
if kargs["remote"]:
args[0][:0] = self.remote_prefix
del kargs["remote"]
log.debug("Exec: " + " ".join(*args))
try:
subprocess.check_call(*args, **kargs)
except subprocess.CalledProcessError:
self.exit_error("Command failed: " + " ".join(*args))
def remove_remote_dir(self, destination):
self.run_or_fail(["rm", "-rf", destination], remote=True)
def transfer_dir(self, source, destination):
"""Transfer directory and previously delete the remote destination"""
self.remove_remote_dir(destination)
self.run_or_fail(["scp", "-qr", source,
self.remote_user + "@" +
self.target_host + ":" +
destination])
def transfer_file(self, source, destination):
"""Transfer file"""
self.run_or_fail(["scp", "-q", source,
self.remote_user + "@" +
self.target_host + ":" +
destination])
def global_explorer_output_path(self, explorer):
"""Returns path of the output for a global explorer"""
return os.path.join(self.global_explorer_out_dir, explorer)
def type_explorer_output_dir(self, cdist_object):
"""Returns and creates dir of the output for a type explorer"""
dir = os.path.join(self.object_dir(cdist_object), "explorer")
if not os.path.isdir(dir):
os.mkdir(dir)
return dir
def remote_global_explorer_path(self, explorer):
"""Returns path to the remote explorer"""
return os.path.join(REMOTE_GLOBAL_EXPLORER_DIR, explorer)
def list_global_explorers(self):
"""Return list of available explorers"""
return os.listdir(self.global_explorer_dir)
def list_type_explorers(self, type):
"""Return list of available explorers for a specific type"""
dir = self.type_explorer_dir(type)
if os.path.isdir(dir):
list = os.listdir(dir)
else:
list = []
log.debug("Explorers for %s in %s: %s", type, dir, list)
return list
def list_types(self):
return os.listdir(self.type_base_dir)
def list_object_paths(self, starting_point):
"""Return list of paths of existing objects"""
object_paths = []
for content in os.listdir(starting_point):
full_path = os.path.join(starting_point, content)
if os.path.isdir(full_path):
object_paths.extend(self.list_object_paths(starting_point = full_path))
# Directory contains .cdist -> is an object
if content == DOT_CDIST:
object_paths.append(starting_point)
return object_paths
def get_type_from_object(self, cdist_object):
"""Returns the first part (i.e. type) of an object"""
return cdist_object.split(os.sep)[0]
def get_object_id_from_object(self, cdist_object):
"""Returns everything but the first part (i.e. object_id) of an object"""
return os.sep.join(cdist_object.split(os.sep)[1:])
def object_dir(self, cdist_object):
"""Returns the full path to the object (including .cdist)"""
return os.path.join(self.object_base_dir, cdist_object, DOT_CDIST)
def remote_object_dir(self, cdist_object):
"""Returns the remote full path to the object (including .cdist)"""
return os.path.join(REMOTE_OBJECT_DIR, cdist_object, DOT_CDIST)
def object_parameter_dir(self, cdist_object):
"""Returns the dir to the object parameter"""
return os.path.join(self.object_dir(cdist_object), "parameter")
def remote_object_parameter_dir(self, cdist_object):
"""Returns the remote dir to the object parameter"""
return os.path.join(self.remote_object_dir(cdist_object), "parameter")
def object_code_paths(self, cdist_object):
"""Return paths to code scripts of object"""
return [os.path.join(self.object_dir(cdist_object), "code-local"),
os.path.join(self.object_dir(cdist_object), "code-remote")]
def list_objects(self):
"""Return list of existing objects"""
objects = []
if os.path.isdir(self.object_base_dir):
object_paths = self.list_object_paths(self.object_base_dir)
for path in object_paths:
objects.append(os.path.relpath(path, self.object_base_dir))
return objects
def type_dir(self, type):
"""Return directory the type"""
return os.path.join(self.type_base_dir, type)
def type_explorer_dir(self, type):
"""Return directory that holds the explorers of a type"""
return os.path.join(self.type_dir(type), "explorer")
def type_gencode_paths(self, type):
"""Return paths to gencode scripts of type"""
return [os.path.join(self.type_base_dir, type, "gencode-local"),
os.path.join(self.type_base_dir, type, "gencode-remote")]
def type_manifest_path(self, type):
"""Return path to manifest of type"""
return os.path.join(self.type_base_dir, type, "manifest")
def remote_type_explorer_dir(self, type):
"""Return remote directory that holds the explorers of a type"""
return os.path.join(REMOTE_TYPE_DIR, type, "explorer")
def transfer_object_parameter(self, cdist_object):
"""Transfer the object parameter to the remote destination"""
# Create base path before using mkdir -p
self.remote_mkdir(self.remote_object_parameter_dir(cdist_object))
# Synchronise parameter dir afterwards
self.transfer_dir(self.object_parameter_dir(cdist_object),
self.remote_object_parameter_dir(cdist_object))
def transfer_global_explorers(self):
"""Transfer the global explorers"""
self.remote_mkdir(REMOTE_GLOBAL_EXPLORER_DIR)
self.transfer_dir(self.global_explorer_dir, REMOTE_GLOBAL_EXPLORER_DIR)
def transfer_type_explorers(self, type):
"""Transfer explorers of a type, but only once"""
if type in self.type_explorers_transferred:
log.debug("Skipping retransfer for explorers of %s", type)
return
else:
# Do not retransfer
self.type_explorers_transferred[type] = 1
src = self.type_explorer_dir(type)
remote_base = os.path.join(REMOTE_TYPE_DIR, type)
dst = self.remote_type_explorer_dir(type)
# Only continue, if there is at least the directory
if os.path.isdir(src):
# Ensure that the path exists
self.remote_mkdir(remote_base)
self.transfer_dir(src, dst)
def link_type_to_emulator(self):
"""Link type names to cdist-type-emulator"""
for type in self.list_types():
source = os.path.join(self.lib_dir, "cdist-type-emulator")
destination = os.path.join(self.bin_dir, type)
log.debug("Linking %s to %s", source, destination)
os.symlink(source, destination)
def run_global_explores(self):
"""Run global explorers"""
explorers = self.list_global_explorers()
if(len(explorers) == 0):
self.exit_error("No explorers found in", self.global_explorer_dir)
self.transfer_global_explorers()
for explorer in explorers:
output = self.global_explorer_output_path(explorer)
output_fd = open(output, mode='w')
cmd = []
cmd.append("__explorer=" + REMOTE_GLOBAL_EXPLORER_DIR)
cmd.append(self.remote_global_explorer_path(explorer))
self.run_or_fail(cmd, stdout=output_fd, remote=True)
output_fd.close()
def run_type_explorer(self, cdist_object):
"""Run type specific explorers for objects"""
# Based on bin/cdist-object-explorer-run
# Transfering explorers for this type
type = self.get_type_from_object(cdist_object)
self.transfer_type_explorers(type)
cmd = []
cmd.append("__explorer=" + REMOTE_GLOBAL_EXPLORER_DIR)
cmd.append("__type_explorer=" + self.remote_type_explorer_dir(type))
cmd.append("__object=" + self.remote_object_dir(cdist_object))
cmd.append("__object_id=" + self.get_object_id_from_object(cdist_object))
cmd.append("__object_fq=" + cdist_object)
# Need to transfer at least the parameters for objects to be useful
self.transfer_object_parameter(cdist_object)
explorers = self.list_type_explorers(type)
for explorer in explorers:
remote_cmd = cmd + [os.path.join(self.remote_type_explorer_dir(type), explorer)]
output = os.path.join(self.type_explorer_output_dir(cdist_object), explorer)
output_fd = open(output, mode='w')
log.debug("%s exploring %s using %s storing to %s",
cdist_object, explorer, remote_cmd, output)
self.run_or_fail(remote_cmd, stdout=output_fd, remote=True)
output_fd.close()
def init_deploy(self):
"""Ensure the base directories are cleaned up"""
log.debug("Creating clean directory structure")
self.remove_remote_dir(REMOTE_BASE_DIR)
self.remote_mkdir(REMOTE_BASE_DIR)
def run_initial_manifest(self):
"""Run the initial manifest"""
env = { "__manifest" : self.manifest_dir }
self.run_manifest(self.initial_manifest, extra_env=env)
def run_type_manifest(self, cdist_object):
"""Run manifest for a specific object"""
type = self.get_type_from_object(cdist_object)
manifest = self.type_manifest_path(type)
log.debug("%s: Running %s", cdist_object, manifest)
if os.path.exists(manifest):
env = { "__object" : self.object_dir(cdist_object),
"__object_id": self.get_object_id_from_object(cdist_object),
"__object_fq": cdist_object,
"__type": self.type_dir(type)
}
self.run_manifest(manifest, extra_env=env)
def run_manifest(self, manifest, extra_env=None):
"""Run a manifest"""
log.debug("Running manifest %s, env=%s", manifest, extra_env)
env = os.environ.copy()
env['PATH'] = self.bin_dir + ":" + env['PATH']
env['__target_host'] = self.target_host
env['__global'] = self.out_dir
# Legacy stuff to make cdist-type-emulator work
env['__cdist_conf_dir'] = self.conf_dir
env['__cdist_core_dir'] = os.path.join(self.base_dir, "core")
env['__cdist_local_base_dir'] = self.temp_dir
env['__cdist_manifest'] = self.initial_manifest
# Other environment stuff
if extra_env:
env.update(extra_env)
self.shell_run_or_debug_fail(manifest, [manifest], env=env)
def list_object_requirements(self, cdist_object):
"""Return list of requirements for specific object"""
file=os.path.join(self.object_dir(cdist_object), "require")
if os.path.isfile(file):
file_fd = open(file, "r")
requirements = file_fd.readlines()
file_fd.close()
# Remove \n from all lines
requirements = map(lambda s: s.strip(), requirements)
log.debug("Requirements for %s: %s", cdist_object, requirements)
else:
requirements = []
return requirements
def object_run(self, cdist_object, mode):
"""Run gencode or code for an object"""
log.debug("Running %s from %s", mode, cdist_object)
requirements = self.list_object_requirements(cdist_object)
for requirement in requirements:
log.debug("Object %s requires %s", cdist_object, requirement)
self.object_run(requirement, mode=mode)
#
# Setup env Variable:
#
env = os.environ.copy()
env['__target_host'] = self.target_host
env['__global'] = self.out_dir
env["__object"] = self.object_dir(cdist_object)
env["__object_id"] = self.get_object_id_from_object(cdist_object)
env["__object_fq"] = cdist_object
if mode == "gencode":
paths = self.type_gencode_paths(self.get_type_from_object(cdist_object))
for bin in paths:
if os.path.isfile(bin):
# omit "gen" from gencode and
outfile=os.path.join(self.object_dir(cdist_object),
os.path.basename(bin)[3:])
outfile_fd = open(outfile, "w")
# Need to flush to ensure our write is done before stdout write
outfile_fd.write(CODE_HEADER)
outfile_fd.flush()
self.shell_run_or_debug_fail(bin, [bin], env=env, stdout=outfile_fd)
outfile_fd.close()
status = os.stat(outfile)
# Remove output if empty, else make it executable
if status.st_size == len(CODE_HEADER):
os.unlink(outfile)
else:
# Add header and make executable - identically to 0o700
os.chmod(outfile, stat.S_IXUSR | stat.S_IRUSR | stat.S_IWUSR)
if mode == "code":
local_dir = self.object_dir(cdist_object)
remote_dir = self.remote_object_dir(cdist_object)
bin = os.path.join(local_dir, "code-local")
if os.path.isfile(bin):
self.run_or_fail([bin], remote=False)
local_remote_code = os.path.join(local_dir, "code-remote")
remote_remote_code = os.path.join(remote_dir, "code-remote")
if os.path.isfile(local_remote_code):
self.transfer_file(local_remote_code, remote_remote_code)
self.run_or_fail([remote_remote_code], remote=True)
def deploy_to(self):
"""Mimic the old deploy to: Deploy to one host"""
log.info("Deploying to " + self.target_host)
time_start = datetime.datetime.now()
self.init_deploy()
self.run_global_explores()
self.run_initial_manifest()
old_objects = []
objects = self.list_objects()
# Continue process until no new objects are created anymore
while old_objects != objects:
log.debug("Prepare stage")
old_objects = list(objects)
for cdist_object in objects:
if cdist_object in self.objects_prepared:
log.debug("Skipping rerun of object %s", cdist_object)
continue
else:
self.run_type_explorer(cdist_object)
self.run_type_manifest(cdist_object)
self.objects_prepared.append(cdist_object)
objects = self.list_objects()
log.debug("Actual run objects")
# Now do the final steps over the existing objects
for cdist_object in objects:
log.debug("Run object: %s", cdist_object)
self.object_run(cdist_object, mode="gencode")
self.object_run(cdist_object, mode="code")
time_end = datetime.datetime.now()
duration = time_end - time_start
log.info("Finished run of %s in %s seconds",
self.target_host,
duration.total_seconds())
def deploy_and_cleanup(self):
"""Do what is most often done: deploy & cleanup"""
self.deploy_to()
self.cleanup()
def banner(args):
"""Guess what :-)"""
print(BANNER)
sys.exit(0)
def config(args):
"""Configure remote system"""
process = {}
time_start = datetime.datetime.now()
for host in args.host:
c = Cdist(host, initial_manifest=args.manifest, home=args.cdist_home, debug=args.debug)
if args.parallel:
log.debug("Creating child process for %s", host)
process[host] = multiprocessing.Process(target=c.deploy_and_cleanup)
process[host].start()
else:
c.deploy_and_cleanup()
if args.parallel:
for p in process.keys():
log.debug("Joining %s", p)
process[p].join()
time_end = datetime.datetime.now()
log.info("Total processing time for %s host(s): %s", len(args.host),
(time_end - time_start).total_seconds())
if __name__ == "__main__":
# Construct parser others can reuse
parser = {}
# Options _all_ parsers have in common
parser['most'] = argparse.ArgumentParser(add_help=False)
parser['most'].add_argument('-d', '--debug',
help='Set log level to debug', action='store_true')
# Main subcommand parser
parser['main'] = argparse.ArgumentParser(description='cdist ' + VERSION)
parser['main'].add_argument('-V', '--version',
help='Show version', action='version',
version='%(prog)s ' + VERSION)
parser['sub'] = parser['main'].add_subparsers(title="Commands")
# Banner
parser['banner'] = parser['sub'].add_parser('banner',
add_help=False)
parser['banner'].set_defaults(func=banner)
# Config
parser['config'] = parser['sub'].add_parser('config',
parents=[parser['most']])
parser['config'].add_argument('host', nargs='+',
help='one or more hosts to operate on')
parser['config'].add_argument('-c', '--cdist-home',
help='Change cdist home (default: .. from bin directory)',
action='store')
parser['config'].add_argument('-i', '--initial-manifest',
help='Path to a cdist manifest',
dest='manifest', required=False)
parser['config'].add_argument('-p', '--parallel',
help='Operate on multiple hosts in parallel',
action='store_true', dest='parallel')
parser['config'].add_argument('-s', '--sequential',
help='Operate on multiple hosts sequentially (default)',
action='store_false', dest='parallel')
parser['config'].set_defaults(func=config)
for p in parser:
parser[p].epilog = "Get cdist at http://www.nico.schottelius.org/software/cdist/"
args = parser['main'].parse_args(sys.argv[1:])
# Most subcommands have --debug, so handle it here
if 'debug' in args:
if args.debug:
logging.root.setLevel(logging.DEBUG)
log.debug(args)
try:
args.func(args)
except KeyboardInterrupt:
sys.exit(0)

50
bin/cdist-code-run
View file

@ -1,50 +0,0 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# This binary is executed on the remote side to execute code
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<object> <type>"
set -ue
__cdist_object_self="$1"; shift
__cdist_code_type="$1"; shift
if [ ! -d "$(__cdist_object_dir "$__cdist_object_self")" ]; then
__cdist_exit_err "Object undefined"
fi
__cdist_code="$(__cdist_object_code "$__cdist_object_self" "${__cdist_code_type}")"
__cdist_echo info "Checking code-${__cdist_code_type}"
if [ -e "$__cdist_code" ]; then
if [ -f "$__cdist_code" ]; then
if [ -x "$__cdist_code" ]; then
__cdist_echo info "Executing code-${__cdist_code_type}"
__cdist_exec_fail_on_error "$__cdist_code"
else
__cdist_exit_err "$__cdist_code exists, but is not executable."
fi
else
__cdist_exit_err "$__cdist_code exists, but is not a file."
fi
fi

59
bin/cdist-deploy-to
View file

@ -1,59 +0,0 @@
#!/bin/sh
#
# 2010-2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Deploy configuration to a host
#
. cdist-config
[ $# -eq 1 ] || __cdist_usage "<target host>"
set -eu
# Kill children on interrupt - only in interactive scripts
trap __cdist_kill_on_interrupt INT TERM
__cdist_target_host="$1"
# Make target host available for non-core
export $__cdist_name_var_target_host="$__cdist_target_host"
export $__cdist_name_var_target_user="$__cdist_remote_user"
# Export variables for core, which others do not reset
export __cdist_local_base_dir
__cdist_echo info "cdist $__cdist_version: Configuring $__cdist_target_host "
################################################################################
# See cdist-stages(7)
#
# Prepare local and remote directories
__cdist_init_deploy "$__cdist_target_host"
# Transfer cdist executables
__cdist_echo info "Transferring cdist binaries to the target host "
cdist-dir push "$__cdist_target_host" \
"${__cdist_abs_mydir}" "${__cdist_remote_bin_dir}"
cdist-explorer-run-global "$__cdist_target_host"
cdist-manifest-run-init "$__cdist_target_host"
cdist-object-all "$__cdist_target_host" cdist-object-prepare
cdist-object-all "$__cdist_target_host" cdist-object-run
cdist-cache "$__cdist_target_host"
__cdist_echo info "cdist $__cdist_version: Successfully finished run"

51
bin/cdist-dir
View file

@ -1,51 +0,0 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Push a directory to a target, both sides have the same name (i.e. explorers)
# or
# Pull a directory from a target, both sides have the same name (i.e. explorers)
#
. cdist-config
[ $# -eq 4 ] || __cdist_usage "<push|pull> <target host> <src dir> <dst dir>"
set -ue
__cdist_action="$1"; shift
__cdist_target_host="$1"; shift
__cdist_src_dir="$1"; shift
__cdist_dst_dir="$1"; shift
# This will be the destination directory, so no subdirectories
# of the same name are created, if the directory is already existing
__cdist_top_dir="${__cdist_dst_dir%/*}"
if [ "$__cdist_action" = "push" ]; then
ssh "${__cdist_remote_user}@${__cdist_target_host}" \
"mkdir -p \"${__cdist_dst_dir}\""
scp -qr "$__cdist_src_dir" \
"${__cdist_remote_user}@${__cdist_target_host}:${__cdist_top_dir}"
elif [ "$__cdist_action" = "pull" ]; then
mkdir -p "${__cdist_dst_dir}"
scp -qr "${__cdist_remote_user}@${__cdist_target_host}:${__cdist_src_dir}" \
"${__cdist_top_dir}"
else
__cdist_exit_err "Unknown action $__cdist_action"
fi

43
bin/cdist-explorer-run-global
View file

@ -1,43 +0,0 @@
#!/bin/sh
#
# 2010-2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Copy & run the global explorers, i.e. not bound to types
#
. cdist-config
[ $# -eq 1 ] || __cdist_usage "<target_host>"
set -ue
__cdist_target_host="$1"; shift
__cdist_echo info "Running global explorers "
# copy the explorers
cdist-dir push "$__cdist_target_host" \
"${__cdist_explorer_dir}" "${__cdist_remote_explorer_dir}"
# run the initial explorers remotely
cdist-run-remote "${__cdist_target_host}" cdist-remote-explorer-run \
"$__cdist_name_var_explorer" "$__cdist_remote_explorer_dir" \
"$__cdist_remote_out_explorer_dir"
# retrieve the results
cdist-dir pull "$__cdist_target_host" \
"${__cdist_remote_out_explorer_dir}" "${__cdist_out_explorer_dir}"

56
bin/cdist-manifest-run
View file

@ -1,56 +0,0 @@
#!/bin/sh
#
# 2010 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Let's build a cconfig tree from a configuration
# And save it into the cache tree
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<target host> <manifest>"
set -u
__cdist_target_host="$1"; shift
__cdist_manifest="$1"; shift
################################################################################
# Export information for cdist-type-emulator or manifest
#
# Config dir should not get reset - FIXME: why did I do this?
export __cdist_conf_dir
# Used to record the source in the object
export __cdist_manifest
# Export information for manifests - __cdist_out_dir comes from cdist-config
export __global="$__cdist_out_dir"
################################################################################
# The actual run
#
# Ensure binaries exist and are up-to-date
cdist-type-build-emulation "${__cdist_out_type_bin_dir}" \
|| __cdist_exit_err "Failed to build type emulation binaries"
# prepend our path, so all cdist tools come before other tools
export PATH="${__cdist_out_type_bin_dir}:$PATH"
__cdist_exec_fail_on_error "${__cdist_manifest}"

75
bin/cdist-mass-deploy
View file

@ -1,75 +0,0 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Deploy configuration to many hosts
#
. cdist-config
[ $# -ge 1 ] || __cdist_usage "[-p] <target host> [target host ]"
set -u
# Kill children on interrupt - only in interactive scripts
trap __cdist_kill_on_interrupt INT TERM
filter()
{
awk -v host=$1 '{ print "[" host "] " $0 }'
}
parallel=""
if [ "$1" = "-p" ]; then
parallel=yes
shift
fi
i=0
while [ $# -gt 0 ]; do
if [ "$parallel" ]; then
cdist-deploy-to "$1" | filter "$1" &
# Record pid and host for use later
i=$((i+1))
eval pid_$i=$!
eval host_$i=\$1
else
cdist-deploy-to "$1" | filter "$1"
fi
shift
done
e=0
if [ "$parallel" ]; then
__cdist_echo info "Waiting for cdist-deploy-to jobs to finish"
while [ "$i" -gt 0 ]; do
eval pid=\$pid_$i
wait "$pid"
if [ $? -ne 0 ]; then
e=$((e+1))
eval e_host_$e=\$host_$i
fi
i=$((i-1))
done
fi
# Display all failed hosts after all runs are done, so the sysadmin gets them
while [ "$e" -gt 0 ]; do
eval host=\$host_$e
__cdist_echo error "Configuration of host $host failed."
e=$((e-1))
done

56
bin/cdist-object-all
View file

@ -1,56 +0,0 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Run the given command for each created object.
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<target host> <command>"
set -eu
__cdist_target_host="$1"; shift
__cdist_command="$1"; shift
__cdist_objects="$__cdist_tmp_dir/objects"
# Ensure object dir exists, so marker can be created
mkdir -p "${__cdist_out_object_dir}"
# Loop until we do not create new objects anymore
# which is equal to all objects have been run
touch "$__cdist_objects_created"
while [ -f "$__cdist_objects_created" ]; do
# Assume we're done after this run
rm "$__cdist_objects_created"
# Get listing of objects
__cdist_object_list "$__cdist_out_object_dir" > "$__cdist_objects"
# NEED TO CREATE ARRAY, SSH DESTROYS WHILE READ LOOP
while read __cdist_object; do
set -- "$@" "$__cdist_object"
done < "$__cdist_objects"
while [ $# -gt 0 ]; do
__cdist_object="$1"; shift
$__cdist_command "$__cdist_target_host" "$__cdist_object"
done
done

38
bin/cdist-object-code-run
View file

@ -1,38 +0,0 @@
#!/bin/sh
#
# 2010-2011 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Exec the code for the given object locally and remote
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<target host> <object>"
set -e
__cdist_target_host="$1"; shift
__cdist_object="$1"; shift
# Code local
export __cdist_out_object_dir="$__cdist_out_object_dir"
cdist-code-run "$__cdist_object" "${__cdist_name_gencode_local}"
# Code remote
cdist-run-remote "$__cdist_target_host" \
"cdist-code-run" "$__cdist_object" "${__cdist_name_gencode_remote}"

67
bin/cdist-object-explorer-run
View file

@ -1,67 +0,0 @@
#!/bin/sh
#
# 2010-2011 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Run the explorers for the given object on the target host.
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<target host> <object>"
set -eu
__cdist_target_host="$1"; shift
__cdist_object_self="$1"; shift
__cdist_object_id="$(__cdist_object_id_from_object "$__cdist_object_self")"
__cdist_type="$(__cdist_type_from_object "$__cdist_object_self")"
# Check if type of object has >= 1 explorer
__cdist_has_explorer="$(__cdist_type_has_explorer "$__cdist_type")"
# Run the type explorers for the current object if any
if [ "$__cdist_has_explorer" ]; then
if ! __cdist_type_explorer_pushed "$__cdist_type"; then
src_dir="$(__cdist_type_explorer_dir "$__cdist_type")"
dst_dir="$(__cdist_remote_type_explorer_dir "$__cdist_type")"
__cdist_echo info "Transfering explorers for $__cdist_type "
cdist-dir push "$__cdist_target_host" "$src_dir" "$dst_dir"
__cdist_type_explorer_pushed_add "$__cdist_type"
fi
__cdist_echo info "Running explorers"
# Copy object parameters
cdist-dir push "$__cdist_target_host" \
"$(__cdist_object_parameter_dir "$__cdist_object_self")" \
"$(__cdist_remote_object_parameter_dir "$__cdist_object_self")"
# Execute explorers
cdist-run-remote "$__cdist_target_host" \
"$__cdist_name_var_object=\"$(__cdist_remote_object_dir "$__cdist_object_self")\"" \
"$__cdist_name_var_object_id=\"$__cdist_object_id\"" \
"$__cdist_name_var_self=\"$__cdist_object_self\"" \
cdist-remote-explorer-run \
"$__cdist_name_var_type_explorer" \
"$(__cdist_remote_type_explorer_dir "$__cdist_type")" \
"$(__cdist_remote_object_type_explorer_dir "$__cdist_object_self")"
# Copy back results
cdist-dir pull "$__cdist_target_host" \
"$(__cdist_remote_object_type_explorer_dir "$__cdist_object_self")" \
"$(__cdist_object_type_explorer_dir "$__cdist_object_self")"
fi

62
bin/cdist-object-gencode
View file

@ -1,62 +0,0 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Generate code from one object (object must be relative path!)
# WARNING: OUTPUT ON STDOUT, ERRORS NEED TO BE ON STDERR!
#
. cdist-config
[ $# -eq 3 ] || __cdist_usage "<target host>" "<object>" "<type>"
set -eu
__cdist_target_host="$1"; shift
__cdist_object_self="$1"; shift
__cdist_gencode_type="$1"; shift
__cdist_type="$(__cdist_type_from_object "$__cdist_object_self")"
__cdist_type_gencode="$(__cdist_type_gencode "$__cdist_type" "$__cdist_gencode_type")"
__cdist_code_output="$(__cdist_object_code "$__cdist_object_self" "$__cdist_gencode_type")"
# export variables for the gencode script
export __object_id="$(__cdist_object_id_from_object "$__cdist_object_self")"
export __object="$(__cdist_object_dir "$__cdist_object_self")"
export __global="$__cdist_out_dir"
if [ -x "$__cdist_type_gencode" ]; then
__cdist_exec_fail_on_error "$__cdist_type_gencode" > "$__cdist_tmp_file"
else
if [ -e "$__cdist_type_gencode" ]; then
__cdist_exit_err "$__cdist_type_gencode exists, but is not executable"
fi
# Ensure it's empty, if there is no gencode
: > "$__cdist_tmp_file"
fi
# Only create code, if gencode created output
if [ "$(wc -l < "$__cdist_tmp_file")" -gt 0 ]; then
cat - "$__cdist_tmp_file" << eof > "$__cdist_code_output"
#
# The following code was generated by $__cdist_type_gencode
#
eof
chmod u+x "${__cdist_code_output}"
fi

38
bin/cdist-object-gencode-run
View file

@ -1,38 +0,0 @@
#!/bin/sh
#
# 2010 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# For the given object create the code to be executed on the target.
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<target host> <object>"
set -eu
__cdist_target_host="$1"; shift
__cdist_object_self="$1"; shift
__cdist_echo info "Generating local code "
cdist-object-gencode "$__cdist_target_host" "$__cdist_object_self" \
"${__cdist_name_gencode_local}"
__cdist_echo info "Generating remote code "
cdist-object-gencode "$__cdist_target_host" "$__cdist_object_self" \
"${__cdist_name_gencode_remote}"

58
bin/cdist-object-manifest-run
View file

@ -1,58 +0,0 @@
#!/bin/sh
#
# 2010 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Run the manifest for the given object.
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<target host> <object>"
set -eu
__cdist_target_host="$1"; shift
__cdist_object_self="$1"; shift
# FIXME: rename to __cdist_object_dir (everywhere!)
__cdist_cur_object_dir="$(__cdist_object_dir "$__cdist_object_self")"
__cdist_object_id="$(__cdist_object_id_from_object "$__cdist_object_self")"
__cdist_echo info "Checking manifest "
__cdist_type="$(__cdist_type_from_object "$__cdist_object_self")"
__cdist_manifest="$(__cdist_type_manifest "$__cdist_type")"
if [ -f "$__cdist_manifest" ]; then
if [ -x "$__cdist_manifest" ]; then
# Make __cdist_manifest available for cdist-type-emulator
export __cdist_manifest
__cdist_echo info "Executing manifest "
export $__cdist_name_var_object="$__cdist_cur_object_dir"
export $__cdist_name_var_object_id="$__cdist_object_id"
export $__cdist_name_var_type="$(__cdist_type_dir "$__cdist_type")"
cdist-manifest-run "$__cdist_target_host" "$__cdist_manifest"
# Tell cdist-object-run-all that there may be new objects
touch "$__cdist_objects_created"
else
__cdist_exit_err "${__cdist_manifest} needs to be executable."
fi
fi

47
bin/cdist-object-prepare
View file

@ -1,47 +0,0 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# For the given object:
# - run type explorers
# - run type manifest
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<target host> <object>"
set -eu
__cdist_target_host="$1"; shift
__cdist_object_self="$1"; shift
__cdist_object_dir="$(__cdist_object_dir "$__cdist_object_self")"
[ -d "$__cdist_object_dir" ] || __cdist_exit_err "Object undefined"
# Export to non-core for use in manifest and gencode scripts
export $__cdist_name_var_self=$__cdist_object_self
__cdist_object_prepared="$(__cdist_object_prepared "$__cdist_object_self")"
if [ ! -f "$__cdist_object_prepared" ]; then
__cdist_echo info "Preparing object"
cdist-object-explorer-run "$__cdist_target_host" "$__cdist_object_self"
cdist-object-manifest-run "$__cdist_target_host" "$__cdist_object_self"
# Mark this object as prepared
touch "$__cdist_object_prepared"
fi

65
bin/cdist-object-run
View file

@ -1,65 +0,0 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# For the given object:
# - run type explorers
# - run type manifest
# - generate code
# - copy object to target
# - execute code on target
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<target host> <object>"
set -eu
__cdist_target_host="$1"; shift
__cdist_object_self="$1"; shift
__cdist_object_dir="$(__cdist_object_dir "$__cdist_object_self")"
[ -d "$__cdist_object_dir" ] || __cdist_exit_err "Object undefined"
# Export to non-core for use in manifest and gencode scripts
export $__cdist_name_var_self=$__cdist_object_self
__cdist_object_finished="$(__cdist_object_finished "$__cdist_object_self")"
if [ ! -f "$__cdist_object_finished" ]; then
# Resolve dependencies, if any
__cdist_object_require="$(__cdist_object_require "$__cdist_object_self")"
if [ -f "$__cdist_object_require" ]; then
# NEED TO CREATE ARRAY, SSH DESTROYS WHILE READ LOOP
while read __cdist_requirement; do
set -- "$@" "$__cdist_requirement"
done < "$__cdist_object_require"
while [ $# -gt 0 ]; do
__cdist_requirement="$1"; shift
__cdist_echo info "Resolving requirement $__cdist_requirement"
cdist-object-run "$__cdist_target_host" "$__cdist_requirement"
done
fi
cdist-object-gencode-run "$__cdist_target_host" "$__cdist_object_self"
cdist-object-push "$__cdist_target_host" "$__cdist_object_self"
cdist-object-code-run "$__cdist_target_host" "$__cdist_object_self"
# Mark this object as done
touch "$__cdist_object_finished"
fi

2
bin/cdist-quickstart
View file

@ -249,7 +249,7 @@ fi
manifestinit="conf/manifest/init"
cat << eof
I'll know setup $manifestinit, containing the following code:
I'll now setup $manifestinit, containing the following code:
--------------------------------------------------------------------------------
# Every machine becomes a marker, so sysadmins know that automatic

68
bin/cdist-remote-explorer-run
View file

@ -1,68 +0,0 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# This binary is executed on the remote side to execute explorers
#
# It supports different variables names to be used, so __explorers
# and __type_explorers can be submitted :-)
#
. cdist-config
[ $# -eq 3 ] || __cdist_usage "<variable name> <explorer dir> <out dir>"
set -ue
# Variable that defines the home of the explorers
__cdist_variable_name="$1"; shift
# Find explorers here
__cdist_explorer_dir="$1"; shift
# Write output here
__cdist_my_out_dir="$1"; shift
# Setup environment
export $__cdist_variable_name="$__cdist_explorer_dir"
export __global="$__cdist_remote_out_dir"
mkdir -p "$__cdist_my_out_dir"
# Ensure there is at least one explorer
num="$(ls -1 "$__cdist_explorer_dir" | wc -l)"
if [ "$num" -lt 1 ]; then
__cdist_exit_err "${__cdist_explorer_dir}: Contains no explorers"
fi
# Execute all explorers
for explorer in "$__cdist_explorer_dir/"*; do
explorer_name="${explorer##*/}"
if [ -f "$explorer" ]; then
if [ ! -x "$explorer" ]; then
__cdist_exit_err "Explorer \"$explorer\" exists, but is not executable."
fi
# Execute explorers and save results in remote destination directory
"$explorer" > "${__cdist_my_out_dir}/$explorer_name"
else
if [ -e "$explorer" ]; then
__cdist_exit_err "Explorer \"$explorer\" exists, but is not a file."
fi
fi
done

1
bin/cdist.py Symbolic link
View file

@ -0,0 +1 @@
cdist

15
build.sh
View file

@ -88,18 +88,21 @@ case "$1" in
speeches)
cd "$SPEECHESDIR"
for speech in *tex; do
pdflatex $speech
pdflatex $speech
pdflatex $speech
pdflatex "$speech"
pdflatex "$speech"
pdflatex "$speech"
done
;;
web)
cp README ${WEBDIR}/${WEBPAGE}
rm -rf ${WEBDIR}/${WEBBASE}/man && mkdir ${WEBDIR}/${WEBBASE}/man
rm -rf ${WEBDIR}/${WEBBASE}/man
mkdir -p ${WEBDIR}/${WEBBASE}/man/man1 ${WEBDIR}/${WEBBASE}/man/man7
rm -rf ${WEBDIR}/${WEBBASE}/speeches && mkdir ${WEBDIR}/${WEBBASE}/speeches
cp ${MAN1DSTDIR}/*.html ${MAN7DSTDIR}/*.html ${WEBDIR}/${WEBBASE}/man
cp ${MAN1DSTDIR}/*.html ${WEBDIR}/${WEBBASE}/man/man1
cp ${MAN7DSTDIR}/*.html ${WEBDIR}/${WEBBASE}/man/man7
cp ${SPEECHESDIR}/*.pdf ${WEBDIR}/${WEBBASE}/speeches
git describe > ${WEBDIR}/${WEBBASE}/man/VERSION
@ -108,7 +111,7 @@ case "$1" in
cd ${WEBDIR} && make pub
;;
pub)
p|pu|pub)
git push --mirror
git push --mirror github
;;

5
conf/explorer/os
View file

@ -65,6 +65,11 @@ if [ -f /etc/SuSE-release ]; then
exit 0
fi
if [ -f /etc/cdist-preos ]; then
echo preos
exit 0
fi
uname_s="$(uname -s)"
# Assume there is no tr on the client -> do lower case ourselves

30
conf/type/__package_rubygem/explorer/pkg_status Executable file
View file

@ -0,0 +1,30 @@
#!/bin/sh
#
# 2011 Chase Allen James (nx-cdist@nu-ex.com)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
# Retrieve the status of a rubygem
#
if [ -f "$__object/parameter/name" ]; then
name="$(cat "$__object/parameter/name")"
else
name="$__object_id"
fi
# Except gem failing, if package is not known / installed
gem list -i "$name" 2>/dev/null || exit 0

51
conf/type/__package_rubygem/gencode-remote Executable file
View file

@ -0,0 +1,51 @@
#!/bin/sh
#
# 2011 Chase Allen James <nx-cdist@nu-ex.com>
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Manage Rubygem packages
#
if [ -f "$__object/parameter/name" ]; then
name="$(cat "$__object/parameter/name")"
else
name="$__object_id"
fi
state="$(cat "$__object/parameter/state")"
is_installed="$(grep "true" "$__object/explorer/pkg_status" || true)"
case "$state" in
installed)
# Install only if non-existent
if [ -z "$is_installed" ]; then
echo gem install \"$name\" --no-ri --no-rdoc
fi
;;
removed)
# Remove only if existent
if [ -n "$is_installed" ]; then
echo gem uninstall \"$name\"
fi
;;
*)
echo "Unknown state: $state" >&2
exit 1
;;
esac

49
conf/type/__package_rubygem/man.text Normal file
View file

@ -0,0 +1,49 @@
cdist-type__package_rubygem(7)
==============================
Chase Allen James <nx-cdist@nu-ex.com>
NAME
----
cdist-type__package_rubygem - Manage rubygem packages
DESCRIPTION
-----------
Rubygems is the default package management system for the Ruby programming language.
REQUIRED PARAMETERS
-------------------
state::
Either "installed" or "removed".
OPTIONAL PARAMETERS
-------------------
name::
If supplied, use the name and not the object id as the package name.
EXAMPLES
--------
--------------------------------------------------------------------------------
# Ensure sinatra is installed
__package_rubygem sinatra --state installed
# Remove package
__package_rubygem rails --state removed
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist-type(7)
- cdist-type__package(7)
COPYING
-------
Copyright \(C) 2011 Chase Allen James. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

1
conf/type/__package_rubygem/parameter/optional Normal file
View file

@ -0,0 +1 @@
name

1
conf/type/__package_rubygem/parameter/required Normal file
View file

@ -0,0 +1 @@
state

2
contrib/examples/nico/conf/type/__nico_afs/files/afs/CellServDB Normal file
View file

@ -0,0 +1,2 @@
>eof
129.132.186.89 # sgv-afs-sur5r

1
contrib/examples/nico/conf/type/__nico_afs/files/afs/ThisCell Normal file
View file

@ -0,0 +1 @@
eof

28
contrib/examples/nico/conf/type/__nico_afs/files/krb5/krb5.conf Normal file
View file

@ -0,0 +1,28 @@
[libdefaults]
default_realm = EOF
# The following krb5.conf variables are only for MIT Kerberos.
krb4_config = /etc/krb.conf
krb4_realms = /etc/krb.realms
kdc_timesync = 1
ccache_type = 4
forwardable = true
proxiable = true
allow_weak_crypto = true
[realms]
EOF = {
kdc = sgv-afs-sur5r.ethz.ch
admin_server = sgv-afs-sur5r.ethz.ch
}
[login]
krb4_convert = true
krb4_get_tickets = false
[logging]
default = FILE:/var/log/krb5.log
kdc = FILE:/var/log/krb5kdc/kdc.log
admin_server = FILE:/var/log/krb5kdc/kadmin.log

1
contrib/examples/nico/conf/type/__nico_afs/files/ssh/JAS.pub Normal file
View file

@ -0,0 +1 @@
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/Gbl64LT3VD5hsVtk1w670S2gue1OaW9XLg/Vk/iBqsYYrLGZh+TXJsAsXOSF1sZH6QSdNlpzTPsno9KFCTQTNlYe0IrWPGRrFGw1or3M6OugJrMeSiMYQ5nhH6HMjhzCFHH8Xh4Yku8fgi3ejPpySW8umx7nBL7ndiEJ9Y+lixNWMirEPLpz9YufFm9u8GX6bPrmjIBz6EhfxaqJ2N/N6gQB+4PmNopzqWHm+n4LToA9N8qwetSfhgEg7DVaD9SrJNjNTGSgii6CritT9sF8ZBq5CZG58DTyrxCndhhHte5OCGMb5ENgO4OBHA0MrD56unHrdAZCCosa9rI+pIll abr@cltHome03

1
contrib/examples/nico/conf/type/__nico_afs/files/ssh/bugblue.pub Normal file
View file

@ -0,0 +1 @@
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA17fnWJMTAuIvWcdnsasdfEkBabeXEhYaR03Qc/KBpS+iToSeUzlc5SeXoAczqSMsC0uYpDnhll9q3aAN82Xo06RI2ytd7TeXvFcVwzvXB+sNUsvtyPZ5Uyx7d2WTI87bm169KhGTJCaww8p+qa2UhkjOOaXZwMGjkHlvZ3WSZr5mLar9O3r4PG8SIqoFF0m+tcc2fcWIK3df3jWIk8g6j/jTaoIa18qsK/rtO90Ql20FMQJOZTKGKjHIOx2FLnXY5WKrXmXyyffgFpqi1rUAkCkjCKnm65fDjecn6FplzSUuZo/IB2GnHGNQVnNkU/18/G8KQKu9clkMxuUl8DYJBQ== bugblue@jabber.eof.name

1
contrib/examples/nico/conf/type/__nico_afs/files/ssh/docsteel.pub Normal file
View file

@ -0,0 +1 @@
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA1xfKoN8D3I0uGvc66E2cpnutdgALbSVIIWy0SBGV1ZLA4CehAL5BpMO0EI1TfH4LGgpjg+CnLXOSMd+bnvWjPTxGUbGcmK45UYCyn1LzSAfVKi9Mr06wbvQj0h5w1zEAwDqt63SHGjGOHO4TeCSrPxEROPMbZ1mP1ECsb4f+3WLWE5icbzOb/QMx2zNDd29rVvFJiJMOg4AiIs7pl/T7Qxg2yN6YlDIXSXLiE2i98O26kBNWRgAFcTNBqoUs5AkZ2F4LPUGbyuLpV+wtlpYcQXOUTLoRlKw+ovBQH3L6ae9n6+rFTIEEAS08C0MOzQPC3QjmfRMC8mxVkn22XnpHbQ== default

1
contrib/examples/nico/conf/type/__nico_afs/files/ssh/downhill.pub Normal file
View file

@ -0,0 +1 @@
ssh-dss AAAAB3NzaC1kc3MAAAIBAMiYvUWD0Yq3vm2Dp6xJCGvRGcGIyiT8+07FQJvAWUAzudyhA9r+h58gm4uCUJTV5W33Npf70Id5LSxyZJA7LcdTXTuMxhVfERURcT/GgtxCrs+aMguitNvf7QVuiKBrvuBmBPMV/6k8UEwf/7eCQRjoXE4jxEYKUT9SW8X94wuH0HUqOqBRD2F21uMrmLgDkb12RK/9yFRV3c7waOSQU/QtO+VFwPvBNDqUTqBL0LOJJif449vMxboOZ1noS94K0Lyfz9yOGEwBYck11c2UzH4KXbv8qNpYgtuCmDjZFM2J8dnhWJgkmThZtmyfnNFbHlW0HZItVvkqLZMPDlCIqR77J9OC4lawjrX2FFKhAzcrJuw7WXr2PcFKQUh/TiypcM5f6zuU3fs2+8ZYQdwvU6j/QNW4A/zqud7v/hjAMYCVe0EIWf2Qt0SS/nFLh7dZRGV21nK9Vq0zDncVPTgDl7/L62TYieO/j/1X3HjEp2JbR+mjBWsfKM7WYZDP16xiQzBuhr0vItnKMyN9V4AbDthjqesezKuXIhv0jP1z40MppM9mr40FJpgRSY3hyt3cZQKoO2vVJevnJMuufLheocAxo037f2PUpmSiJDOF1dLywmS2Gqk4GgzNN40IPrOcz0umtKjMAtXeU6MeapfmmEbwk+2zOo75gUt6SWU8UiFhAAAAFQC/Fm0V7OYGdazrUNuyn7mcPknZhQAAAgBzO496WPYnn7/H336kdMOUoue4Kgr9shpgjyyTJ8K9UsFwm+IEh9iS0QKPgGnj54AY1FpspfrbCmRI0tma+pj2QlnRRwIGcHd7eh2nCOmr+DSD/36VmoRPvGZaJDSTVotN+qgyjddNhCGx0417fqHXSKypbASphBqyvcKwkpk8S7o5nkMeOhufeCNdTHYsenKha4W+p8srGBsIZBISNeaGAsESIK4LuaShuolciTXtT7Nsqo123EXmjdHrT7DHQuwKI+jJpvHcz/UrissLs1JD+lFLOE07lkHPDc4KKK6IDUjm/DzsVTlTWdrcn8wUZ7fhUTt7e8UDNHs6bMlnhtVkXEsHS3sbYRsbF9179jufUumXKLKkjLzpTZwni8D6GxnjUn51hQ9Ifb9UOlSlkRq+cqOU+TRQd5aQSdiXy5Ymao719MHBrAhH1aLbi9pk4VO3GGFNy/w54ZY2LRIXZsGMBgFTXzHyzqv1ejeQtOuQZw9xOzt9IZU3WuMKVGGR9D61rsgxbGI1aWGaDlyhMSL9LYyqkmEqVqnAyydAVmhpSxhoXjbDrwE/IRdjJwjlK/6NxUck9g7Ekc9pHrow0OmqH9k6SCd2npXBUybTXPYqwHUjY/KccJsW0Ia2OECGN6KgNWdSfFeAGJBrv2ct78laSyNgIguM+0MNOZQSr6QfSQAAAgEAqkHqwfmEtrlc7hKtvenEf4Dkgt6H33U7MJILNOo9qrn/StDeuuO1snbO2wbd8weJop7gnp29zJGRKYcs/p2T3YjOd9R6aRGLOlT+jEZjP1RMPeuT7W4UFajP34SezYc1MAMeT9wkABEBQyj6s+4CvC2tKJWoirziAvkSPfkYdOc3u9I1LuXHu21tP+Lky3K9KylsbnHDG6vw05GH3HbeXIa6LtxAkXiPp/+r0dABO5NzglHS36uaD4mbQGh2dIzWPm4j8mLrjg911R8XnLPdTgT2EB9mvXksLjWEEq7qzTSdacFG0127O+i4Be3h1+5wG5HM8ST1n5hTOX8tTywF3DJL5HbLNHdDQo/YlT6l/wk6HbnYdcZHviVHi6va3/lFTdhoTEPz5sgDYQp5/0vobiMyIRHSZwzcYmswHumpf6Wql/phff8xigJBDAbGdFgx1Jk2OoOVGNWEchZuzlXyfgQpatnzBcR9CAAJvAfQLPqcHb8jGyBINuoNY/0OsAbsQDDzSjOLkBEBVgBTVZbykcik2n7kBQFeNj8zAUeLQCnQcEGIGY7OkPoxaAHBGKh4+e5Wymz0fryKTJ9io78qLHTmc6xs/ep4UwlO4Ee/e9p7cGb2pOl98jGFCWUklqyzeksBTOQcfKZBPV5dYap4m6nrw59XYpVbV9yPRJ/yMhE= downhill@downhill.homelinux.net

1
contrib/examples/nico/conf/type/__nico_afs/files/ssh/sur5r@samsa.pub Normal file
View file

@ -0,0 +1 @@
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAABAEApDTX05FmMS1ArryWL6MmgcQeRFRU6E4Rgg+cPTeBuHCtBnnUoCUw7lhD2OICHIzQnfyMQasJOHH/4PnsrAxyO1Lm4KtK3zsdSdA5auunxAy0n7PZwaRKDTXCgpfXvi7ZAlzA+Mq/OjqtltfqdJgToYKxWoehwmltlwibuU1fke2v85VcTbCQRAttc6+0Y+3fToyTWecZM+X0uCsz1B5s07CLrMddo3lPVAlhYStSCbHflsM6C+NQAbxlsjkVFavii7WDYMYOd5FU0jIJt2Fy6u7Lx0ubY6hgGsP+mDOCCaRGA0JdjvWctBruj227CLswtAnYIFxvPi82R/okfx2X4YPtbqUPyLaUbr0G93g+raJEr/uXbour+wRd0TewOLq4VstwOsDfj9pm0wMFyIbsY2y97k6UZc0TE5pu3USGyW9/ainy5zD4TK4Al8lMkDHil6eItlud66KDZ4p5n5gzwuBj+ZOpOcBD5ZqLqDKst3YlHx6EuA1ddObBTrfy/nGphYYhWl1rbJ9+XOhSD8f/LIr5mjLEpSta6rHS/3dpLpSRGIy8ReG0RLbfay/fS74Iw0rEGOe/XgrfNDT0VwsgJMNV81sReepk27DaFD/vES/iPAymbbId8e8IQ7kDbhV0yK0yTkCZ5capqa9HXcut0SFRVJYGxzGT+ji5o/DcyAcaQHK9IW5i89sp04aVtZO+KZZDd2GmcDy+v9+fmBsSx2AFsoOQSXX31jJVdAXNw8idTNb88/3XDZIIEl+1KJ8Pv4UFXBW72RArpOxOrsDZYQjtaLQ4ZjTP3h823ZrBh4W3osb1znS7x4MmWBLPkmLCS0zmN8nbqhKi5EsTmSheCjCzySGShkyeqSGRIRGFk5PUsgh7hYvE+f7BhWD6x8MHbuUp9y0ODQonp022Dc4WzTc6Aa023MSNRuwV373tOqPYveuoPXDTS6vzV3IjXfv9a844HIkUTZbErxkavGBN5TEMgrALJkc8LS6M6Zg8odou8N2vWoNaKPn/DC5+H+FxJ2PSrK93hfzRMgvFvPSFzzDnixUFJClSqxf1Wvx9OW2pUuePAXBlcuFhAAnWV6w7fkmII8+qGk3m438dt2Sq6owmItzqIeJ6bohMsb9ejxeDyfk9DAQwmjS7S+BY47bYgAfsesZNRnlbeffp4rP0MAx4KoUXoNzb8tl1Jljulq58C0e5+EzRILqOYLM7WytY3+giqxN4zQJsqDp3mgSUaZ5SEHnA8JVi77MvABSoVclrEaujSLVEkxTBOiR252SFUWp3lWbxpGzBjd7gX4JAiytJql5xr+o4+nCy2O2laE6c2xS7en6SVEgC7jWflCsbDfvy9U+w1qOxa0j9fXE9Rw== sur5r@samsa

38
bin/cdist-type-build-emulation → contrib/examples/nico/conf/type/__nico_afs/manifest
View file

@ -1,6 +1,6 @@
#!/bin/sh
#
# 2010-2011 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
@ -17,27 +17,29 @@
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
# Build pseudo binaries for type emulation
#
. cdist-config
[ $# -eq 1 ] || __cdist_usage "<out dir>"
set -eu
__cdist_output_dir="$1"; shift
__directory /root/.ssh --mode 0700
__cdist_type_emulator="$__cdist_abs_mydir/cdist-type-emulator"
__file /root/.ssh/authorized_keys --mode 0600
if [ ! -d "${__cdist_type_dir}" ]; then
__cdist_exit_err "$__cdist_type_dir must exist and contain available types"
fi
cd "$__type/files/ssh"
for key in *; do
require="__directory/root/.ssh" \
__addifnosuchline ssh-$key --file /root/.ssh/authorized_keys \
--line "$(cat "$key")"
done
# Get Types
cd "${__cdist_type_dir}"
ls -1 > "${__cdist_tmp_file}"
for pkg in openafs-client openafs-krb5 krb5-user; do
__package $pkg --state installed
done
# Create binaries
mkdir -p "${__cdist_output_dir}"
while read type; do
ln -sf "${__cdist_type_emulator}" "${__cdist_output_dir}/${type}"
done < "${__cdist_tmp_file}"
# Kerberos Config
__file /etc/krb5.conf --source "$__type/files/krb5/krb5.conf"
# AFS config
cd "$__type/files/afs"
for afsconf in *; do
__file /etc/openafs/$afsconf --source "$__type/files/afs/$afsconf"
done

0
contrib/examples/nico/conf/type/__nico_afs/singleton Normal file
View file

1
contrib/examples/nico/conf/type/__nico_desktop/files/hostname Normal file
View file

@ -0,0 +1 @@
scheibe

4
contrib/examples/nico/conf/type/__nico_desktop/files/slim-preseed Normal file
View file

@ -0,0 +1,4 @@
# Use slim, not gdm, if both are available
# Setup for slim, but value is available for gdm as well:
# gdm shared/default-x-display-manager select slim
slim shared/default-x-display-manager select slim

47
contrib/examples/nico/conf/type/__nico_desktop/manifest Executable file
View file

@ -0,0 +1,47 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
#
# Ensure hostname is setup
#
__file /etc/hostname --source "$__object/parameter/hostname"
# All Linux distros have those
for pkg in dvdbackup mplayer x11vnc xfmpc; do
__package $pkg --state installed
done
case "$(cat "$__global/explorer/os")" in
debian|ubuntu)
require="__package/zsh" __user lyni --uid 1000 --shell /bin/zsh
for pkg in chromium-browser xfce4 xtightvncviewer; do
__package $pkg --state installed
done
# Make slim default desktop on Debian/Ubuntu
__debconf_set_selections slim --file "$__type/files/slim-preseed"
require="__debconf_set_selections/slim" __package_apt slim --state installed
;;
esac
# Including gaming fun - not within examples, too big for core inclusion :-p
# __nico_dosbox

1
contrib/examples/nico/conf/type/__nico_desktop/parameter/required Normal file
View file

@ -0,0 +1 @@
hostname

0
contrib/examples/nico/conf/type/__nico_desktop/singleton Normal file
View file

4
contrib/examples/nico/conf/type/__nico_mpd/files/slim-preseed Normal file
View file

@ -0,0 +1,4 @@
# Use slim, not gdm, if both are available
# Setup for slim, but value is available for gdm as well:
# gdm shared/default-x-display-manager select slim
slim shared/default-x-display-manager select slim

24
contrib/examples/nico/conf/type/__nico_mpd/manifest Normal file
View file

@ -0,0 +1,24 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
__package mpd --state installed
require="__package/mpd" __file /etc/mpd.conf --source "$__type/files/mpd.conf"

1
contrib/examples/nico/conf/type/__nico_mpd/parameter/required Normal file
View file

@ -0,0 +1 @@
hostname

0
contrib/examples/nico/conf/type/__nico_mpd/singleton Normal file
View file

6
contrib/examples/nico/conf/type/__nico_network/files/interfaces-eth0 Normal file
View file

@ -0,0 +1,6 @@
# generated by cdist
auto lo eth0
iface lo inet loopback
iface eth0 inet dhcp

11
contrib/examples/nico/conf/type/__nico_network/files/interfaces-wlan0 Normal file
View file

@ -0,0 +1,11 @@
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
auto wlan0
iface wlan0 inet dhcp
wpa-ssid undef
wpa-psk rotrussland

40
bin/cdist-cache → contrib/examples/nico/conf/type/__nico_network/manifest
View file

@ -1,6 +1,6 @@
#!/bin/sh
#
# 2010 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
@ -18,22 +18,26 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Let's build a cconfig tree from a configuration
# And save it into the cache tree
case "$(cat "$__global/explorer/os")" in
debian|ubuntu)
interface="$(cat $__object/parameter/interface)"
#
# Only Debian and alike supported currently
#
destination=/etc/network/interfaces
case "$interface" in
eth0|wlan0)
source="$__type/files/interfaces-${interface}"
;;
*)
echo "Unknown interface: $interface" >&2
exit 1
;;
esac
. cdist-config
[ $# -eq 1 ] || __cdist_usage "<target host>"
set -u
__cdist_target_host="$1"; shift
# Create base to move into
mkdir -p "${__cdist_local_base_cache_dir}"
# Now determine absolute path
__cdist_ddir="$(__cdist_host_cache_dir "$__cdist_target_host")"
__cdist_echo info "Saving cache to $__cdist_ddir "
rm -rf "$__cdist_ddir"
mv "$__cdist_local_base_dir" "$__cdist_ddir"
__file "$destination" --source "$source"
;;
esac

1
contrib/examples/nico/conf/type/__nico_network/parameter/required Normal file
View file

@ -0,0 +1 @@
interface

0
contrib/examples/nico/conf/type/__nico_network/singleton Normal file
View file

4
contrib/examples/nico/conf/type/__nico_nfs_client/files/slim-preseed Normal file
View file

@ -0,0 +1,4 @@
# Use slim, not gdm, if both are available
# Setup for slim, but value is available for gdm as well:
# gdm shared/default-x-display-manager select slim
slim shared/default-x-display-manager select slim

47
contrib/examples/nico/conf/type/__nico_nfs_client/manifest Executable file
View file

@ -0,0 +1,47 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
__directory /home/services
for nfsshare in audio video; do
dir="/home/services/$nfsshare"
__addifnosuchline nfs-$nfsshare --file /etc/fstab \
--line "192.168.42.1:$dir $dir nfs defaults 0 0"
require="__directory/home/services" __directory $dir
done
__directory /home/services/eingehend
for nfsshare in bibliothek buch spiegel; do
dir="/home/services/eingehend/$nfsshare"
__addifnosuchline nfs-$nfsshare --file /etc/fstab \
--line "192.168.42.1:$dir $dir nfs defaults,noauto 0 0"
require="__directory/home/services" __directory $dir
done
case "$(cat "$__global/explorer/os")" in
debian|ubuntu)
__package nfs-common --state installed
;;
fedora|archlinux)
__package nfs-utils --state installed
;;
esac

0
contrib/examples/nico/conf/type/__nico_nfs_client/singleton Normal file
View file

27
bin/cdist-run-remote → contrib/examples/nico/conf/type/__nico_notebook/manifest
View file

@ -18,16 +18,21 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Run a cdist binary on the remote side
require="__package/zsh" __user nico --uid 1000 --shell /bin/zsh
#
# Backup HD
#
for hd in usbhd eth-usbhd; do
dir="/home/services/$hd"
__addifnosuchline hd-$hd --file /etc/fstab \
--line "/dev/mapper/$hd $dir auto defaults,noauto 0 0"
__directory $dir
done
. cdist-config
[ $# -ge 2 ] || __cdist_usage "<target_host> <binary> [opts]"
set -ue
__cdist_target_host="$1"; shift
ssh "${__cdist_remote_user}@${__cdist_target_host}" \
"export PATH=\"${__cdist_remote_bin_dir}:\$PATH\";" \
"export __cdist_out_object_dir=\"$__cdist_remote_out_object_dir\";" \
"$@"
#
# Standard everywhere packages
#
__nico_packages
__motd

0
contrib/examples/nico/conf/type/__nico_notebook/singleton Normal file
View file

33
contrib/examples/nico/conf/type/__nico_packages/manifest Executable file
View file

@ -0,0 +1,33 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
for pkg in atop screen vim wget zsh; do
__package "$pkg" --state installed
done
case "$(cat "$__global/explorer/os")" in
fedora)
__package nc --state installed
;;
*)
__package netcat --state installed
;;
esac

0
contrib/examples/nico/conf/type/__nico_packages/singleton Normal file
View file

13
contrib/examples/nico/conf/type/__nico_sudo/files/sudo-nico Normal file
View file

@ -0,0 +1,13 @@
#
# Cdist managed file
#
# Personal one: nico, shared one: lyni
User_Alias NICO = nico, lyni
Defaults timestamp_timeout=5
Defaults !tty_tickets
# Give out permissions
NICO ALL=(ALL) ALL
NICO ALL=(ALL) NOPASSWD: /usr/sbin/pm-suspend

18
bin/cdist-object-push → contrib/examples/nico/conf/type/__nico_sudo/manifest
View file

@ -1,7 +1,6 @@
#!/bin/sh
#
# 2010 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
@ -19,17 +18,10 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Transfer the given object to the target host.
#
. cdist-config
[ $# -eq 2 ] || __cdist_usage "<target host> <object>"
set -eu
destination=/etc/sudoers.d/nico
source="$__type/files/sudo-nico"
__cdist_target_host="$1"; shift
__cdist_object_self="$1"; shift
require="__package/sudo" __file "$destination" --source "$source" --mode 0440
__cdist_echo info "Transferring object"
cdist-dir push "$__cdist_target_host" \
"$(__cdist_object_dir "$__cdist_object_self")" \
"$(__cdist_remote_object_dir "$__cdist_object_self")"
__package sudo --state installed

31
contrib/examples/nico/conf/type/__nico_sudo/parameter/gencode Normal file
View file

@ -0,0 +1,31 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# USEFUL DESCRIPTION
#
#
# This file should generate code on stdout, which will be collected by cdist
# and run on the target.
#
# To tell cdist to make use of it, you need to make it executable (chmod +x)
#
#

31
contrib/examples/nico/conf/type/__nico_sudo/parameter/manifest Normal file
View file

@ -0,0 +1,31 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# USEFUL DESCRIPTION
#
#
# This is the manifest, which can be used to create other objects like this:
# __file /path/to/destination --source /from/where/
#
# To tell cdist to make use of it, you need to make it executable (chmod +x)
#
#

0
contrib/examples/nico/conf/type/__nico_sudo/parameter/optional Normal file
View file

0
contrib/examples/nico/conf/type/__nico_sudo/parameter/required Normal file
View file

0
contrib/examples/nico/conf/type/__nico_sudo/singleton Normal file
View file

2
contrib/examples/nico/conf/type/__nico_tee/files/99-apt-nico Normal file
View file

@ -0,0 +1,2 @@
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

1
contrib/examples/nico/conf/type/__nico_tee/files/hostname Normal file
View file

@ -0,0 +1 @@
scheibe

16
bin/cdist-manifest-run-init → contrib/examples/nico/conf/type/__nico_tee/manifest
View file

@ -1,6 +1,6 @@
#!/bin/sh
#
# 2010-2011 Nico Schottelius (nico-cdist at schottelius.org)
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
@ -18,17 +18,11 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Let's build a cconfig tree from a configuration
# And save it into the cache tree
#
. cdist-config
[ $# -eq 1 ] || __cdist_usage "<target host>"
set -e
__cdist_target_host="$1"; shift
eval export $__cdist_name_var_manifest=\"\$__cdist_manifest_dir\"
__package unattended-upgrades --state installed
__cdist_echo info "Running initial manifest for $__cdist_target_host "
cdist-manifest-run "$__cdist_target_host" "$__cdist_manifest_init"
__file /etc/apt/apt.conf.d/99-apt-nico \
--source "$__type/files/99-apt-nico" \
--mode 0644

0
contrib/examples/nico/conf/type/__nico_tee/singleton Normal file
View file

5
contrib/types_pending_inclusion/__package_zypper/README Normal file
View file

@ -0,0 +1,5 @@
This type was not accepted, because cleanups are needed and the
manpage does not build.
If you read this and want this code available in the cdist core,
just fix it and submit a git url :-)

15
bin/cdist-env → contrib/types_pending_inclusion/__package_zypper/explorer/pkg_version
View file

@ -18,16 +18,13 @@
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Setup environment for use with cdist - must be standalone!
# Retrieve the status of a package - parsed dpkg output
#
export PATH="$(cd "${0%/*}" && pwd -P):$PATH"
export MANPATH="$(cd "${0%/*}/../doc/man" && pwd -P):$MANPATH"
if [ "$(echo ${SHELL##*/} | grep 'csh$')" ]; then
echo setenv PATH $PATH \;
echo setenv MANPATH $MANPATH
if [ -f "$__object/parameter/name" ]; then
name="$(cat "$__object/parameter/name")"
else
echo export PATH=$PATH
echo export MANPATH=$MANPATH
name="$__object_id"
fi
rpm -q --whatprovides "$name" 2>/dev/null || true

51
contrib/types_pending_inclusion/__package_zypper/gencode-remote Executable file
View file

@ -0,0 +1,51 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Manage packages with yum (mostly Fedora)
#
if [ -f "$__object/parameter/name" ]; then
name="$__object/parameter/name"
else
name="$__object_id"
fi
state="$(cat "$__object/parameter/state")"
opts="-n -q"
not_installed="^no package provides"
case "$state" in
installed)
if grep -q "$not_installed" "$__object/explorer/pkg_version"; then
echo zypper $opts install \"$name\"
fi
;;
removed)
if ! grep -q "$not_installed" "$__object/explorer/pkg_version"; then
echo zypper $opts remove \"$name\"
fi
;;
*)
echo "Unknown state: $state" >&2
exit 1
;;
esac

52
contrib/types_pending_inclusion/__package_zypper/man.text Normal file
View file

@ -0,0 +1,52 @@
cdist-type__package_zypper(7)
==========================
Franky Van Liedekerke <franky.van.liedekerke--@--telenet.be>
NAME
----
cdist-type__package_zypper - Manage packages with zypper
DESCRIPTION
-----------
zypper is usually used on the Suse distribution to manage packages.
REQUIRED PARAMETERS
-------------------
state::
Either "installed" or "removed".
OPTIONAL PARAMETERS
-------------------
name::
If supplied, use the name and not the object id as the package name.
EXAMPLES
--------
--------------------------------------------------------------------------------
# Ensure zsh in installed
__package_zypper zsh --state installed
# If you don't want to follow pythonX packages, but always use python
__package_zypper python --state installed --name python2
# Remove obsolete package
__package_zypper puppet --state removed
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist-type(7)
- cdist-type__package(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

1
contrib/types_pending_inclusion/__package_zypper/parameter/optional Normal file
View file

@ -0,0 +1 @@
name

1
contrib/types_pending_inclusion/__package_zypper/parameter/required Normal file
View file

@ -0,0 +1 @@
state

7
doc/changelog
View file

@ -1,4 +1,9 @@
1.7.1:
2.0.0:
* New Type: __package_rubygem (Chase Allen James)
* __self replaced by __object_fq (or so)
* Rewrote cdist in Python
1.7.1: 2011-07-26
* Documentation: Add explorers to reference
* Documentation: Typo cleanup (Derek Brost)
* Type __key_value: Bugfix (Steven Armstrong)

12
doc/dev/benchmark-parallel-deploy Executable file
View file

@ -0,0 +1,12 @@
#!/bin/sh
outfile="$1"; shift
(
for host in "$@"; do
hosts="$hosts $host"
cdist config -c ~/p/cdist-nutzung -p $hosts 2>&1
done
) | tee "$outfile"
echo "----------"
grep 'INFO: Total processing time for' "$outfile" | sed 's/.*: //'

24
doc/dev/lastchanges Executable file
View file

@ -0,0 +1,24 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Show the lastest changes from this (upcoming) release
#
awk '/^$/ { exit } { print $0 } END { print "-----" }' < "${0%/*}/../changelog"

56
doc/dev/logs/2011-04-27.benchmark Normal file
View file

@ -0,0 +1,56 @@
% x200
data = [ 1 104; 2 129; 6 249 ; 25 1267 ]
% x201, 4.2.2.4 dns, eth
data = [ 1 143; 2 159; 3 198; 4 244; 5 299; 6 350; 7 435; 8 429 ];
% x201, von zuhause aus
data2 = [ 226 242 275 296 306 357 403 400 409 685 617 672 ]
plot(0:size(data)(2)-1, data)
hold off;
plot(data(:,1), data(:,2))
% per host time:
data(:,2)' ./ data(:,1)'
hold on;
plot(data(:,1), data(:,2)' ./ data(:,1)')
Testing on
Intel(R) Core(TM)2 Duo CPU P8400 @ 2.26GHz
4 GiB RAM
Intel Gbit Nic
- 169 objects, all done (i.e. rerun)
- runs are cpu bound
1 host:
core: cdist 1.6.2: Successfully finished run
Run: 104s (1 minutes)
2 hosts:
[13:48] kr:cdist-nutzung% mytime cdist-mass-deploy -p ikq03.ethz.ch ikq04.ethz.ch
Run: 129s (2 minutes)
6 hosts: (7 specified, but ikq01.ethz.ch is dead)
cdist-mass-deploy -p ikq01.ethz.ch ikq02.ethz.ch ikq03.ethz.ch ikq04.ethz.ch ikq05.ethz.ch ikq06.ethz.ch ikq07.ethz.ch
Run: 294s (4 minutes)
25 hosts
[13:58] kr:cdist-nutzung% mytime cdist-mass-deploy -p $(sed 's/^root@//' ~/.dsh/group/ikr)
Run: 723s (12 minutes)
25 hosts without proxy command now:
Run: 1267s (21 minutes)
Run: 1212s (20 minutes)
[14:11] kr:cdist-nutzung% mytime cdist-mass-deploy -p $(sed 's/^root@//' ~/.dsh/group/ikr)

9
doc/dev/logs/2011-04-27.benchmark.dash Normal file
View file

@ -0,0 +1,9 @@
Run: 78s (1 minutes)
[13:25] kr:cdist-nutzung% mytime cdist-deploy-to ikq02.ethz.ch
% mit dash, eth, x201, Mit Apr 27 13:41:49 CEST 2011
data = [ 0 73 77 89 107 130 151 180 197 228 251 260 199 295 335 276 ]
plot(0:size(data2)(2)-1, data2)

4
doc/dev/logs/2011-09-08.obsolete_debugging Normal file
View file

@ -0,0 +1,4 @@
Debugging cdist:
[0:13] kr:cdist-nutzung% ./local/update-local-core && __cdist_debug=1 __cdist_local_base_dir=/tmp/cdist cdist-deploy-to ikq04.ethz.ch

110
doc/dev/logs/2011-09-12 Normal file
View file

@ -0,0 +1,110 @@
Benchmark from home/X201 (2 cores, 4 threads):
ikq (1): 72.192397
ikq (2): INFO: Total processing time for 2 hosts: 74.1845
ikq* (6): INFO: Total processing time: 117.572312
ikq* + ikr3 (9): INFO: Total processing time: 120.307662
ikq* + ikr (14): INFO: Total processing time: 139.769807
ikq* + ikr (18): INFO: Total processing time: 186.354398
ikq* + ikr (22): INFO: Total processing time: 225.793533
ikq* + ikr (26): INFO: Total processing time: 237.06687
ikq* + ikr (31): INFO: Total processing time: 276.912414
ikr07.ethz.ch ikr09.ethz.ch ikr10.ethz.ch ikr11.ethz.ch
ikr13.ethz.ch ikr14.ethz.ch ikr15.ethz.ch ikr16.ethz.ch
ikr17.ethz.ch ikr19.ethz.ch ikr20.ethz.ch ikr21.ethz.ch
ikr23.ethz.ch ikr24.ethz.ch ikr25.ethz.ch ikr26.ethz.ch
ikr27.ethz.ch ikr28.ethz.ch ikr29.ethz.ch ikr30.ethz.ch
ikr31.ethz.ch
cdist -c ~/p/cdist-nutzung -p ikq02.ethz.ch ikq03.ethz.ch ikq04.ethz.ch ikq05.ethz.ch ikq06.ethz.ch ikq07.ethz.ch ikr01.ethz.ch ikr02.ethz.ch ikr03.ethz.ch ikr05.ethz.ch ikr07.ethz.ch ikr09.ethz.ch ikr10.ethz.ch ikr11.ethz.ch ikr13.ethz.ch ikr14.ethz.ch ikr15.ethz.ch ikr16.ethz.ch ikr17.ethz.ch ikr19.ethz.ch ikr20.ethz.ch ikr21.ethz.ch
--------------------------------------------------------------------------------
INFO: Total processing time for 1 hosts: 72.166661
INFO: Total processing time for 2 hosts: 76.633228
INFO: Total processing time for 3 host(s): 77.199817
INFO: Total processing time for 4 host(s): 94.045175
INFO: Total processing time for 5 host(s): 103.226354
INFO: Total processing time for 6 host(s): 107.76097
INFO: Total processing time for 7 host(s): 101.571705
INFO: Total processing time for 8 host(s): 107.600093
INFO: Total processing time for 9 host(s): 116.500371
INFO: Total processing time for 10 host(s): 119.445805
INFO: Total processing time for 11 host(s): 123.944385
INFO: Total processing time for 12 host(s): 130.499098
INFO: Total processing time for 13 host(s): 137.250861
INFO: Total processing time for 14 host(s): 154.9841
INFO: Total processing time for 15 host(s): 139.659637
INFO: Total processing time for 16 host(s): 142.70005
INFO: Total processing time for 17 host(s): 148.541452
INFO: Total processing time for 18 host(s): 159.360809
INFO: Total processing time for 19 host(s): 171.907864
INFO: Total processing time for 20 host(s): 178.76695
INFO: Total processing time for 21 host(s): 183.856671
INFO: Total processing time for 22 host(s): 194.504221
INFO: Total processing time for 23 host(s): 207.314842
INFO: Total processing time for 24 host(s): 215.846502
INFO: Total processing time for 25 host(s): 217.223581
INFO: Total processing time for 26 host(s): 238.591705
INFO: Total processing time for 27 host(s): 238.478493
INFO: Total processing time for 28 host(s): 246.058718
INFO: Total processing time for 29 host(s): 264.208372
INFO: Total processing time for 30 host(s): 265.560685
INFO: Total processing time for 31 host(s): 282.264488
--------------------------------------------------------------------------------
Use:
grep "^INFO: Total processing time" doc/dev/logs/2011-09-12 | sed 's/.*: //'
octave
times = [ /* paste here ]
plot(times)
# keep the graph
hold on
# Scale linearly with the single host value
plot((1:31)*times(1))
--------------------------------------------------------------------------------
code:
octave
time = [
72.166661
76.633228
77.199817
94.045175
103.226354
107.76097
101.571705
107.600093
116.500371
119.445805
123.944385
130.499098
137.250861
154.9841
139.659637
142.70005
148.541452
159.360809
171.907864
178.76695
183.856671
194.504221
207.314842
215.846502
217.223581
238.591705
238.478493
246.058718
264.208372
265.560685
282.264488
]
plot(times, "-;cdist;", times(1)*[1:length(times)]', "-;linear;")
title("Configuration duration (cdist-2.0.0-rc4)")
ylabel("Number of hosts")
xlabel("Time in seconds")
print('cdist-2.0.0-rc4.png', '-dpng')

3
doc/dev/logs/2011-09-13 Normal file
View file

@ -0,0 +1,3 @@
Name for installer:
installer (flag)
$__installer (variable) - gesetzt oder nicht

36
doc/dev/todo/TAKEME
View file

@ -5,26 +5,13 @@ Feel free to pick one!
CORE
----
- Inconsistent error messages if object is not existing!
-> always use "Object undefined"
- Add echo function / beautify output
__cdist_echo [level] [messages...]
level := syslog alike:
debug, notice, err
Include object_self prefixing, if given!
- Think about moving cdist-type-build-emulation out of cdist-manifest-run to
cdist-deploy-to: more dependency of cdist-manifest-run, but a lot of
less cycles consumed
- cdist-object-gencode: remove code if output empty?
- also take care of that in cdist-code-run!
- Remove cdist-object-push, covers only one line and is used only once:
[20:22] kr:bin% grep cdist-object-push *
cdist-object-run: cdist-object-push "$__cdist_target_host" "$__cdist_object"
[20:22] kr:bin%
- probably remove or improve cdist-type-template
- add $__tmp?
- allow cdist to run without $PATH setup: ./bin/cdist-deploy-to
- support non-ssh access?
USER INTERFACE
--------------
- add support $__tmp?
- for use in manifest, code, etc.?
- for creating temporary files, etc.
@ -36,18 +23,11 @@ CORE
-> for current host
-> add function to cdist-config, import from cdist-cache
- check all all internal variables are prefixed with __cdist
- Define / document "this is what should be on host X"
and have it parsable by different (shinken) tool
-> given after manifest run already!
- Allow types to have parameters without values (boolean flags).
e.g. __chair fancychair --pink --wood
would result in:
$__object/parameter/
pink # empty file
wood # empty file
- use absent/present for state by default?
TYPES
------
@ -64,5 +44,3 @@ DOCUMENTATION
- asciidoc interprets __, which we use for variables
names -> seek through docs and replace with \_\_!
- reference explorers in cdist-reference!
- compare running times:
one, 5, 10, 50, 100, 1000 hosts => how does cdist scale?

19
doc/dev/todo/niconext
View file

@ -1 +1,18 @@
Catch broken instances in cdist-mass-deploy -p and report broken deployements at the end!
- Support parallel execution
- and maximum number of parallel runs (-p X)
- error handling / report failed hosts
- Support different home instead of ../
- Bug: os.path.join() may be wrong for the remote side!
-> does not matter for now!
- Rewrite cdist-type-emulator
- Remove legacy code in cdist
- Remove cdist-config
- Remove man1/cdist-type-emulator.text
- Remove the PATH=... part from the README
- Replace bin/cdist-type-template
- Allow manifest to be read from stdin
- Create new video for cdist 2.0.0
http://www.youtube.com/watch?v=PRMjzy48eTI

12
doc/man/cdist-reference.text.sh
View file

@ -164,19 +164,19 @@ __manifest::
Available for: initial manifest
__global::
Directory that contains generic output like explorer.
Available for: initial manifest, type manifest, type explorer, type codegen
Available for: initial manifest, type manifest, type gencode
__object::
Directory that contains the current object.
Available for: type manifest, type explorer, type codegen
Available for: type manifest, type explorer, type gencode
__object_id::
The type unique object id.
Available for: type manifest, type explorer, type codegen
__self::
Available for: type manifest, type explorer, type gencode
__object_fq::
The full qualified name of the current object.
Available for: type manifest, type explorer, type codegen
Available for: type manifest, type explorer, type gencode
__target_host::
The host we are deploying to.
Available for: initial manifest, type manifest, type codegen
Available for: initial manifest, type manifest, type gencode
__target_user::
User to use for authentication on remote host.
Currently static in core.

31
doc/man/man1/cdist-cache.text
View file

@ -1,31 +0,0 @@
cdist-cache(1)
==============
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-cache - Cache output of last run
SYNOPSIS
--------
cdist-cache TARGET_HOST
DESCRIPTION
-----------
cdist-cache moves away the objects created during last run so the
next run can use the previous information and compare them with
the current status.
SEE ALSO
--------
cdist(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

34
doc/man/man1/cdist-code-run.text
View file

@ -1,34 +0,0 @@
cdist-code-run(1)
=================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-code-run - Run explorer remotely
SYNOPSIS
--------
cdist-code-run OBJECT_DIR OBJECT TYPE
DESCRIPTION
-----------
cdist-code-run executes generated code from a given OBJECT.
The OBJECT must be located below OBJECT_DIR.
TYPE must be either local or remote and determines which
code part is to be executed.
SEE ALSO
--------
- cdist(7)
- cdist-object-gencode(1)
- cdist-object-gencode-all(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

43
doc/man/man1/cdist-deploy-to.text
View file

@ -1,43 +0,0 @@
cdist-deploy-to(1)
==================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-deploy-to - Deploy configuration to host
SYNOPSIS
--------
cdist-deploy-to HOSTNAME
DESCRIPTION
-----------
Deploy configurations to the specified host, as configured in the initial
manifest. This script triggers the execution of several other scripts, in so
called stages. It is intented to run either from the command line or from cron.
ENVIRONMENT
-----------
If the environment variable **__cdist_conf_dir** is not set, the
configuration is read from <git-checkout>/conf. The local output directory can
be changed by the variable **__cdist_local_base_dir**. All environment
variables are handled by cdist-config.
SEE ALSO
--------
- cdist(7)
- cdist-config(1)
- cdist-mass-deploy(1)
- cdist-reference(7)
- cdist-stages(7)
COPYING
-------
Copyright \(C) 2010-2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

38
doc/man/man1/cdist-dir.text
View file

@ -1,38 +0,0 @@
cdist-dir(1)
============
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-dir - Poor man's directory synchronisation
SYNOPSIS
--------
cdist-dir <push|pull> TARGET_HOST SRC_DIR DST_DIR
DESCRIPTION
-----------
cdist-dir either pushes a local directory to the target host
or pulls a remote directory from a target host to the local host.
In the push case SRC_DIR is local, in the pull case remote.
In the push case DST_DIR is remote, in the pull case local.
cdist-dir does not cleanup DST_DIR and thus it may contain old
stuff if used multiple times.
cdist-dir does not rely on rsync or other high level tools, because
it cannot expect its existence on the local or target host.
SEE ALSO
--------
cdist(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

31
doc/man/man1/cdist-explorer-run-global.text
View file

@ -1,31 +0,0 @@
cdist-explorer-run-global(1)
============================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-explorer-run-global - Run the global explorers
SYNOPSIS
--------
cdist-explorer-run-global HOSTNAME
DESCRIPTION
-----------
Transfer the global explorers to HOSTNAME, execute them and transfer
back the results.
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

32
doc/man/man1/cdist-manifest-run-init.text
View file

@ -1,32 +0,0 @@
cdist-manifest-run-init(1)
==========================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-manifest-run-init - Run the initial manifest
SYNOPSIS
--------
cdist-manifest-run-init HOSTNAME
DESCRIPTION
-----------
cdist-manifest-run-init executes the initial manifest, which creates
the first objects.
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
- cdist-manifest-run-all(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

31
doc/man/man1/cdist-manifest-run.text
View file

@ -1,31 +0,0 @@
cdist-manifest-run(1)
=====================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-manifest-run - Run a given manifest
SYNOPSIS
--------
cdist-manifest-run HOSTNAME MANIFEST
DESCRIPTION
-----------
cdist-manifest-run executes the given MANIFEST.
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
- cdist-manifest-run-init(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

41
doc/man/man1/cdist-mass-deploy.text
View file

@ -1,41 +0,0 @@
cdist-mass-deploy(1)
====================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-mass-deploy - Deploy configuration to many hosts
SYNOPSIS
--------
cdist-mass-deploy [-p] HOSTNAME [HOSTNAME ...]
DESCRIPTION
-----------
cdist-mass-deploy is essentially a wrapper around cdist-deploy-to to
be able to deploy to many hosts on one command line.
EXAMPLES
--------
Deploy in parallel to all hosts specfied in the dsh group configuration ikr,
which is prefixed by "root@":
--------------------------------------------------------------------------------
cdist-mass-deploy -p $(cat ~/.dsh/group/ikr | sed 's/^root@//')
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
COPYING
-------
Copyright \(C) 2010-2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

31
doc/man/man1/cdist-object-all.text
View file

@ -1,31 +0,0 @@
cdist-object-all(1)
===================
Steven Armstrong <steven-cdist--@--armstrong.cc>
NAME
----
cdist-object-all - Run the given command on all objects
SYNOPSIS
--------
cdist-object-all HOSTNAME COMMAND
DESCRIPTION
-----------
Iterates over all defined objects and executes the given command on each
of them.
SEE ALSO
--------
- cdist(7)
- cdist-type(1)
COPYING
-------
Copyright \(C) 2011 Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

32
doc/man/man1/cdist-object-code-run.text
View file

@ -1,32 +0,0 @@
cdist-object-code-run(1)
========================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-code-run - Execute the generated code for a object
SYNOPSIS
--------
cdist-object-code-run HOSTNAME OBJECT
DESCRIPTION
-----------
Execute the local and remote code for the given object.
SEE ALSO
--------
- cdist(7)
- cdist-object-run(1)
- cdist-code-run(1)
- cdist-run-remote(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius, Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

31
doc/man/man1/cdist-object-explorer-run.text
View file

@ -1,31 +0,0 @@
cdist-object-explorer-run(1)
============================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-explorer-run - Run type explorers for a object
SYNOPSIS
--------
cdist-object-explorer-run HOSTNAME OBJECT
DESCRIPTION
-----------
Runs the explorers for the given object on the target host.
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
- cdist-remote-explorer-run(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius, Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

32
doc/man/man1/cdist-object-gencode-run.text
View file

@ -1,32 +0,0 @@
cdist-object-gencode-run(1)
===========================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-gencode-run - Generate code for a object
SYNOPSIS
--------
cdist-object-gencode-run HOSTNAME OBJECT
DESCRIPTION
-----------
For the given object, generate the code for local and remote execution.
SEE ALSO
--------
- cdist(7)
- cdist-code-run(1)
- cdist-object-run(1)
- cdist-object-gencode(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius, Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

33
doc/man/man1/cdist-object-gencode.text
View file

@ -1,33 +0,0 @@
cdist-object-gencode(1)
=======================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-gencode - Generate code for a given object
SYNOPSIS
--------
cdist-object-gencode HOSTNAME OBJECT
DESCRIPTION
-----------
For the given object, run the gencode executable. The output of this
executable on stdout will be used by cdist-object-gencode-all(1).
SEE ALSO
--------
- cdist(7)
- cdist-code-run(1)
- cdist-deploy-to(1)
- cdist-object-gencode-all(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

Some files were not shown because too many files have changed in this diff Show more