merge cdist/master

Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
This commit is contained in:
Steven Armstrong 2011-09-19 15:05:12 +02:00
commit cd5b0cc50c
116 changed files with 3513 additions and 2405 deletions

View file

@ -1,4 +1,12 @@
1.7.1:
2.0.1:
* Bugfix cdist: Always print source of error in case of exec errors
2.0.0: 2011-09-16
* New Type: __package_rubygem (Chase Allen James)
* __self replaced by __object_fq
* Rewrote cdist in Python
1.7.1: 2011-07-26
* Documentation: Add explorers to reference
* Documentation: Typo cleanup (Derek Brost)
* Type __key_value: Bugfix (Steven Armstrong)

View file

@ -0,0 +1,12 @@
#!/bin/sh
outfile="$1"; shift
(
for host in "$@"; do
hosts="$hosts $host"
cdist config -c ~/p/cdist-nutzung -p $hosts 2>&1
done
) | tee "$outfile"
echo "----------"
grep 'INFO: Total processing time for' "$outfile" | sed 's/.*: //'

24
doc/dev/lastchanges Executable file
View file

@ -0,0 +1,24 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Show the lastest changes from this (upcoming) release
#
awk '/^$/ { exit } { print $0 } END { print "-----" }' < "${0%/*}/../changelog"

View file

@ -0,0 +1,56 @@
% x200
data = [ 1 104; 2 129; 6 249 ; 25 1267 ]
% x201, 4.2.2.4 dns, eth
data = [ 1 143; 2 159; 3 198; 4 244; 5 299; 6 350; 7 435; 8 429 ];
% x201, von zuhause aus
data2 = [ 226 242 275 296 306 357 403 400 409 685 617 672 ]
plot(0:size(data)(2)-1, data)
hold off;
plot(data(:,1), data(:,2))
% per host time:
data(:,2)' ./ data(:,1)'
hold on;
plot(data(:,1), data(:,2)' ./ data(:,1)')
Testing on
Intel(R) Core(TM)2 Duo CPU P8400 @ 2.26GHz
4 GiB RAM
Intel Gbit Nic
- 169 objects, all done (i.e. rerun)
- runs are cpu bound
1 host:
core: cdist 1.6.2: Successfully finished run
Run: 104s (1 minutes)
2 hosts:
[13:48] kr:cdist-nutzung% mytime cdist-mass-deploy -p ikq03.ethz.ch ikq04.ethz.ch
Run: 129s (2 minutes)
6 hosts: (7 specified, but ikq01.ethz.ch is dead)
cdist-mass-deploy -p ikq01.ethz.ch ikq02.ethz.ch ikq03.ethz.ch ikq04.ethz.ch ikq05.ethz.ch ikq06.ethz.ch ikq07.ethz.ch
Run: 294s (4 minutes)
25 hosts
[13:58] kr:cdist-nutzung% mytime cdist-mass-deploy -p $(sed 's/^root@//' ~/.dsh/group/ikr)
Run: 723s (12 minutes)
25 hosts without proxy command now:
Run: 1267s (21 minutes)
Run: 1212s (20 minutes)
[14:11] kr:cdist-nutzung% mytime cdist-mass-deploy -p $(sed 's/^root@//' ~/.dsh/group/ikr)

View file

@ -0,0 +1,9 @@
Run: 78s (1 minutes)
[13:25] kr:cdist-nutzung% mytime cdist-deploy-to ikq02.ethz.ch
% mit dash, eth, x201, Mit Apr 27 13:41:49 CEST 2011
data = [ 0 73 77 89 107 130 151 180 197 228 251 260 199 295 335 276 ]
plot(0:size(data2)(2)-1, data2)

View file

@ -0,0 +1,20 @@
- new executable cinst-deploy-to
- types used by cinst are marked as such ('cinst-only')
- cdist-deploy-to and cinst-deploy-to read the same manifest
- cdist ignores types marked as 'cinst-only'
- cinst ignores types not marked as 'cinst-only'
- update $__explorer/os to recognize preos
- cinst types will ONLY BE CALLED if $__explorer/os == 'preos'
--------------------------------------------------------------------------------
cinst types:
__partition_msdos /dev/sda1 --type 83 --size 100M --bootable
__partition_msdos /dev/sda2 --type 82 --size 512M
__fs_jfs /dev/sda1 --args "-c -q"
__fstab_entry /dev/sda1 --type jfs --mountpoint / --options noatime --freq 0 --passno 0
__fstab_entry /dev/sda2 --type swap

View file

@ -0,0 +1,4 @@
Debugging cdist:
[0:13] kr:cdist-nutzung% ./local/update-local-core && __cdist_debug=1 __cdist_local_base_dir=/tmp/cdist cdist-deploy-to ikq04.ethz.ch

110
doc/dev/logs/2011-09-12 Normal file
View file

@ -0,0 +1,110 @@
Benchmark from home/X201 (2 cores, 4 threads):
ikq (1): 72.192397
ikq (2): INFO: Total processing time for 2 hosts: 74.1845
ikq* (6): INFO: Total processing time: 117.572312
ikq* + ikr3 (9): INFO: Total processing time: 120.307662
ikq* + ikr (14): INFO: Total processing time: 139.769807
ikq* + ikr (18): INFO: Total processing time: 186.354398
ikq* + ikr (22): INFO: Total processing time: 225.793533
ikq* + ikr (26): INFO: Total processing time: 237.06687
ikq* + ikr (31): INFO: Total processing time: 276.912414
ikr07.ethz.ch ikr09.ethz.ch ikr10.ethz.ch ikr11.ethz.ch
ikr13.ethz.ch ikr14.ethz.ch ikr15.ethz.ch ikr16.ethz.ch
ikr17.ethz.ch ikr19.ethz.ch ikr20.ethz.ch ikr21.ethz.ch
ikr23.ethz.ch ikr24.ethz.ch ikr25.ethz.ch ikr26.ethz.ch
ikr27.ethz.ch ikr28.ethz.ch ikr29.ethz.ch ikr30.ethz.ch
ikr31.ethz.ch
cdist -c ~/p/cdist-nutzung -p ikq02.ethz.ch ikq03.ethz.ch ikq04.ethz.ch ikq05.ethz.ch ikq06.ethz.ch ikq07.ethz.ch ikr01.ethz.ch ikr02.ethz.ch ikr03.ethz.ch ikr05.ethz.ch ikr07.ethz.ch ikr09.ethz.ch ikr10.ethz.ch ikr11.ethz.ch ikr13.ethz.ch ikr14.ethz.ch ikr15.ethz.ch ikr16.ethz.ch ikr17.ethz.ch ikr19.ethz.ch ikr20.ethz.ch ikr21.ethz.ch
--------------------------------------------------------------------------------
INFO: Total processing time for 1 hosts: 72.166661
INFO: Total processing time for 2 hosts: 76.633228
INFO: Total processing time for 3 host(s): 77.199817
INFO: Total processing time for 4 host(s): 94.045175
INFO: Total processing time for 5 host(s): 103.226354
INFO: Total processing time for 6 host(s): 107.76097
INFO: Total processing time for 7 host(s): 101.571705
INFO: Total processing time for 8 host(s): 107.600093
INFO: Total processing time for 9 host(s): 116.500371
INFO: Total processing time for 10 host(s): 119.445805
INFO: Total processing time for 11 host(s): 123.944385
INFO: Total processing time for 12 host(s): 130.499098
INFO: Total processing time for 13 host(s): 137.250861
INFO: Total processing time for 14 host(s): 154.9841
INFO: Total processing time for 15 host(s): 139.659637
INFO: Total processing time for 16 host(s): 142.70005
INFO: Total processing time for 17 host(s): 148.541452
INFO: Total processing time for 18 host(s): 159.360809
INFO: Total processing time for 19 host(s): 171.907864
INFO: Total processing time for 20 host(s): 178.76695
INFO: Total processing time for 21 host(s): 183.856671
INFO: Total processing time for 22 host(s): 194.504221
INFO: Total processing time for 23 host(s): 207.314842
INFO: Total processing time for 24 host(s): 215.846502
INFO: Total processing time for 25 host(s): 217.223581
INFO: Total processing time for 26 host(s): 238.591705
INFO: Total processing time for 27 host(s): 238.478493
INFO: Total processing time for 28 host(s): 246.058718
INFO: Total processing time for 29 host(s): 264.208372
INFO: Total processing time for 30 host(s): 265.560685
INFO: Total processing time for 31 host(s): 282.264488
--------------------------------------------------------------------------------
Use:
grep "^INFO: Total processing time" doc/dev/logs/2011-09-12 | sed 's/.*: //'
octave
times = [ /* paste here ]
plot(times)
# keep the graph
hold on
# Scale linearly with the single host value
plot((1:31)*times(1))
--------------------------------------------------------------------------------
code:
octave
time = [
72.166661
76.633228
77.199817
94.045175
103.226354
107.76097
101.571705
107.600093
116.500371
119.445805
123.944385
130.499098
137.250861
154.9841
139.659637
142.70005
148.541452
159.360809
171.907864
178.76695
183.856671
194.504221
207.314842
215.846502
217.223581
238.591705
238.478493
246.058718
264.208372
265.560685
282.264488
]
plot(times, "-;cdist;", times(1)*[1:length(times)]', "-;linear;")
title("Configuration duration (cdist-2.0.0-rc4)")
ylabel("Number of hosts")
xlabel("Time in seconds")
print('cdist-2.0.0-rc4.png', '-dpng')

3
doc/dev/logs/2011-09-13 Normal file
View file

@ -0,0 +1,3 @@
Name for installer:
installer (flag)
$__installer (variable) - gesetzt oder nicht

View file

@ -0,0 +1,63 @@
48 core
model name : AMD Opteron(tm) Processor 6174
128 GB RAM
Dell PowerEdge R815
root@sgs-r815-01:~/cdist-nutzung# grep "Total processing time for" deploy-all-benchmark-noikr13
INFO: Total processing time for 1 host(s): 257.641541
INFO: Total processing time for 2 host(s): 257.025783
INFO: Total processing time for 3 host(s): 258.933088
INFO: Total processing time for 4 host(s): 259.253074
INFO: Total processing time for 5 host(s): 260.331896
INFO: Total processing time for 6 host(s): 262.051349
INFO: Total processing time for 7 host(s): 323.820878
INFO: Total processing time for 8 host(s): 329.081856
INFO: Total processing time for 9 host(s): 333.346278
INFO: Total processing time for 10 host(s): 334.832419
INFO: Total processing time for 11 host(s): 330.572375
INFO: Total processing time for 12 host(s): 331.726628
INFO: Total processing time for 13 host(s): 331.740591
INFO: Total processing time for 14 host(s): 331.237139
INFO: Total processing time for 15 host(s): 331.718861
INFO: Total processing time for 16 host(s): 332.374645
INFO: Total processing time for 17 host(s): 331.510445
INFO: Total processing time for 18 host(s): 332.030743
INFO: Total processing time for 19 host(s): 332.193198
INFO: Total processing time for 20 host(s): 333.933765
INFO: Total processing time for 21 host(s): 335.292953
INFO: Total processing time for 22 host(s): 337.253608
INFO: Total processing time for 23 host(s): 337.831493
INFO: Total processing time for 24 host(s): 339.024737
INFO: Total processing time for 25 host(s): 343.515044
INFO: Total processing time for 26 host(s): 339.759678
INFO: Total processing time for 27 host(s): 339.378998
INFO: Total processing time for 28 host(s): 339.640378
INFO: Total processing time for 29 host(s): 340.885614
INFO: Total processing time for 30 host(s): 341.836923
INFO: Total processing time for 31 host(s): 343.825758
INFO: Total processing time for 32 host(s): 344.176089
INFO: Total processing time for 33 host(s): 345.408518
INFO: Total processing time for 34 host(s): 347.15322
INFO: Total processing time for 35 host(s): 351.330649
INFO: Total processing time for 36 host(s): 347.640758
INFO: Total processing time for 37 host(s): 347.381126
INFO: Total processing time for 38 host(s): 347.053406
INFO: Total processing time for 39 host(s): 347.453166
INFO: Total processing time for 40 host(s): 347.84804
INFO: Total processing time for 41 host(s): 349.035272
INFO: Total processing time for 42 host(s): 349.41507
INFO: Total processing time for 43 host(s): 351.208072
INFO: Total processing time for 44 host(s): 351.788401
INFO: Total processing time for 45 host(s): 351.730259
INFO: Total processing time for 46 host(s): 515.693497
INFO: Total processing time for 47 host(s): 352.702677
INFO: Total processing time for 48 host(s): 353.418003
INFO: Total processing time for 49 host(s): 355.07111
INFO: Total processing time for 50 host(s): 354.622388
INFO: Total processing time for 51 host(s): 355.192521
INFO: Total processing time for 52 host(s): 355.283238
INFO: Total processing time for 53 host(s): 358.112329
INFO: Total processing time for 54 host(s): 357.717426
INFO: Total processing time for 55 host(s): 357.748707
INFO: Total processing time for 56 host(s): 358.902118
INFO: Total processing time for 57 host(s): 367.817594

File diff suppressed because it is too large Load diff

View file

@ -5,26 +5,13 @@ Feel free to pick one!
CORE
----
- Inconsistent error messages if object is not existing!
-> always use "Object undefined"
- Add echo function / beautify output
__cdist_echo [level] [messages...]
level := syslog alike:
debug, notice, err
Include object_self prefixing, if given!
- Think about moving cdist-type-build-emulation out of cdist-manifest-run to
cdist-deploy-to: more dependency of cdist-manifest-run, but a lot of
less cycles consumed
- cdist-object-gencode: remove code if output empty?
- also take care of that in cdist-code-run!
- Remove cdist-object-push, covers only one line and is used only once:
[20:22] kr:bin% grep cdist-object-push *
cdist-object-run: cdist-object-push "$__cdist_target_host" "$__cdist_object"
[20:22] kr:bin%
- probably remove or improve cdist-type-template
- add $__tmp?
- allow cdist to run without $PATH setup: ./bin/cdist-deploy-to
- support non-ssh access?
USER INTERFACE
--------------
- add support $__tmp?
- for use in manifest, code, etc.?
- for creating temporary files, etc.
@ -36,18 +23,11 @@ CORE
-> for current host
-> add function to cdist-config, import from cdist-cache
- check all all internal variables are prefixed with __cdist
- Define / document "this is what should be on host X"
and have it parsable by different (shinken) tool
-> given after manifest run already!
- Allow types to have parameters without values (boolean flags).
e.g. __chair fancychair --pink --wood
would result in:
$__object/parameter/
pink # empty file
wood # empty file
- use absent/present for state by default?
TYPES
------
@ -64,5 +44,3 @@ DOCUMENTATION
- asciidoc interprets __, which we use for variables
names -> seek through docs and replace with \_\_!
- reference explorers in cdist-reference!
- compare running times:
one, 5, 10, 50, 100, 1000 hosts => how does cdist scale?

View file

@ -1 +1,43 @@
Catch broken instances in cdist-mass-deploy -p and report broken deployements at the end!
2.0.1:
- Rewrite cdist-type-emulator
- Remove legacy code in cdist
- Remove cdist-config
- Remove man1/cdist-type-emulator.text
- Remove the PATH=... part from the README
- how to access output dir?
Test:
__cdist_type_base_dir=$(pwd -P)/conf/type __file
- Fix / rewrite cdist-quickstart
--------------------------------------------------------------------------------
- Initial install support
- setup $__install = "yes" for
manifest(s), gencode-*
- run standard manifest (?)
- creates initial objects
- only those having the installer flag?
- requires changegs to cdist-type-emulator!
- Goto Rewrite cdist-type-emulator
- run all other manifests
- creates all objects
- what about type explorer?
- do not run, create empty output (types should be able
to handle this!)
via __global/
- Support parallel execution
- and maximum number of parallel runs (-p X)
- error handling / report failed hosts
- Allow manifest to be read from stdin
- Create new video for cdist 2.0.0
http://www.youtube.com/watch?v=PRMjzy48eTI
- Setup __debug, if -d is given, so other tools can reuse it

View file

@ -164,25 +164,25 @@ __manifest::
Available for: initial manifest
__global::
Directory that contains generic output like explorer.
Available for: initial manifest, type manifest, type explorer, type codegen
Available for: initial manifest, type manifest, type gencode
__object::
Directory that contains the current object.
Available for: type manifest, type explorer, type codegen
Available for: type manifest, type explorer, type gencode
__object_id::
The type unique object id.
Available for: type manifest, type explorer, type codegen
__self::
Available for: type manifest, type explorer, type gencode
__object_fq::
The full qualified name of the current object.
Available for: type manifest, type explorer, type codegen
Available for: type manifest, type explorer, type gencode
__target_host::
The host we are deploying to.
Available for: initial manifest, type manifest, type codegen
Available for: initial manifest, type manifest, type gencode
__target_user::
User to use for authentication on remote host.
Currently static in core.
__type::
Path to the current type.
Available for: type manifest
Available for: type manifest, type gencode
__type_explorer::
Directory that contains the type explorers.
Available for: type explorer

View file

@ -1,31 +0,0 @@
cdist-cache(1)
==============
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-cache - Cache output of last run
SYNOPSIS
--------
cdist-cache TARGET_HOST
DESCRIPTION
-----------
cdist-cache moves away the objects created during last run so the
next run can use the previous information and compare them with
the current status.
SEE ALSO
--------
cdist(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,34 +0,0 @@
cdist-code-run(1)
=================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-code-run - Run explorer remotely
SYNOPSIS
--------
cdist-code-run OBJECT_DIR OBJECT TYPE
DESCRIPTION
-----------
cdist-code-run executes generated code from a given OBJECT.
The OBJECT must be located below OBJECT_DIR.
TYPE must be either local or remote and determines which
code part is to be executed.
SEE ALSO
--------
- cdist(7)
- cdist-object-gencode(1)
- cdist-object-gencode-all(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,43 +0,0 @@
cdist-deploy-to(1)
==================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-deploy-to - Deploy configuration to host
SYNOPSIS
--------
cdist-deploy-to HOSTNAME
DESCRIPTION
-----------
Deploy configurations to the specified host, as configured in the initial
manifest. This script triggers the execution of several other scripts, in so
called stages. It is intented to run either from the command line or from cron.
ENVIRONMENT
-----------
If the environment variable **__cdist_conf_dir** is not set, the
configuration is read from <git-checkout>/conf. The local output directory can
be changed by the variable **__cdist_local_base_dir**. All environment
variables are handled by cdist-config.
SEE ALSO
--------
- cdist(7)
- cdist-config(1)
- cdist-mass-deploy(1)
- cdist-reference(7)
- cdist-stages(7)
COPYING
-------
Copyright \(C) 2010-2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,38 +0,0 @@
cdist-dir(1)
============
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-dir - Poor man's directory synchronisation
SYNOPSIS
--------
cdist-dir <push|pull> TARGET_HOST SRC_DIR DST_DIR
DESCRIPTION
-----------
cdist-dir either pushes a local directory to the target host
or pulls a remote directory from a target host to the local host.
In the push case SRC_DIR is local, in the pull case remote.
In the push case DST_DIR is remote, in the pull case local.
cdist-dir does not cleanup DST_DIR and thus it may contain old
stuff if used multiple times.
cdist-dir does not rely on rsync or other high level tools, because
it cannot expect its existence on the local or target host.
SEE ALSO
--------
cdist(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,31 +0,0 @@
cdist-explorer-run-global(1)
============================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-explorer-run-global - Run the global explorers
SYNOPSIS
--------
cdist-explorer-run-global HOSTNAME
DESCRIPTION
-----------
Transfer the global explorers to HOSTNAME, execute them and transfer
back the results.
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,32 +0,0 @@
cdist-manifest-run-init(1)
==========================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-manifest-run-init - Run the initial manifest
SYNOPSIS
--------
cdist-manifest-run-init HOSTNAME
DESCRIPTION
-----------
cdist-manifest-run-init executes the initial manifest, which creates
the first objects.
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
- cdist-manifest-run-all(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,31 +0,0 @@
cdist-manifest-run(1)
=====================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-manifest-run - Run a given manifest
SYNOPSIS
--------
cdist-manifest-run HOSTNAME MANIFEST
DESCRIPTION
-----------
cdist-manifest-run executes the given MANIFEST.
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
- cdist-manifest-run-init(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,41 +0,0 @@
cdist-mass-deploy(1)
====================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-mass-deploy - Deploy configuration to many hosts
SYNOPSIS
--------
cdist-mass-deploy [-p] HOSTNAME [HOSTNAME ...]
DESCRIPTION
-----------
cdist-mass-deploy is essentially a wrapper around cdist-deploy-to to
be able to deploy to many hosts on one command line.
EXAMPLES
--------
Deploy in parallel to all hosts specfied in the dsh group configuration ikr,
which is prefixed by "root@":
--------------------------------------------------------------------------------
cdist-mass-deploy -p $(cat ~/.dsh/group/ikr | sed 's/^root@//')
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
COPYING
-------
Copyright \(C) 2010-2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,31 +0,0 @@
cdist-object-all(1)
===================
Steven Armstrong <steven-cdist--@--armstrong.cc>
NAME
----
cdist-object-all - Run the given command on all objects
SYNOPSIS
--------
cdist-object-all HOSTNAME COMMAND
DESCRIPTION
-----------
Iterates over all defined objects and executes the given command on each
of them.
SEE ALSO
--------
- cdist(7)
- cdist-type(1)
COPYING
-------
Copyright \(C) 2011 Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,32 +0,0 @@
cdist-object-code-run(1)
========================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-code-run - Execute the generated code for a object
SYNOPSIS
--------
cdist-object-code-run HOSTNAME OBJECT
DESCRIPTION
-----------
Execute the local and remote code for the given object.
SEE ALSO
--------
- cdist(7)
- cdist-object-run(1)
- cdist-code-run(1)
- cdist-run-remote(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius, Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,31 +0,0 @@
cdist-object-explorer-run(1)
============================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-explorer-run - Run type explorers for a object
SYNOPSIS
--------
cdist-object-explorer-run HOSTNAME OBJECT
DESCRIPTION
-----------
Runs the explorers for the given object on the target host.
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
- cdist-remote-explorer-run(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius, Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,32 +0,0 @@
cdist-object-gencode-run(1)
===========================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-gencode-run - Generate code for a object
SYNOPSIS
--------
cdist-object-gencode-run HOSTNAME OBJECT
DESCRIPTION
-----------
For the given object, generate the code for local and remote execution.
SEE ALSO
--------
- cdist(7)
- cdist-code-run(1)
- cdist-object-run(1)
- cdist-object-gencode(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius, Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,33 +0,0 @@
cdist-object-gencode(1)
=======================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-gencode - Generate code for a given object
SYNOPSIS
--------
cdist-object-gencode HOSTNAME OBJECT
DESCRIPTION
-----------
For the given object, run the gencode executable. The output of this
executable on stdout will be used by cdist-object-gencode-all(1).
SEE ALSO
--------
- cdist(7)
- cdist-code-run(1)
- cdist-deploy-to(1)
- cdist-object-gencode-all(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,31 +0,0 @@
cdist-object-manifest-run(1)
============================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-manifest-run - Run an objects manifest
SYNOPSIS
--------
cdist-object-manifest-run HOSTNAME OBJECT
DESCRIPTION
-----------
Run the manifest for the given object.
SEE ALSO
--------
- cdist(7)
- cdist-deploy-to(1)
- cdist-manifest-run(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius, Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,35 +0,0 @@
cdist-object-prepare(1)
=======================
Steven Armstrong <steven-cdist--@--armstrong.cc>
NAME
----
cdist-object-prepare - Prepare an object
SYNOPSIS
--------
cdist-object-prepare HOSTNAME OBJECT
DESCRIPTION
-----------
Prepare the given object by running it through stage 3 (object information
retrieval) and stage 4 (run the object manifest).
See related man pages for details.
SEE ALSO
--------
- cdist(7)
- cdist-stages(7)
- cdist-object-explorer-run(1)
- cdist-object-manifest-run(1)
- cdist-type(1)
COPYING
-------
Copyright \(C) 2011 Steven Armstrong. Free use of this software is granted
under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,31 +0,0 @@
cdist-object-push(1)
====================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-object-push - Transfer a object to the target host
SYNOPSIS
--------
cdist-object-push HOSTNAME OBJECT
DESCRIPTION
-----------
Transfers the given object to the target host.
SEE ALSO
--------
- cdist(7)
- cdist-object-run(1)
- cdist-type(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius, Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,36 +0,0 @@
cdist-object-run(1)
===================
Steven Armstrong <steven-cdist--@--armstrong.cc>
NAME
----
cdist-object-run - Run an object
SYNOPSIS
--------
cdist-object-run HOSTNAME OBJECT
DESCRIPTION
-----------
Applies the given object on the target host by running it through stage 5
(code generation) and stage 6 (code execution).
See related man pages for details.
SEE ALSO
--------
- cdist(7)
- cdist-stages(7)
- cdist-object-gencode-run(1)
- cdist-object-push(1)
- cdist-object-code-run(1)
- cdist-type(1)
COPYING
-------
Copyright \(C) 2011 Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,33 +0,0 @@
cdist-remote-explorer-run(1)
============================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-remote-explorer-run - Run explorer remotely
SYNOPSIS
--------
cdist-remote-explorer-run VARIABLE_NAME EXPLORER_DIR OUT_DIR
DESCRIPTION
-----------
cdist-remote-explorer-run is executed on the target.
It sets up the variable VARIABLE_NAME to point to the given
EXPLORER_DIR and runs all explorer found in EXPLORER_DIR.
The output of every run explorer is saved into OUT_DIR.
SEE ALSO
--------
- cdist(7)
- cdist-explorer-run-global(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,33 +0,0 @@
cdist-run-remote(1)
===================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-run-remote - Execute something on the target
SYNOPSIS
--------
cdist-run-remote HOSTNAME EXECUTABLE [ARGUMENTS FOR EXECUTABLE]
DESCRIPTION
-----------
cdist-run-remote runs the given executable on the remote host.
It ensures PATH is setup correctly on the target side.
SEE ALSO
--------
- cdist(7)
- cdist-object-code-run(1)
- cdist-deploy-to(1)
- cdist-remote-code-run-all(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,33 +0,0 @@
cdist-type-build-emulation(1)
=============================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-type-build-emulation - Build executables for types
SYNOPSIS
--------
cdist-type-build-emulation OUT_DIR
DESCRIPTION
-----------
cdist-type-build-emulation creates a link to cdist-type-emulator
for every TYPE. These links are placed in a OUT_DIR, which
is prepended into $PATH. This way the user can use TYPE in the
manifests like any other executable.
SEE ALSO
--------
- cdist(7)
- cdist-type-emulator(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,30 +0,0 @@
cdist-type-template(1)
======================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-type-template - Create a new type
SYNOPSIS
--------
cdist-type-template NAME
DESCRIPTION
-----------
cdist-type-template creates a new type and adds the usual files to it.
It is thought to be helpful when writing new types.
SEE ALSO
--------
cdist(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

103
doc/man/man1/cdist.text Normal file
View file

@ -0,0 +1,103 @@
cdist(1)
========
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist - Configuration management
SYNOPSIS
--------
cdist [-h] [-V]
cdist banner
cdist config [-h] [-d] [-V] [-c CDIST_HOME] [-i MANIFEST] [-p] [-s] host [host ...]
DESCRIPTION
-----------
cdist is the frontend executable to the cdist configuration management.
cdist supports different as explained below. The options to the main
program are:
-h, --help::
Show the help screen
-V, --version::
Show version and exit
BANNER
-------
Displays the cdist banner.
CONFIG
------
Configure a system
-h, --help::
Show the help screen
-c CDIST_HOME, --cdist-home CDIST_HOME::
Instead of using the parent of the bin directory as cdist home,
use the specified directory
-d, --debug::
Enable debug output
-i MANIFEST, --initial-manifest MANIFEST::
Path to a cdist manifest or - to read from stdin
-p, --parallel::
Operate on multiple hosts in parallel
-s, --sequential::
Operate on multiple hosts sequentially
EXAMPLES
--------
--------------------------------------------------------------------------------
# Configure ikq05.ethz.ch with debug enabled
cdist config -d ikq05.ethz.ch
# Configure hosts in parallel and use a different home directory
cdist config -c ~/p/cdist-nutzung \
-p ikq02.ethz.ch ikq03.ethz.ch ikq04.ethz.ch
# Display banner
cdist banner
# Show help
cdist --help
# Show Version
cdist --version
--------------------------------------------------------------------------------
ENVIRONMENT
-----------
TMPDIR, TEMP, TMP::
Setup the base directory for the temporary directory.
See http://docs.python.org/py3k/library/tempfile.html for
more information. This is rather useful, if the standard
directory used does not allow executables.
SEE ALSO
--------
- cdist(7)
- cdist-type-emulator(1)
- cdist-reference(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -0,0 +1,310 @@
#!/bin/sh
#
# 2010-2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Give the user an introduction into cdist
#
. cdist-config
set -eu
banner="cdist-quickstart>"
continue="Press enter to continue or ctrl-c to abort."
create_continue="Press enter to create the described files/directories"
__prompt()
{
echo -n "$banner" "$@"
read answer
}
################################################################################
# Intro of quickstart
#
cat << eof
$banner cdist version $__cdist_version
Welcome to the interactive guide to cdist!
This is the interactive tutorial and beginners help for cdist and here's
our schedule:
- Stages: How cdist operates
- Explorer: Explore facts of the target host
- Manifest: Map configurations to hosts
- Types: Bundled functionality
- Deploy a configuration to the local host!
eof
__prompt "$continue"
################################################################################
# Stages
#
cat << eof
To deploy configurations to a host, you call
cdist-deploy-to <hostname>
which makes calls to other scripts, which realise the so called "stages".
Usually you'll not notice this, but in case you want to debug or hack cdist,
you can run each stage on its own. Besides that, you just need to remember
that the command cdist-deploy-to is the main cdist command.
See also:
Source of cdist-deploy-to(1), cdist-stages(7)
eof
__prompt "$continue"
################################################################################
# Explorer
#
cat << eof
The first thing cdist always does is running different explorers on the
target host. The explorers can be found in the directory
${__cdist_explorer_dir}
An explorer is executed on the target host and its output is saved to a file.
You can use these files later to decide what or how to configure the host.
For a demonstration, we'll call the OS explorer locally now, but remember:
This is only for demonstration, normally it is run on the target host.
The os explorer will which either displays the detected operating system or
nothing if it does not know your OS.
See also:
cdist-explorer(7)
eof
explorer="${__cdist_explorer_dir}/os"
__prompt "Press enter to execute $explorer"
set -x
"$explorer"
set +x
################################################################################
# Manifest
#
cat << eof
The initial manifest is the entry point for cdist to find out, what you would
like to have configured. It is located at
${__cdist_manifest_init}
And can be as simple as
--------------------------------------------------------------------------------
__file /etc/cdist-configured --type file
--------------------------------------------------------------------------------
See also:
cdist-manifest(7)
eof
__prompt "$continue"
cat << eof
Let's take a deeper look at the initial manifest to understand what it means:
__file /etc/cdist-configured --type file
| | | \\
| | The parameter type \\ With the value file
| |
| |
| | This is the object id
|
__file is a so called "type"
This essentially looks like a standard command executed in the shell.
eof
__prompt "$continue"
cat << eof
And that's exactly true. Manifests are shell snippets that can use
types as commands with arguments. cdist prepends a special path
that contain links to the cdist-type-emulator, to \$PATH, so you
can use your types as a command.
This is also the reason why types should always be prefixed with
"__", to prevent collisions with existing binaries.
The object id is unique per type and used to prevent you from creating
the same object twice.
Parameters are type specific and are always specified as --parameter <value>.
See also:
cdist-type-build-emulation(1), cdist-type-emulator(1)
eof
__prompt "$continue"
################################################################################
# Types
#
cat << eof
Types are bundled functionality and are the main component of cdist.
If you want to have a feature x, you write the type __x. Types are stored in
${__cdist_type_dir}
And cdist ships with some types already!
See also:
cdist-type(7)
eof
__prompt "Press enter to see available types"
set -x
ls ${__cdist_type_dir}
set +x
cat << eof
Types consist of the following parts:
- ${__cdist_name_parameter} (${__cdist_name_parameter_required}/${__cdist_name_parameter_optional}
- ${__cdist_name_manifest}
- ${__cdist_name_explorer}
- ${__cdist_name_gencode}
eof
__prompt "$continue"
cat << eof
Every type must have a directory named ${__cdist_name_parameter}, which
contains required or optional parameters (in newline seperated files).
If an object of a specific type was created in the initial manifest,
the manifest of the type is run and may create other objects.
A type may have ${__cdist_name_explorer}, which are very similar to the
${__cdist_name_explorer} seen above, but with a different purpose:
They are specific to the type and are not relevant for other types.
You may use them for instance to find out details on the target host,
so you can decide what to do on the target host eventually.
After the ${__cdist_name_manifest} and the ${__cdist_name_explorer} of
a type have been run, ${__cdist_name_gencode} is executed, which creates
code to be executed on the target on stdout.
eof
__prompt "$continue"
################################################################################
# Deployment
#
cat << eof
Now you've got some basic knowledge about cdist, let's configure your a host!
Ensure that you have a ssh server running on the host and that you can login as root.
eof
__prompt "Enter hostname or press enter for localhost: "
if [ "$answer" ]; then
host="$answer"
else
host="localhost"
fi
manifestinit="conf/manifest/init"
cat << eof
I'll now setup $manifestinit, containing the following code:
--------------------------------------------------------------------------------
# Every machine becomes a marker, so sysadmins know that automatic
# configurations are happening
__file /etc/cdist-configured
case "\$__target_host" in
$host)
__link /tmp/cdist-testfile --source /etc/cdist-configured --type symbolic
__addifnosuchline /tmp/cdist-welcome --line "Welcome to cdist"
;;
esac
--------------------------------------------------------------------------------
WARNING: This will overwrite ${manifestinit}.
eof
cat > "$__cdist_abs_mydir/../$manifestinit" << eof
# Every machine becomes a marker, so sysadmins know that automatic
# configurations are happening
__file /etc/cdist-configured
case "\$__target_host" in
$host)
__link /tmp/cdist-testfile --source /etc/cdist-configured --type symbolic
__addifnosuchline /tmp/cdist-welcome --line "Welcome to cdist"
;;
esac
eof
chmod u+x "$__cdist_abs_mydir/../$manifestinit"
cmd="cdist-deploy-to $host"
__prompt "Press enter to run \"$cmd\""
# No quotes, we need field splitting
$cmd
################################################################################
# End
#
cat << eof
--------------------------------------------------------------------------------
That's it, this is the end of the cdist-quickstart.
I hope you've got some impression on how cdist works, here are again some
pointers on where to continue to read:
cdist(7), cdist-deploy-to(1), cdist-type(7), cdist-stages(7)
eof

View file

@ -0,0 +1,242 @@
% first presentation about cmtp
\pdfminorversion=4
%\documentclass[ucs]{beamer}
\documentclass{beamer}
%\documentclass[utf8]{beamer}
\usepackage[utf8]{inputenc}
\usepackage{german}
\usepackage{graphicx}
\usepackage{beamerthemesplit}
\setbeamercovered{dynamic}
\usetheme{Malmoe}
\usecolortheme{crane}
\title{cdist - nutzbare Konfigurationsverwaltung}
\subtitle{Cosin 2011}
\author{Nico -telmich- Schottelius}
\date{25. Juni 2011}
\begin{document}
\frame{\titlepage}
%\section[Outline]{}
\frame{\tableofcontents}
\section{Einleitung}
\frame
{
\frametitle{Was ist das Problem?}
\begin{itemize}
\item Einmal konfigurieren = toll
\item Zweimal konfigurieren = naja, ...
\item Neue Sachen machen mehr Spass als alte wiederholen
\item Viele Rechner = viel Mühe?
\end{itemize}
}
\frame
{
\frametitle{Das ist nicht neu...}
\begin{itemize}
\item cfengine
\item Puppet
\item bcfg2
\item chef
\item ...
\end{itemize}
}
\frame
{
\frametitle{Warum cdist?}
\begin{itemize}
\item Klein
\item Unix
\item Leicht zu bedienen
\item ... zu erweitern
\item Shell
\item Weil es Spaß macht!
\end{itemize}
}
\section{Installieren}
\frame
{
\frametitle{Vorraussetzungen}
\begin{itemize}
\item sshd
\item root login via sshd
\item Besser: ssh-pubkey konfiguriert (PermitRootLogin without-password)
\item git
\item Asciidoc für dia manpages
\end{itemize}
}
\frame
{
\frametitle{Installation}
\begin{center}
git clone git://git.schottelius.org/cdist
\end{center}
}
\begin{frame}[fragile]
\frametitle{Erstellen der Manpages}
\begin{verbatim}
# Braucht asciidoc / a2x
./build.sh man
\end{verbatim}
\end{frame}
\section{Nutzen}
\begin{frame}[fragile]
\frametitle{Vorbereitung PATH und MANPATH}
\begin{verbatim}
cd cdist
eval `./bin/cdist-env`
echo $PATH
echo $MANPATH
\end{verbatim}
\end{frame}
\begin{frame}[fragile]
\frametitle{Nun los}
\begin{verbatim}
# Fangen wir bei uns an
cdist-deploy-to localhost
\end{verbatim}
\end{frame}
\begin{frame}[fragile]
\frametitle{Der Einstiegspunkt}
\begin{small}
\begin{verbatim}
cat << eof > conf/manifest/init
__file /etc/cdist-configured
case "$__target_host" in
localhost)
__link /tmp/cdist-testfile \
--source /etc/cdist-configured \
--type symbolic
__addifnosuchline /tmp/cdist-welcome \
--line "Welcome to cdist"
;;
esac
eof
# Muss ausführbar sein
chmod u+x conf/manifest/init
\end{verbatim}
\end{small}
\end{frame}
\begin{frame}[fragile]
\frametitle{Nun los}
\begin{verbatim}
# Nun läuft es!
cdist-deploy-to localhost
\end{verbatim}
\end{frame}
\frame
{
\frametitle{Funktionalität zusammenfassen}
\begin{itemize}[<+->]
\item "`Typen"' (types)
\item conf/type/*
\item \_\_ vor jedem Namen (Shell-Umgebung)
\item z.B. Netzseite, Mailserver, Wiki, ...
\end{itemize}
}
\begin{frame}[fragile]
\frametitle{Ein neuer Typ}
\begin{small}
\begin{verbatim}
% mkdir conf/type/__my_mailserver
% cat << eof > conf/type/__my_mailserver/manifest
# Dieser Typ konfiguriert meinen Mailserver
require="__package/nullmailer" \
__file /etc/nullmailer/remotes \
--source "$__type/files/remotes"
# Reihenfolge spielt keine Rolle
__package nullmailer --state installed
eof
\end{verbatim}
\end{small}
\end{frame}
\begin{frame}[fragile]
\frametitle{Ein neuer Typ (2)}
\begin{small}
\begin{verbatim}
# Wichtig: Wird ausgeführt
% chmod u+x conf/type/__my_mailserver/manifest
# Darf nur einmal verwendet werden pro Rechner
% touch conf/type/__my_mailserver/singleton
# Nullmailer Konfiguration
% mkdir conf/type/__my_mailserver/files
% echo my.fancy.smart.host > \
conf/type/__my_mailserver/files/remotes
\end{verbatim}
\end{small}
\end{frame}
\begin{frame}[fragile]
\frametitle{Neuen Typ nutzen}
\begin{small}
\begin{verbatim}
% $EDITOR conf/manifest/init
case "$__target_host" in
localhost)
...
__my_mailserver
...
;;
\end{verbatim}
\end{small}
\end{frame}
\section{Aktualisieren}
\begin{frame}[fragile]
\frametitle{Versionen}
\begin{itemize}[<+->]
\item x.y: Stabile Version
\item master: Entwicklung
\end{itemize}
\end{frame}
\begin{frame}[fragile]
\frametitle{Stabile Version auswählen}
\begin{center}
git checkout -b 1.7 origin/1.7
\end{center}
\end{frame}
\begin{frame}[fragile]
\frametitle{Aktualisieren}
\begin{center}
git pull
\end{center}
\end{frame}
\frame
{
\frametitle{Ende}
\begin{block}{Das war's}
Viel Spaß - Mehr Infos gibt's auf http://www.nico.schottelius.org/software/cdist/\\
und http://l.schottelius.org/pipermail/cdist
\end{block}
}
\end{document}