tom/cdist
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

merge from upstream

Browse source 
Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>
This commit is contained in:
Steven Armstrong 2011-04-04 13:18:04 +02:00
commit b9b90dad36
25 changed files with 420 additions and 176 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
.gitignore vendored
View file

@ -3,13 +3,13 @@
# Ignore generated manpages
doc/man/.marker
doc/man/man*/*.1
doc/man/man*/*.7
doc/man/man1/*.1
doc/man/man7/*.7
doc/man/man*/*.html
doc/man/man*/*.xml
doc/man/man*/cdist-reference.text
# Ignore type manpages
doc/man/man*/*__*.text
doc/man/man7/cdist-type__*.text
doc/man/man7/cdist-reference.text
doc/man/man*/docbook-xsl.css
# Ignore cache for version control
cache/

69
README
View file

@ -24,63 +24,42 @@ other configuration management systems like
[cfengine](http://www.cfengine.org/),
[bcfg2](http://trac.mcs.anl.gov/projects/bcfg2),
[chef](http://wiki.opscode.com/display/chef/)
and [puppet](http://www.puppetlabs.com/), but
it ticks differently:
and [puppet](http://www.puppetlabs.com/), but it ticks differently.
Here are some features that makes it unique:
* cdist sticks completly to the KISS (keep it simple and stupid) paradigma
* cdist's core is very small (< 1k lines of code)
* There is only one type to extend cdist called ***type***.
* One main development target: ***It must be incredible easy to add new types.***
* cdist is UNIX
* It reuses existing tools like cat, find, mv, ...
* cdist's documentation is bundled as manpages
* cdist is written in POSIX shell
* No special requirements like high level interpreters needed on server or target
[[!table data="""
Keywords | Description
Simplicity | There is only one type to extend cdist called ***type***
Design | Type and core cleanly seperated
Design | Sticks completly to the KISS (keep it simple and stupid) paradigma
Design | Meaningful error messages - do not lose time debugging error messages
Design | Consistency in behaviour, naming and documentation
Design | No surprise factor: Only do what is obviously clear, no magic
Design | Define target state, do not focus on methods or scripts
Small core | cdist's core is very small - less code, less bugs
Fast development | Focus on straightforwardness of type creation is a main development objective
Requirements, Scalability | No central server needed, cdist operates in push mode and can be run from any computer
Requirements, Scalability, Upgrade | cdist only needs to be updated on the master, not on the target hosts
Requirements, Security | Uses well-know [SSH](http://www.openssh.com/) as transport protocol
Requirements, Simplicity | Requires only shell and SSH server on the target
UNIX | Reuse of existing tools like cat, find, mv, ...
UNIX, familar environment, documentation | Is available as manpages and HTML
UNIX, simplicity, familar environment | cdist is written in POSIX shell
UNIX, simplicity, familar environment | cdist is configured in POSIX shell
"""]]
### Documentation
The cdist documentation is included as manpages in the distribution.
You can [browse the documentation for the latest version online](man) as well.
### Architecture
* Push mode (server pushes configuration)
* User defines configuration in shell scripts (called ***manifests***)
* Generates internal configuration (cconfig style)
* Uses ***types*** to generate code be executed on the target
* And finally executes the code on the target / applies the configuration
### Features
* Elegant code and clean design
* Type and core cleanly seperated
* Small codebase in core
* Good documentation (man pages)
* Consistency in behaviour, naming and documentation
* Meaningful error messages
* Either standard error messages from tools or added description for clearification
* The no surprise factor
* No magic guessing of what the user wants
* Simple and well-known DSL
* Posix shell
* Easy integration into bare metal installations
* requires only ssh + sh
* Easy upgrade
* ***There is no need to update cdist on target hosts!***
* cdist only needs to be updated on the master server(s)
* Very easy to extend
* Can be done via types, which can be stacked on top of others
* Reuse of existing functionality
* sh, ssh, find, rm, mv, ...
* Very easy to debug
* Just add set -x in the scripts
### OS support
cdist was tested or is know to run on at least
* [Archlinux](http://www.archlinux.org/)
* [Debian](http://www.debian.org/)
* [Fedora](http://fedoraproject.org/)
* [Gentoo](http://www.gentoo.org/)
* [Mac OS X](http://www.apple.com/macosx/)
* [OpenBSD](http://www.openbsd.org)
@ -230,7 +209,7 @@ Yes, I'm actually eating my own dogfood and currently managing
* [xfce](http://www.xfce.org/) (lightweight desktop environment)
* [slim](http://slim.berlios.de/) (graphical login manager for X11)
with cdist on a total of **9** production machines of the
with cdist on a total of **20** production machines of the
[Systems Group](http://www.systems.ethz.ch) at the
[ETH Zurich](http://www.ethz.ch) as well at home.

2
bin/cdist-config
View file

@ -158,7 +158,7 @@ __cdist_debug_echo()
__cdist_exec_fail_on_error()
{
sh -e "$@"
[ "$?" -eq 0 ] || __cdist_exit_err "Error: $1 exited non-zero."
[ "$?" -eq 0 ] || __cdist_exit_err "$1 exited non-zero."
}
__cdist_exit_err()

6
conf/explorer/os
View file

@ -43,6 +43,12 @@ if [ -f /etc/gentoo-release ]; then
exit 0
fi
# Fedora is also Redhat, thus return before redhat!
if grep -q ^Fedora /etc/redhat-release 2>/dev/null; then
echo fedora
exit 0
fi
if [ -f /etc/redhat-release ]; then
echo redhat
exit 0

26
conf/type/__debconf_set_selections/gencode-remote Executable file
View file

@ -0,0 +1,26 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Setup selections
#
echo "debconf-set-selections << __file-eof"
cat "$__object/parameter/file"
echo "__file-eof"

43
conf/type/__debconf_set_selections/man.text Normal file
View file

@ -0,0 +1,43 @@
cdist-type__debconf_set_selections(7)
=====================================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-type__debconf_set_selections - Setup debconf selections
DESCRIPTION
-----------
On Debian and alike systems debconf-set-selections(1) can be used
to setup configuration parameters.
REQUIRED PARAMETERS
-------------------
file::
If supplied, use the given filename as input for debconf-set-selections(1)
EXAMPLES
--------
--------------------------------------------------------------------------------
# Setup configuration for nslcd
__debconf_set_selections nslcd --file /path/to/file
# Setup configuration for nslcd from another type
__debconf_set_selections nslcd --file "$__type/files/preseed/nslcd"
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist-type(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

1
conf/type/__debconf_set_selections/parameter/required Normal file
View file

@ -0,0 +1 @@
file

62
conf/type/__group/gencode-remote
View file

@ -1,6 +1,7 @@
#!/bin/sh
#
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
@ -23,44 +24,31 @@
name="$__object_id"
command=
if grep -q "^$name" "$__object/explorer/group"; then
# group exists
command="groupmod"
else
# group does not exist
command="groupadd"
fi
get_current_value() {
local key="$1"
local index
case "$key" in
password)
cut -d':' -f 2 "$__object/explorer/gshadow"
break
;;
gid) index=3;;
esac
cut -d':' -f $index "$__object/explorer/group"
}
set -- "$@"
cd "$__object/parameter"
for property in $(ls .); do
current_value=$(get_current_value "$property")
new_value="$(cat "$property")"
if [ "$new_value" != "$current_value" ]; then
# Shedule changed properties for update
if grep -q "^${name}:" "$__object/explorer/group"; then
for property in $(ls .); do
new_value="$(cat "$property")"
case "$key" in
password)
current_value="$(awk -F: '{ print $2 }' < "$__object/explorer/gshadow")"
;;
gid)
current_value="$(awk -F: '{ print $3 }' < "$__object/explorer/group")"
;;
esac
if [ "$new_value" != "$current_value" ]; then
set -- "$@" "--$property" \"$new_value\"
fi
done
[ $# -gt 0 ] && echo groupmod $@ $name
else
for property in $(ls .); do
new_value="$(cat "$property")"
set -- "$@" "--$property" \"$new_value\"
fi
done
done
if [ $# -gt 0 ]; then
# Update changed properties
echo $command $@ $name
echo groupadd "$@" "$name"
fi

1
conf/type/__package/manifest
View file

@ -33,6 +33,7 @@ else
archlinux) type="pacman" ;;
debian|ubuntu) type="apt" ;;
gentoo) type="emerge" ;;
fedora) type="yum" ;;
*)
echo "Don't know how to manage packages on: $os" >&2
exit 1

6
conf/type/__package_apt/gencode-remote
View file

@ -37,17 +37,19 @@ fi
state="$(cat "$__object/parameter/state")"
is_installed="$(grep "^Status: install ok installed" "$__object/explorer/pkg_status" || true)"
aptget="DEBIAN_FRONTEND=noninteractive apt-get --quiet --yes"
case "$state" in
installed)
# Install only if non-existent
if [ -z "$is_installed" ]; then
echo apt-get --quiet --yes install \"$name\"
echo $aptget install \"$name\"
fi
;;
uninstalled)
# Remove only if existent
if [ -n "$is_installed" ]; then
echo apt-get --quiet --yes remove \"$name\"
echo $aptget remove \"$name\"
fi
;;
esac

6
conf/type/__package_apt/man.text
View file

@ -25,9 +25,6 @@ OPTIONAL PARAMETERS
name::
If supplied, use the name and not the object id as the package name.
preseed::
If supplied, use the given filename as input for debconf-set-selections(1)
EXAMPLES
--------
@ -39,9 +36,6 @@ __package_apt zsh --state installed
# In case you only want *a* webserver, but don't care which one
__package_apt webserver --state installed --name nginx
# Install package with defaults (from a type)
__package_apt postfix --state installed --preseed "$__type/files/postfix-seed"
# Remove obsolete package
__package_apt puppet --state deinstalled
--------------------------------------------------------------------------------

1
conf/type/__package_apt/parameter/optional
View file

@ -1,3 +1,2 @@
name
preseed
version

30
conf/type/__package_yum/explorer/pkg_version Executable file
View file

@ -0,0 +1,30 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Retrieve the status of a package - parsed dpkg output
#
if [ -f "$__object/parameter/name" ]; then
name="$(cat "$__object/parameter/name")"
else
name="$__object_id"
fi
rpm -q --whatprovides "$name" 2>/dev/null || true

47
conf/type/__package_yum/gencode-remote Executable file
View file

@ -0,0 +1,47 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Manage packages with yum (mostly Fedora)
#
if [ -f "$__object/parameter/name" ]; then
name="$__object/parameter/name"
else
name="$__object_id"
fi
state="$(cat "$__object/parameter/state")"
opts="--assumeyes --quiet"
not_installed="^no package provides"
case "$state" in
installed)
if grep -q "$not_installed" "$__object/explorer/pkg_version"; then
echo yum $opts install \"$name\"
fi
;;
uninstalled)
if ! grep -q "$not_installed" "$__object/explorer/pkg_version"; then
echo yum $opts remove \"$name\"
fi
;;
esac

54
conf/type/__package_yum/man.text Normal file
View file

@ -0,0 +1,54 @@
cdist-type__package_yum(7)
==========================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-type__package_yum - Manage packages with yum
DESCRIPTION
-----------
Yum is usually used on the Fedora distribution to manage packages.
If you specify an unknown package, yum will display the
slightly confusing error message "Error: Nothing to do".
REQUIRED PARAMETERS
-------------------
state::
Either "installed" or "deinstalled".
OPTIONAL PARAMETERS
-------------------
name::
If supplied, use the name and not the object id as the package name.
EXAMPLES
--------
--------------------------------------------------------------------------------
# Ensure zsh in installed
__package_yum zsh --state installed
# If you don't want to follow pythonX packages, but always use python
__package_yum python --state installed --name python2
# Remove obsolete package
__package_yum puppet --state deinstalled
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist-type(7)
- cdist-type__package(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

1
conf/type/__package_yum/parameter/optional Normal file
View file

@ -0,0 +1 @@
name

1
conf/type/__package_yum/parameter/required Normal file
View file

@ -0,0 +1 @@
state

94
conf/type/__user/gencode-remote
View file

@ -1,6 +1,7 @@
#!/bin/sh
#
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
@ -23,60 +24,47 @@
name="$__object_id"
command=
if grep -q "^$name" "$__object/explorer/passwd"; then
# user exists
command="usermod"
else
# user does not exist
command="useradd"
fi
get_current_value() {
local key="$1"
local new_value="$2"
local explorer="$__object/explorer/passwd"
local index
case "$key" in
password)
explorer="$__object/explorer/shadow"
index=2
;;
uid) index=3;;
gid)
if [[ $new_value =~ ^[0-9]+$ ]]; then
# numeric gid
index=4
else
# group name
explorer="$__object/explorer/group"
index=1
fi
;;
comment) index=5;;
home) index=6;;
shell) index=7;;
esac
cut -d':' -f $index "$explorer"
}
set -- "$@"
cd "$__object/parameter"
for property in $(ls .); do
new_value="$(cat "$property")"
current_value=$(get_current_value "$property" "$new_value")
if [ "$new_value" != "$current_value" ]; then
# Shedule changed properties for update
if grep -q "^${name}:" "$__object/explorer/passwd"; then
for property in $(ls .); do
new_value="$(cat "$property")"
file="$__object/explorer/passwd"
case "$property" in
password)
field=3
file="$__object/explorer/shadow"
;;
gid)
if $(echo "$new_value" | grep -q '^[0-9][0-9]*$'); then
field=4
else
# group name
file="$__object/explorer/group"
field=1
fi
;;
uid) field=3 ;;
comment) field=5 ;;
home) field=6 ;;
shell) field=7 ;;
esac
export field
current_value="$(awk -F: '{ print $ENVIRON["field"] }' < "$file")"
if [ "$new_value" != "$current_value" ]; then
set -- "$@" "--$property" \"$new_value\"
fi
done
[ $# -gt 0 ] && echo usermod "$@" "$name"
else
for property in $(ls .); do
new_value="$(cat "$property")"
set -- "$@" "--$property" \"$new_value\"
fi
done
done
if [ $# -gt 0 ]; then
# Update changed properties
echo $command $@ $name
echo useradd "$@" "$name"
fi

9
doc/changelog
View file

@ -1,3 +1,12 @@
1.6.0:
* New Type __package_yum
* New type __debconf_set_selections
* Fix Type __group in case of __group NAME syntax
* Type __package gained Fedora support
* Removed --preseed support from __package_apt
* explorer/os: gained Fedora support
* Simplified types __user and __group
1.5.0: 2011-04-01
* Add basic cache functionality
* New type __process

9
doc/dev/logs/2011-04-02.yum Normal file
View file

@ -0,0 +1,9 @@
Fun with yum:
[root@brett ~]# yum --assumeyes --quiet install "vim"
Package 2:vim-enhanced-7.3.056-1.fc14.x86_64 already installed and latest version
[root@brett ~]# rpm -q vim
package vim is not installed
[root@brett ~]#
(Me || yum) == dumb?

1
doc/dev/todo/1.6 Normal file
View file

@ -0,0 +1 @@
- adjust documentation / stages

19
doc/dev/todo/TAKEME
View file

@ -26,13 +26,14 @@ CORE
TYPES
------
Types to be written/extended:
- __ssh-keys (host/user)
- Think about __service - necessary?
- __file_edit
- regexp replace (can probably cover all?)
-> aka sed.
- __cron
- __ssh-keys (host/user)
- __file_edit
- regexp replace (can probably cover all?)
-> aka sed.
- __cron
- __user:
add option to include --create-home
fix __user NAME case (same issue as __group)
DOCUMENTATION
--------------
@ -45,7 +46,3 @@ Cache:
- export variable $__cache
-> for current host
-> add function to cdist-config, import from cdist-cache
remove --preseed from package_apt and add debconf_set_selection or similar
-> much cleaner!

3
doc/dev/todo/niconext
View file

@ -1 +1,2 @@
Release 1.5.0 correctly :-)
remove --preseed from package_apt and add debconf_set_selection or similar
-> much cleaner!

17
doc/man/cdist-reference.text.sh
View file

@ -135,19 +135,8 @@ tmp_dir::
TYPES
-----
The following types are available:
eof
for type in man7/cdist-type__*; do
name_1="${type#man7/cdist-type}"
name_2="${name_1%.7}"
name="$name_2"
echo "- $name"
done
cat << eof
The available types are listed in the SEE ALSO section
and are referenced as cdist-type__TYPENAME.
VARIABLES
---------
@ -187,7 +176,7 @@ SEE ALSO
--------
- cdist(7)
eof
for type in man7/cdist-type__*; do
for type in man7/cdist-type__*.7; do
name_1="${type#man7/}"
name_2="${name_1%.7}"

78
doc/man/man7/cdist-best-practice.text
View file

@ -60,6 +60,84 @@ machine-a % git clone git://your-git-server/cdist
machine-b % git clone git://your-git-server/cdist
--------------------------------------------------------------------------------
SEPERATING WORK BY GROUPS
-------------------------
If you are working with different groups on one cdist-configuration,
you can delegate to other manifests and have the groups edit only
their manifests. You can use the following snippet in
**conf/manifests/init**:
--------------------------------------------------------------------------------
# Include other groups
sh -e "$__manifest/systems"
sh -e "$__manifest/cbrg"
--------------------------------------------------------------------------------
MAINTAINING MULTIPLE CONFIGURATIONS
-----------------------------------
When you need to manage multiple sites with cdist, like company_a, company_b
and private for instance, you can easily use git for this purpose.
Including a possible common base that is reused accross the different sites:
--------------------------------------------------------------------------------
# create branches
git branch company_a company_b common private
# make stuff for company a
git checkout company_a
# work, commit, etc.
# make stuff for company b
git checkout company_b
# work, commit, etc.
# make stuff relevant for all sites
git checkout common
# work, commit, etc.
# change to private and include latest common stuff
git checkout private
git merge common
--------------------------------------------------------------------------------
The following **.git/config** is taken from a a real world scenario:
--------------------------------------------------------------------------------
# Track upstream, merge from time to time
[remote "upstream"]
url = git://git.schottelius.org/cdist
fetch = +refs/heads/*:refs/remotes/upstream/*
# Same as upstream, but works when being offline
[remote "local"]
fetch = +refs/heads/*:refs/remotes/local/*
url = /home/users/nico/p/cdist
# Remote containing various ETH internal branches
[remote "eth"]
url = sans.ethz.ch:/home/services/sans/git/cdist-eth
fetch = +refs/heads/*:refs/remotes/eth/*
# Public remote that contains my private changes to cdist upstream
[remote "nico"]
url = git.schottelius.org:/home/services/git/cdist-nico
fetch = +refs/heads/*:refs/remotes/nico/*
# The "nico" branch will be synced with the remote nico, branch master
[branch "nico"]
remote = nico
merge = refs/heads/master
# ETH stable contains rock solid configurations used in various places
[branch "eth-stable"]
remote = eth
merge = refs/heads/stable
--------------------------------------------------------------------------------
Have a look at git-remote(1) to adjust the remote configuration, which allows
you to push certain branches to certain remotes.
SEE ALSO
--------