merge from upstream

Signed-off-by: Steven Armstrong <steven@icarus.ethz.ch>

.gitignore

@@ -3,13 +3,13 @@
 
 # Ignore generated manpages
 doc/man/.marker
-doc/man/man*/*.1
+doc/man/man1/*.1
-doc/man/man*/*.7
+doc/man/man7/*.7
 doc/man/man*/*.html
 doc/man/man*/*.xml
-doc/man/man*/cdist-reference.text
+doc/man/man7/cdist-type__*.text
-# Ignore type manpages
+doc/man/man7/cdist-reference.text
-doc/man/man*/*__*.text
+doc/man/man*/docbook-xsl.css
 
 # Ignore cache for version control
 cache/

README

@@ -24,63 +24,42 @@ other configuration management systems like
 [cfengine](http://www.cfengine.org/),
 [bcfg2](http://trac.mcs.anl.gov/projects/bcfg2),
 [chef](http://wiki.opscode.com/display/chef/)
-and [puppet](http://www.puppetlabs.com/), but
+and [puppet](http://www.puppetlabs.com/), but it ticks differently.
-it ticks differently:
+Here are some features that makes it unique: 
 
- * cdist sticks completly to the KISS (keep it simple and stupid) paradigma
+[[!table  data="""
-  * cdist's core is very small (< 1k lines of code)
+Keywords | Description
-  * There is only one type to extend cdist called ***type***.
+Simplicity | There is only one type to extend cdist called ***type***
-  * One main development target: ***It must be incredible easy to add new types.***
+Design | Type and core cleanly seperated
- * cdist is UNIX
+Design | Sticks completly to the KISS (keep it simple and stupid) paradigma
-  * It reuses existing tools like cat, find, mv, ...
+Design | Meaningful error messages - do not lose time debugging error messages
-  * cdist's documentation is bundled as manpages
+Design | Consistency in behaviour, naming and documentation
- * cdist is written in POSIX shell
+Design | No surprise factor: Only do what is obviously clear, no magic
-  * No special requirements like high level interpreters needed on server or target
+Design | Define target state, do not focus on methods or scripts
+Small core | cdist's core is very small - less code, less bugs
+Fast development | Focus on straightforwardness of type creation is a main development objective
+Requirements, Scalability | No central server needed, cdist operates in push mode and can be run from any computer
+Requirements, Scalability, Upgrade | cdist only needs to be updated on the master, not on the target hosts
+Requirements, Security | Uses well-know [SSH](http://www.openssh.com/) as transport protocol
+Requirements, Simplicity | Requires only shell and SSH server on the target
+UNIX | Reuse of existing tools like cat, find, mv, ...
+UNIX, familar environment, documentation | Is available as manpages and HTML
+UNIX, simplicity, familar environment | cdist is written in POSIX shell
+UNIX, simplicity, familar environment | cdist is configured in POSIX shell
+"""]]
 
 ### Documentation
 
 The cdist documentation is included as manpages in the distribution.
 You can [browse the documentation for the latest version online](man) as well.
 
-### Architecture
-
- * Push mode (server pushes configuration)
- * User defines configuration in shell scripts (called ***manifests***)
- * Generates internal configuration (cconfig style)
- * Uses ***types*** to generate code be executed on the target
- * And finally executes the code on the target / applies the configuration
-
-### Features
-
- * Elegant code and clean design
-  * Type and core cleanly seperated
-  * Small codebase in core
- * Good documentation (man pages)
- * Consistency in behaviour, naming and documentation
- * Meaningful error messages
-  * Either standard error messages from tools or added description for clearification
- * The no surprise factor
-  * No magic guessing of what the user wants
- * Simple and well-known DSL
-  * Posix shell
- * Easy integration into bare metal installations
-  * requires only ssh + sh
- * Easy upgrade
-  * ***There is no need to update cdist on target hosts!***
-  * cdist only needs to be updated on the master server(s)
- * Very easy to extend
-  * Can be done via types, which can be stacked on top of others
- * Reuse of existing functionality
-  * sh, ssh, find, rm, mv, ...
- * Very easy to debug
-  * Just add set -x in the scripts
-
 ### OS support
 
 cdist was tested or is know to run on at least
 
  * [Archlinux](http://www.archlinux.org/)
  * [Debian](http://www.debian.org/)
+ * [Fedora](http://fedoraproject.org/)
  * [Gentoo](http://www.gentoo.org/)
  * [Mac OS X](http://www.apple.com/macosx/)
  * [OpenBSD](http://www.openbsd.org)
@@ -230,7 +209,7 @@ Yes, I'm actually eating my own dogfood and currently managing
  * [xfce](http://www.xfce.org/) (lightweight desktop environment)
  * [slim](http://slim.berlios.de/) (graphical login manager for X11)
 
-with cdist on a total of **9** production machines of the
+with cdist on a total of **20** production machines of the
 [Systems Group](http://www.systems.ethz.ch) at the
 [ETH Zurich](http://www.ethz.ch) as well at home.
 

bin/cdist-config

@@ -158,7 +158,7 @@ __cdist_debug_echo()
 __cdist_exec_fail_on_error()
 {
    sh -e "$@"
-   [ "$?" -eq 0 ] || __cdist_exit_err "Error: $1 exited non-zero."
+   [ "$?" -eq 0 ] || __cdist_exit_err "$1 exited non-zero."
 }
 
 __cdist_exit_err()

conf/explorer/os

@@ -43,6 +43,12 @@ if [ -f /etc/gentoo-release ]; then
    exit 0
 fi
 
+# Fedora is also Redhat, thus return before redhat!
+if grep -q ^Fedora /etc/redhat-release 2>/dev/null; then
+   echo fedora
+   exit 0
+fi
+
 if [ -f /etc/redhat-release ]; then
    echo redhat
    exit 0

conf/type/__debconf_set_selections/gencode-remote

@@ -0,0 +1,26 @@
+#!/bin/sh
+#
+# 2011 Nico Schottelius (nico-cdist at schottelius.org)
+#
+# This file is part of cdist.
+#
+# cdist is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# cdist is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with cdist. If not, see <http://www.gnu.org/licenses/>.
+#
+#
+# Setup selections
+#
+
+echo "debconf-set-selections << __file-eof"
+cat "$__object/parameter/file"
+echo "__file-eof"

conf/type/__debconf_set_selections/man.text

@@ -0,0 +1,43 @@
+cdist-type__debconf_set_selections(7)
+=====================================
+Nico Schottelius <nico-cdist--@--schottelius.org>
+
+
+NAME
+----
+cdist-type__debconf_set_selections - Setup debconf selections
+
+
+DESCRIPTION
+-----------
+On Debian and alike systems debconf-set-selections(1) can be used
+to setup configuration parameters.
+
+
+REQUIRED PARAMETERS
+-------------------
+file::
+   If supplied, use the given filename as input for debconf-set-selections(1)
+
+
+EXAMPLES
+--------
+
+--------------------------------------------------------------------------------
+# Setup configuration for nslcd
+__debconf_set_selections nslcd --file /path/to/file
+
+# Setup configuration for nslcd from another type
+__debconf_set_selections nslcd --file "$__type/files/preseed/nslcd"
+--------------------------------------------------------------------------------
+
+
+SEE ALSO
+--------
+- cdist-type(7)
+
+
+COPYING
+-------
+Copyright \(C) 2011 Nico Schottelius. Free use of this software is
+granted under the terms of the GNU General Public License version 3 (GPLv3).

conf/type/__debconf_set_selections/parameter/required

@@ -0,0 +1 @@
+file

conf/type/__group/gencode-remote

@@ -1,6 +1,7 @@
 #!/bin/sh
 #
 # 2011 Steven Armstrong (steven-cdist at armstrong.cc)
+# 2011 Nico Schottelius (nico-cdist at schottelius.org)
 #
 # This file is part of cdist.
 #
@@ -23,44 +24,31 @@
 
 name="$__object_id"
 
-command=
+cd "$__object/parameter"
-if grep -q "^$name" "$__object/explorer/group"; then
+if grep -q "^${name}:" "$__object/explorer/group"; then
-   # group exists
+   for property in $(ls .); do
-   command="groupmod"
+      new_value="$(cat "$property")"
-else
-   # group does not exist
-   command="groupadd"
-fi
 
-
-get_current_value() {
-   local key="$1"
-   local index
       case "$key" in
          password)
-         cut -d':' -f 2 "$__object/explorer/gshadow"
+            current_value="$(awk -F: '{ print $2 }' < "$__object/explorer/gshadow")"
-         break
+         ;;
+         gid)
+            current_value="$(awk -F: '{ print $3 }' < "$__object/explorer/group")"
          ;;
-      gid) index=3;;
       esac
-   cut -d':' -f $index "$__object/explorer/group"
-}
 
-
-set -- "$@"
-cd "$__object/parameter"
-for property in $(ls .); do
-   current_value=$(get_current_value "$property")
-   new_value="$(cat "$property")"
       if [ "$new_value" != "$current_value" ]; then
-      # Shedule changed properties for update
          set -- "$@" "--$property" \"$new_value\"
       fi
-done
+   done
 
+   [ $# -gt 0 ] && echo groupmod $@ $name
+else
+   for property in $(ls .); do
+      new_value="$(cat "$property")"
+      set -- "$@" "--$property" \"$new_value\"
+   done
 
-if [ $# -gt 0 ]; then
+   echo groupadd "$@" "$name"
-   # Update changed properties
-   echo $command $@ $name
 fi
-

conf/type/__package/manifest

@@ -33,6 +33,7 @@ else
          archlinux) type="pacman" ;;
          debian|ubuntu) type="apt" ;;
          gentoo) type="emerge" ;;
+         fedora) type="yum" ;;
          *)
             echo "Don't know how to manage packages on: $os" >&2
             exit 1

conf/type/__package_apt/gencode-remote

@@ -37,17 +37,19 @@ fi
 state="$(cat "$__object/parameter/state")"
 is_installed="$(grep "^Status: install ok installed" "$__object/explorer/pkg_status" || true)"
 
+aptget="DEBIAN_FRONTEND=noninteractive apt-get --quiet --yes"
+
 case "$state" in
    installed)
       # Install only if non-existent
       if [ -z "$is_installed" ]; then
-         echo apt-get --quiet --yes install \"$name\"
+         echo $aptget install \"$name\"
       fi
    ;;
    uninstalled)
       # Remove only if existent
       if [ -n "$is_installed" ]; then
-         echo apt-get --quiet --yes remove \"$name\"
+         echo $aptget remove \"$name\"
       fi
    ;;
 esac

conf/type/__package_apt/man.text

@@ -25,9 +25,6 @@ OPTIONAL PARAMETERS
 name::
    If supplied, use the name and not the object id as the package name.
 
-preseed::
-   If supplied, use the given filename as input for debconf-set-selections(1)
-
 
 EXAMPLES
 --------
@@ -39,9 +36,6 @@ __package_apt zsh --state installed
 # In case you only want *a* webserver, but don't care which one
 __package_apt webserver --state installed --name nginx
 
-# Install package with defaults (from a type)
-__package_apt postfix --state installed --preseed "$__type/files/postfix-seed"
-
 # Remove obsolete package
 __package_apt puppet --state deinstalled
 --------------------------------------------------------------------------------

conf/type/__package_apt/parameter/optional

@@ -1,3 +1,2 @@
 name
-preseed
 version

conf/type/__package_yum/explorer/pkg_version

@@ -0,0 +1,30 @@
+#!/bin/sh
+#
+# 2011 Nico Schottelius (nico-cdist at schottelius.org)
+#
+# This file is part of cdist.
+#
+# cdist is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# cdist is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with cdist. If not, see <http://www.gnu.org/licenses/>.
+#
+#
+# Retrieve the status of a package - parsed dpkg output
+#
+
+if [ -f "$__object/parameter/name" ]; then
+   name="$(cat "$__object/parameter/name")"
+else
+   name="$__object_id"
+fi
+
+rpm -q --whatprovides "$name" 2>/dev/null || true

conf/type/__package_yum/gencode-remote

@@ -0,0 +1,47 @@
+#!/bin/sh
+#
+# 2011 Nico Schottelius (nico-cdist at schottelius.org)
+#
+# This file is part of cdist.
+#
+# cdist is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# cdist is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with cdist. If not, see <http://www.gnu.org/licenses/>.
+#
+#
+# Manage packages with yum (mostly Fedora)
+#
+
+if [ -f "$__object/parameter/name" ]; then
+   name="$__object/parameter/name"
+else
+   name="$__object_id"
+fi
+
+state="$(cat "$__object/parameter/state")"
+
+opts="--assumeyes --quiet"
+
+not_installed="^no package provides"
+
+case "$state" in
+   installed)
+      if grep -q "$not_installed" "$__object/explorer/pkg_version"; then
+         echo yum $opts install \"$name\"
+      fi
+   ;;
+   uninstalled)
+      if ! grep -q "$not_installed" "$__object/explorer/pkg_version"; then
+         echo yum $opts remove \"$name\"
+      fi
+   ;;
+esac

conf/type/__package_yum/man.text

@@ -0,0 +1,54 @@
+cdist-type__package_yum(7)
+==========================
+Nico Schottelius <nico-cdist--@--schottelius.org>
+
+
+NAME
+----
+cdist-type__package_yum - Manage packages with yum
+
+
+DESCRIPTION
+-----------
+Yum is usually used on the Fedora distribution to manage packages.
+If you specify an unknown package, yum will display the
+slightly confusing error message "Error: Nothing to do".
+
+
+REQUIRED PARAMETERS
+-------------------
+state::
+   Either "installed" or "deinstalled".
+
+
+OPTIONAL PARAMETERS
+-------------------
+name::
+   If supplied, use the name and not the object id as the package name.
+
+
+EXAMPLES
+--------
+
+--------------------------------------------------------------------------------
+# Ensure zsh in installed
+__package_yum zsh --state installed
+
+# If you don't want to follow pythonX packages, but always use python
+__package_yum python --state installed --name python2
+
+# Remove obsolete package
+__package_yum puppet --state deinstalled
+--------------------------------------------------------------------------------
+
+
+SEE ALSO
+--------
+- cdist-type(7)
+- cdist-type__package(7)
+
+
+COPYING
+-------
+Copyright \(C) 2011 Nico Schottelius. Free use of this software is
+granted under the terms of the GNU General Public License version 3 (GPLv3).

conf/type/__package_yum/parameter/optional

@@ -0,0 +1 @@
+name

conf/type/__package_yum/parameter/required

@@ -0,0 +1 @@
+state

conf/type/__user/gencode-remote

@@ -1,6 +1,7 @@
 #!/bin/sh
 #
 # 2011 Steven Armstrong (steven-cdist at armstrong.cc)
+# 2011 Nico Schottelius (nico-cdist at schottelius.org)
 #
 # This file is part of cdist.
 #
@@ -23,60 +24,47 @@
 
 name="$__object_id"
 
-command=
+cd "$__object/parameter"
-if grep -q "^$name" "$__object/explorer/passwd"; then
+if grep -q "^${name}:" "$__object/explorer/passwd"; then
-   # user exists
+   for property in $(ls .); do
-   command="usermod"
+      new_value="$(cat "$property")"
-else
-   # user does not exist
-   command="useradd"
-fi
 
+      file="$__object/explorer/passwd"
 
-get_current_value() {
+      case "$property" in
-   local key="$1"
-   local new_value="$2"
-
-   local explorer="$__object/explorer/passwd"
-   local index
-   case "$key" in
          password)
-         explorer="$__object/explorer/shadow"
+            field=3 
-         index=2
+            file="$__object/explorer/shadow"
          ;;
-      uid) index=3;;
          gid)
-         if [[ $new_value =~ ^[0-9]+$ ]]; then
+            if $(echo "$new_value" | grep -q '^[0-9][0-9]*$'); then
-            # numeric gid
+               field=4
-            index=4
             else
                # group name
-            explorer="$__object/explorer/group"
+               file="$__object/explorer/group"
-            index=1
+               field=1
             fi
          ;;
-      comment) index=5;;
+         uid)     field=3 ;;
-      home) index=6;;
+         comment) field=5 ;;
-      shell) index=7;;
+         home)    field=6 ;;
+         shell)   field=7 ;;
       esac
-   cut -d':' -f $index "$explorer"
-}
 
+      export field
+      current_value="$(awk -F: '{ print $ENVIRON["field"] }' < "$file")"
 
-set -- "$@"
-cd "$__object/parameter"
-for property in $(ls .); do
-   new_value="$(cat "$property")"
-   current_value=$(get_current_value "$property" "$new_value")
       if [ "$new_value" != "$current_value" ]; then
-      # Shedule changed properties for update
          set -- "$@" "--$property" \"$new_value\"
       fi
-done
+   done
 
+   [ $# -gt 0 ] && echo usermod "$@" "$name"
+else
+   for property in $(ls .); do
+      new_value="$(cat "$property")"
+      set -- "$@" "--$property" \"$new_value\"
+   done
 
-if [ $# -gt 0 ]; then
+   echo useradd "$@" "$name"
-   # Update changed properties
-   echo $command $@ $name
 fi
-

doc/changelog

@@ -1,3 +1,12 @@
+1.6.0:
+	* New Type __package_yum
+	* New type __debconf_set_selections
+	* Fix Type __group in case of __group NAME syntax
+	* Type __package gained Fedora support
+	* Removed --preseed support from __package_apt
+	* explorer/os: gained Fedora support
+	* Simplified types __user and __group
+
 1.5.0: 2011-04-01
 	* Add basic cache functionality
 	* New type __process

doc/dev/logs/2011-04-02.yum

@@ -0,0 +1,9 @@
+Fun with yum:
+
+[root@brett ~]# yum --assumeyes --quiet install "vim"
+Package 2:vim-enhanced-7.3.056-1.fc14.x86_64 already installed and latest version
+[root@brett ~]# rpm -q vim
+package vim is not installed
+[root@brett ~]# 
+
+(Me || yum) == dumb?

doc/dev/todo/1.6

@@ -0,0 +1 @@
+- adjust documentation / stages

doc/dev/todo/TAKEME

@@ -26,13 +26,14 @@ CORE
 
 TYPES
 ------
-Types to be written/extended:
+- __ssh-keys (host/user)
-   - __ssh-keys (host/user)
+- __file_edit
-   - Think about __service - necessary?
-   - __file_edit
    - regexp replace (can probably cover all?)
       -> aka sed.
-   - __cron
+- __cron
+- __user:
+   add option to include --create-home
+   fix __user NAME case (same issue as __group)
 
 DOCUMENTATION
 --------------
@@ -45,7 +46,3 @@ Cache:
    - export variable $__cache
       -> for current host
       -> add function to cdist-config, import from cdist-cache
-
-
-remove --preseed from package_apt and add debconf_set_selection or similar
-   -> much cleaner!

doc/dev/todo/niconext

@@ -1 +1,2 @@
-Release 1.5.0 correctly :-)
+remove --preseed from package_apt and add debconf_set_selection or similar
+   -> much cleaner!

doc/man/cdist-reference.text.sh

@@ -135,19 +135,8 @@ tmp_dir::
 
 TYPES
 -----
-The following types are available:
+The available types are listed in the SEE ALSO section
-
+and are referenced as cdist-type__TYPENAME.
-eof
-for type in man7/cdist-type__*; do
-   name_1="${type#man7/cdist-type}"
-   name_2="${name_1%.7}"
-
-   name="$name_2"
-   echo "- $name"
-done
-
-cat << eof
-
 
 VARIABLES
 ---------
@@ -187,7 +176,7 @@ SEE ALSO
 --------
 - cdist(7)
 eof
-for type in man7/cdist-type__*; do
+for type in man7/cdist-type__*.7; do
    name_1="${type#man7/}"
    name_2="${name_1%.7}"
 

doc/man/man7/cdist-best-practice.text

@@ -60,6 +60,84 @@ machine-a % git clone git://your-git-server/cdist
 machine-b % git clone git://your-git-server/cdist
 --------------------------------------------------------------------------------
 
+SEPERATING WORK BY GROUPS
+-------------------------
+If you are working with different groups on one cdist-configuration,
+you can delegate to other manifests and have the groups edit only
+their manifests. You can use the following snippet in
+**conf/manifests/init**:
+
+--------------------------------------------------------------------------------
+# Include other groups
+sh -e "$__manifest/systems"
+
+sh -e "$__manifest/cbrg"
+--------------------------------------------------------------------------------
+
+
+MAINTAINING MULTIPLE CONFIGURATIONS
+-----------------------------------
+When you need to manage multiple sites with cdist, like company_a, company_b
+and private for instance, you can easily use git for this purpose.
+Including a possible common base that is reused accross the different sites:
+
+--------------------------------------------------------------------------------
+# create branches
+git branch company_a company_b common private
+
+# make stuff for company a
+git checkout company_a
+# work, commit, etc.
+
+# make stuff for company b
+git checkout company_b
+# work, commit, etc.
+
+# make stuff relevant for all sites
+git checkout common
+# work, commit, etc.
+
+# change to private and include latest common stuff
+git checkout private
+git merge common
+--------------------------------------------------------------------------------
+
+The following **.git/config** is taken from a a real world scenario:
+--------------------------------------------------------------------------------
+# Track upstream, merge from time to time
+[remote "upstream"]
+   url = git://git.schottelius.org/cdist
+   fetch = +refs/heads/*:refs/remotes/upstream/*
+
+# Same as upstream, but works when being offline 
+[remote "local"]
+   fetch = +refs/heads/*:refs/remotes/local/*
+   url = /home/users/nico/p/cdist
+
+# Remote containing various ETH internal branches
+[remote "eth"]
+   url = sans.ethz.ch:/home/services/sans/git/cdist-eth
+   fetch = +refs/heads/*:refs/remotes/eth/*
+
+# Public remote that contains my private changes to cdist upstream
+[remote "nico"]
+   url = git.schottelius.org:/home/services/git/cdist-nico
+   fetch = +refs/heads/*:refs/remotes/nico/*
+
+# The "nico" branch will be synced with the remote nico, branch master
+[branch "nico"]
+   remote = nico
+   merge = refs/heads/master
+
+# ETH stable contains rock solid configurations used in various places
+[branch "eth-stable"]
+   remote = eth
+   merge = refs/heads/stable
+--------------------------------------------------------------------------------
+
+Have a look at git-remote(1) to adjust the remote configuration, which allows
+you to push certain branches to certain remotes.
+
 
 SEE ALSO
 --------