Merge branch 'master' into 4.0-pre-not-stable

docs/changelog

@@ -15,12 +15,41 @@ Changelog
 	* Core: Integrate initial preos support
 
 
-3.1.6:
+3.1.10:
+	* Core: Fix too many open files bug (#343)
+	* Type __ssh_authorized_keys: Remove unneeded explorer (Steven Armstrong)
+	* Type __ssh_authorized_keys: Fix empty output bug of entry explorer (Steven Armstrong)
+	* Type __package_apt: Add support for --target-release
+
+3.1.9: 2014-10-17
+	* Type __package_emerge: Fix handling of slotted packages (Daniel Heule)
+	* Type __package_apt: Use --force-confdef (Ricardo Catalinas Jiménez)
+	* Type __package_update_index: Decrease verbosity (Ricardo Catalinas Jiménez)
+	* Type __package_upgrade_all: Decrease verbosity (Ricardo Catalinas Jiménez)
+
+3.1.8: 2014-10-01
+	* New Type: __package_update_index (Ricardo Catalinas Jiménez)
+	* New Type: __package_upgrade_all  (Ricardo Catalinas Jiménez)
+
+3.1.7: 2014-09-29
+	* Type __cdistmarker: Fix typo (Ricardo Catalinas Jiménez)
+	* Core: Bugfix: Export messaging to manifests (Ricardo Catalinas Jiménez)
+	* Explorer cpu_cores, cpu_sockets, memory: Add Mac OS X support (Manuel Hutter)
+	* Type __ssh_authorized_keys: Ensure keys are correctly added (Steven Armstrong)
+	* New Type: __ssh_authorized_key (Steven Armstrong)
+	* New Type: __package_pkgng_freebsd (Jake Guffey)
+
+3.1.6: 2014-08-18
+	* New Type: __ssh_dot_ssh
 	* Type __package_yum: Support retrieving package via URL
 	* Type __hostname: Support SuSE and have CentOS use sysconfig value
 	* Type __locale: Support SuSE
+	* Type __locale: Support Archlinux
 	* Type __timezone: Support SuSE
 	* Type __file: Support MacOS X (Manuel Hutter)
+	* Type __iptables_apply: Add "reset" to init.d script of iptables
+	* Type __ssh_authorized_key: Use new type __ssh_dot_ssh
+	* Type __zypper_repo: Bugfix for pattern matching (Daniel Heule)
 
 3.1.5: 2014-05-05
 	* Type __zypper_repo: Automatically import gpg keys (Daniel Heule)

docs/dev/logs/2014-09-10.preos-keys

@@ -0,0 +1,18 @@
+
+cdist preos keyramfs --keyfile --keyfile
+
+[17:51] freiheit:vcs% mkdir preos-keys
+[17:51] freiheit:vcs% mkdir -p preos-keys/root/.ssh 
+[17:56] freiheit:vcs% 
+
+chown root:root -R preos-keys/
+chmod 0600 preos-keys/root/.ssh/authorized_keys 
+
+chmod 0700 preos-keys/root/
+chmod 0700 preos-keys/root/.ssh/
+
+[18:20:17] freiheit:/home/users/nico/.ungleich/ungleich/vcs/preos-keys# find . | cpio -H newc -o | gzip -9 >  ../initramfs.cpio.gz
+4 blocks
+[18:21:08] freiheit:/home/users/nico/.ungleich/ungleich/vcs/preos-keys# 
+
+

docs/dev/logs/2014-11-11.discussion

@@ -0,0 +1,104 @@
+- logging/cache destination
+    local:
+        ~/.cdist/log/by-host/$__target/host/config/YYYY/MM/DD/hhmmss/
+        ~/.cdist/log/by-session/YYYY/MM/DD/hhmmss/$__target_host/
+            config/
+            install/
+            export/
+    remote:
+        /var/lib/cdist/YYYY-MM-DD-hhmmss-$sourcehost.$pid
+
+    rm old directories on remote side
+
+- support for tags
+    - for partial configuration
+    - supports also install
+    - on object definition, define
+        - a) don't care (i.e. no tags)
+        - b) require tag (only if this tag is setup)
+        - c) require not tag (only if this tag is *not* setup)
+        - d) what if both given (conflicting)
+    - names for parameters:
+        - cdist config / cdist type
+            --if-tag / --not-if-tag / --require-tag
+            --not-if-tag
+
+- logging
+    - command line
+    - stdin of cdist
+    - stdout/stderr/stdin of types
+        - new: stdout/stderr
+    - initial manifest
+        - if coming from stdin
+
+- logging configurable
+    - to be discussed
+
+- sudo remote
+    - cp to tmp & mv
+    - umask issue?
+
+- install
+    - via tagged types
+    - 
+
+- export
+    - one /export script per type
+    - exports of type running after object's code is done
+    - global export should also exist after everything
+    - PR & merge
+    - change DONE status to CODE_DONE
+    - introduce EXPORT_DONE
+
+
+- preos
+    - merge with debian support only
+    - we are open to support --os-type later
+
+- stackable remotes
+    - change API for remote_exec and remote_copy
+    - new minor version
+    - PR & merge
+
+- locking
+    - optional
+    - remote lock
+    - based on $(ls /var/lib/cdist/) > 0
+
+- ideas for parallelisation
+    - run explorer in parallel
+        - type
+        - object
+    - objects without dependencies can be run in parallel
+
+- connection test
+    - just implement
+
+- multi user environment
+    - not really needed [at the moment]
+    - can be implemented by 
+        - git branches
+        - setting the output dir
+
+- python2 support with __future__
+    - steven votes against it
+    - nico does not care too much to object
+
+
+- pull based
+    - sshd / stdin + stdout
+    - use Use ProxyCommand with stdin/stdout
+        - http://www.nico.schottelius.org/blog/openssh-6.2-add-callback-functionality-using-dynamic-remote-port-forwarding/
+
+    - cdist grant-pull-access <targethost>
+        - generate user
+        - ssh pubkeypair
+    - call wraper script on targethost
+        - it is shell!
+        - ssh cdistuser@controlhost 
+
+- config replay/redo/undo
+    - not now
+
+
+- have a new discussion about handling uris

docs/dev/todo/TAKEME

@@ -5,10 +5,6 @@ Feel free to pick one!
 
 CORE
 ----
-- support default parameter
-- document and add paremeters for remote-copy and remote-exec!
-    - remove hack, make a feature of it
-
 - remove var=foo calls on remote side. Use -o SendEnv (yeah, see ssh_config(5))
 
 TESTS
@@ -23,9 +19,6 @@ TESTS
 
 USER INTERFACE
 --------------
-- How to cleanly implement "restart service if config file changed"
-    -> document
-
 - Cache
    - add example how to use 
    - export variable $__cache
@@ -45,7 +38,6 @@ TYPES
 - Add testing framework (proposed by Evax Software)
 - __user
    add option to include --create-home
-- Merge __addifnosuchline and __removeline into __line + --state present|absent
 - __cron: Support --file to be used instead of user cron (probably direct support
     of /etc/cron.d)
 

docs/man/cdist-reference.text.sh

@@ -203,10 +203,10 @@ __global::
     Directory that contains generic output like explorer.
     Available for: initial manifest, type manifest, type gencode, shell
 __messages_in::
-    File to read messages from
+    File to read messages from.
     Available for: initial manifest, type manifest, type gencode
 __messages_out::
-    File to write messages
+    File to write messages.
     Available for: initial manifest, type manifest, type gencode
 __object::
     Directory that contains the current object.

docs/man/man7/cdist-explorer.text

@@ -50,7 +50,7 @@ else
    name="$__object_id"
 fi
 
-# Except dpkg failing, if package is not known / installed
+# Expect dpkg failing, if package is not known / installed
 dpkg -s "$name" 2>/dev/null || exit 0
 --------------------------------------------------------------------------------
 
@@ -64,5 +64,5 @@ SEE ALSO
 
 COPYING
 -------
-Copyright \(C) 2010-2012 Nico Schottelius. Free use of this software is
+Copyright \(C) 2010-2014 Nico Schottelius. Free use of this software is
 granted under the terms of the GNU General Public License version 3 (GPLv3).

docs/man/man7/cdist-manifest.text

@@ -110,6 +110,7 @@ setup the variable "require" to contain the requirements. Multiple
 requirements can be added white space separated.
 
 --------------------------------------------------------------------------------
+
  1 # No dependency
  2 __file /etc/cdist-configured
  3 
@@ -121,21 +122,43 @@ requirements can be added white space separated.
  9 require="__file/etc/cdist-configured __link/tmp/cdist-testfile" \
 10    __file /tmp/cdist-another-testfile
 
-
 --------------------------------------------------------------------------------
 
 Above the "require" variable is only set for the command that is 
-immediately following it. Dependencies should allways be declared that way.
+immediately following it. Dependencies should always be declared that way.
 
-On line 4 you can see that the instantion of a type "__link" object needs
+On line 4 you can see that the instantion of a type "\__link" object needs
 the object "__file/etc/cdist-configured" to be present, before it can proceed.
 
-This also means that the "__link" command must make sure, that either
-"__file/etc/cdist-configured" allready is present, or, if it's not, it needs
+This also means that the "\__link" command must make sure, that either
+"\__file/etc/cdist-configured" allready is present, or, if it's not, it needs
 to be created. The task of cdist is to make sure, that the dependency will be
-resolved appropriately and thus "__file/etc/cdist-configured" be created
+resolved appropriately and thus "\__file/etc/cdist-configured" be created
 if necessary before "__link" proceeds (or to abort execution with an error).
 
+If you really need to make all types depend on a common dependency, you can
+export the "require" variable as well. But then, if you need to add extra
+dependencies to a specific type, you have to make sure that you append these
+to the globally already defined one.
+
+--------------------------------------------------------------------------------
+
+# First of all, update the package index
+__package_update_index
+# Upgrade all the installed packages afterwards
+require="__package_update_index" __package_upgrade_all
+# Create a common dependency for all the next types so that they get to
+# be executed only after the package upgrade has finished
+export require="__package_upgrade_all"
+
+# Ensure that lighttpd is installed after we have upgraded all the packages
+__package lighttpd --state present
+# Ensure that munin is installed after lighttpd is present and after all
+# the packages are upgraded
+require="$require __package/lighttpd" __package munin --state present
+
+--------------------------------------------------------------------------------
+
 All objects that are created in a type manifest are automatically required
 from the type that is calling them. This is called "autorequirement" in
 cdist jargon.

docs/man/man7/cdist-messaging.text

@@ -57,6 +57,48 @@ if grep -q "^__your_type/object/id:something" "$__messages_in"; then
 fi
 --------------------------------------------------------------------------------
 
+Some real life examples:
+--------------------------------------------------------------------------------
+# Reacting on changes from block for keepalive
+if grep -q "^__block/keepalive-vrrp" "$__messages_in"; then
+    echo /etc/init.d/keepalived restart
+fi
+
+# Reacting on changes of configuration files
+if grep -q "^__file/etc/one" $__messages_in; then
+    echo 'for init in /etc/init.d/opennebula*; do $init restart; done'
+fi
+--------------------------------------------------------------------------------
+
+Restart sshd on changes
+--------------------------------------------------------------------------------
+os="$(cat "$__global/explorer/os")"
+
+case "$os" in
+    centos|redhat|suse)
+        restart="/etc/init.d/sshd restart"
+    ;;
+    debian|ubuntu)
+        restart="/etc/init.d/ssh restart"
+    ;;
+    *)
+        cat << eof >&2
+Unsupported os $os.
+If you would like to have this type running on $os,
+you can either develop the changes and send a pull
+request or ask for a quote at www.ungleich.ch
+eof
+        exit 1
+    ;;
+esac
+
+if grep -q "^__key_value/PermitRootLogin" "$__messages_in"; then
+    echo $restart
+fi
+--------------------------------------------------------------------------------
+
+
+
 
 SEE ALSO
 --------

docs/man/man7/cdist-type.text

@@ -100,7 +100,7 @@ echo use_ssl >> cdist/conf/type/__nginx_vhost/parameter/boolean
 USING PARAMETERS
 ----------------
 The parameters given to a type can be accessed and used in all type scripts
-(e.g manifest, gencode-*, explorer/*). Note that boolean parameters are
+(e.g manifest, gencode, explorer). Note that boolean parameters are
 represented by file existence. File exists -> True,
 file does not exist -> False
 
@@ -281,7 +281,7 @@ on the target, there must be another type that provides this tool and the first
 type should create an object of the specific type.
 
 If your type wants to save temporary data, that may be used by other types
-later on (for instance __file), you can save them in the subdirectory
+later on (for instance \__file), you can save them in the subdirectory
 "files" below $__object (but you must create it yourself).
 cdist will not touch this directory.