sparrowhawk/cdist
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Merge remote branch 'nico/master' into changerunningorder

Browse source
This commit is contained in:
Steven Armstrong 2011-03-25 23:08:50 +01:00
commit c7ebbb4125
21 changed files with 411 additions and 241 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

13
HACKERS_README
View file

@ -3,21 +3,12 @@ cat << eof
Hey hackers,
this README is for you, for those who want to dig into cdist, hack it or try
to get a deeper understanding.
A lot of documentation is still missing, but running cdist-quickstart should
give you an impression of how cdist works.
to get a deeper understanding. Please read doc/man/cdist-hacker.text.
I hope you have a lot of fun with cdist, because it was also a lot of fun to
develop it!
-- Nico, 20110304
## Conventions
- All variables exported by cdist are prefixed with a double underscore (__)
- All cdist-internal variables are prefixed with __cdist_ and are generally not exported.
-- Nico, 20110324
## Running cdist when developing

126
Makefile
View file

@ -1,126 +0,0 @@
# General
PREFIX=/usr
BINDIR=$(PREFIX)/bin
MANDIR=$(PREFIX)/share/man
# Manpage and HTML
A2XM=a2x -f manpage --no-xmllint
# A2XH=a2x -f xhtml --no-xmllint
A2XH=asciidoc -b xhtml11
# Developer only
WEBDIR=$$HOME/niconetz
WEBBASE=software/cdist
WEBPAGE=$(WEBBASE).mdwn
# Documentation
MANDIR=doc/man
MANGENERATED=$(MANDIR)/cdist-reference.text
MANSRC=$(MANDIR)/cdist.text \
$(MANDIR)/cdist-best-practise.text \
$(MANDIR)/cdist-code-run.text \
$(MANDIR)/cdist-code-run-all.text \
$(MANDIR)/cdist-config.text \
$(MANDIR)/cdist-dir.text \
$(MANDIR)/cdist-env.text \
$(MANDIR)/cdist-explorer-run-global.text \
$(MANDIR)/cdist-deploy-to.text \
$(MANDIR)/cdist-explorer.text \
$(MANDIR)/cdist-manifest.text \
$(MANDIR)/cdist-manifest-run.text \
$(MANDIR)/cdist-manifest-run-all.text \
$(MANDIR)/cdist-manifest-run-init.text \
$(MANDIR)/cdist-object-explorer-all.text \
$(MANDIR)/cdist-object-gencode.text \
$(MANDIR)/cdist-object-gencode-all.text \
$(MANDIR)/cdist-quickstart.text \
$(MANDIR)/cdist-remote-explorer-run.text \
$(MANDIR)/cdist-run-remote.text \
$(MANDIR)/cdist-stages.text \
$(MANDIR)/cdist-type.text \
$(MANDIR)/cdist-type-build-emulation.text \
$(MANDIR)/cdist-type-emulator.text \
$(MANDIR)/cdist-type-template.text \
################################################################################
# User targets
#
all:
@echo ''
@echo 'Welcome to cdist!'
@echo ''
@echo 'Here are the possible targets:'
@echo ''
@echo ' man: Build manpages (requires Asciidoc)'
@echo ' clean: Remove build stuff'
@echo ''
@echo ''
man: doc/man/.marker
doc/man/.marker: $(MANDIR)/cdist-reference.text
touch $@
# Manual from core
mancore: $(MANSRC)
for mansrc in $^; do $(A2XM) $$mansrc; $(A2XH) $$mansrc; done
# Manuals from types
mantype:
for man in conf/type/*/man.text; do $(A2XM) $$man; $(A2XH) $$man; done
# Move into manpath directories
manmove: mantype mancore
for manpage in $(MANDIR)/*.[1-9] conf/type/*/*.7; do \
cat=$${manpage##*.}; \
mandir=$(MANDIR)/man$$cat; \
mkdir -p $$mandir; \
mv $$manpage $$mandir; \
done
mkdir -p doc/html
mv doc/man/*.html doc/html
for mantype in conf/type/*/man.html; do \
mannew=$$(echo $$mantype | sed -e 's;conf/;cdist-;' -e 's;/;;' -e 's;/man;;');\
mv $$mantype doc/html/$$mannew; \
done
# Reference depends on conf/type/*/man.text - HOWTO with posix make?
$(MANDIR)/cdist-reference.text: manmove $(MANDIR)/cdist-reference.text.sh
$(MANDIR)/cdist-reference.text.sh
$(A2XM) $(MANDIR)/cdist-reference.text
$(A2XH) $(MANDIR)/cdist-reference.text
# Move us to the destination as well
make manmove
clean:
rm -rf doc/man/*.html doc/man/*.[1-9] doc/man/man[1-9] $(MANGENERATED)
rm -f conf/type/*/man.html
rm -rf doc/html
################################################################################
# Developer targets
#
test:
# ubuntu
.rsync lyni@tablett:cdist
# redhat
.rsync nicosc@free.ethz.ch:cdist
# gentoo
.rsync nicosc@ru3.inf.ethz.ch:cdist
#web: manmove
web:
cp README $(WEBDIR)/$(WEBPAGE)
cp -r doc/html/* $(WEBDIR)/$(WEBBASE)/man
cd $(WEBDIR) && git commit -m "cdist update" $(WEBBASE) $(WEBPAGE)
cd $(WEBDIR) && make pub
pub:
git push --mirror
git push --mirror github

4
README
View file

@ -107,7 +107,7 @@ To install cdist, execute the following commands:
cd cdist
export PATH=$PATH:$(pwd -P)/bin
# If you want the manpages (requires asciidoc to be installed)
# If you want the manpages (requires gmake and asciidoc to be installed)
make man
export MANPATH=$MANPATH:$(pwd -P)/doc/man
@ -136,7 +136,7 @@ may vanish at any point. To select a specific branch use
git checkout -b <name> origin/<name>
# Stay on a specific version
git checkout -b 1.3 origin/1.3
git checkout -b 1.4 origin/1.4
### Mirrors

2
bin/cdist-config
View file

@ -19,7 +19,7 @@
#
#
__cdist_version="1.4.0"
__cdist_version="1.4.1"
# Fail if something bogus is going on
set -u

131
build.sh Executable file
View file

@ -0,0 +1,131 @@
#!/bin/sh
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Push a directory to a target, both sides have the same name (i.e. explorers)
# or
# Pull a directory from a target, both sides have the same name (i.e. explorers)
#
# Manpage and HTML
A2XM="a2x -f manpage --no-xmllint"
A2XH="a2x -f xhtml --no-xmllint"
# Developer webbase
WEBDIR=$HOME/niconetz
WEBBASE=software/cdist
WEBPAGE=${WEBBASE}.mdwn
# Documentation
MANDIR=doc/man
MAN1DSTDIR=${MANDIR}/man1
MAN7DSTDIR=${MANDIR}/man7
case "$1" in
man)
set -e
"$0" mandirs
"$0" mantype
"$0" man1
"$0" man7
"$0" manbuild
;;
manbuild)
for src in ${MAN1DSTDIR}/*.text ${MAN7DSTDIR}/*.text; do
echo "Compiling manpage and html for $src"
$A2XM "$src"
$A2XH "$src"
done
;;
mandirs)
# Create destination directories
mkdir -p "${MAN1DSTDIR}" "${MAN7DSTDIR}"
;;
mantype)
for mansrc in ${MAN7TYPESRC}; do
dst="$(echo $mansrc | sed -e 's;conf/;cdist-;' -e 's;/;;' -e 's;/man;;' -e 's;^;doc/man/man7/;')"
ln -sf "../../../$mansrc" "$dst"
done
;;
man1)
for man in cdist-code-run.text cdist-code-run-all.text cdist-config.text \
cdist-dir.text cdist-env.text cdist-explorer-run-global.text \
cdist-deploy-to.text cdist-explorer.text cdist-manifest.text \
cdist-manifest-run.text cdist-manifest-run-init.text \
cdist-manifest-run-all.text cdist-object-explorer-all.text \
cdist-object-gencode.text cdist-object-gencode-all.text \
cdist-remote-explorer-run.text cdist-run-remote.text \
cdist-type-build-emulation.text cdist-type-emulator.text \
cdist-type-template.text
do
ln -sf ../$man ${MAN1DSTDIR}
done
;;
man7)
for man in cdist.text cdist-best-practise.text cdist-hacker.text \
cdist-quickstart.text cdist-reference.text cdist-stages.text \
cdist-type.text
do
ln -sf ../$man ${MAN7DSTDIR}
done
;;
mangen)
${MANDIR}/cdist-reference.text.sh
;;
web)
cp README ${WEBDIR}/${WEBPAGE}
rm -rf ${WEBDIR}/${WEBBASE}/man && mkdir ${WEBDIR}/${WEBBASE}/man
cp ${MAN1DSTDIR}/*.html ${MAN7DSTDIR}/*.html ${WEBDIR}/${WEBBASE}/man
cd ${WEBDIR} && git add ${WEBBASE}/man
cd ${WEBDIR} && git commit -m "cdist update" ${WEBBASE} ${WEBPAGE}
cd ${WEBDIR} && make pub
;;
pub)
git push --mirror
git push --mirror github
;;
clean)
rm -rf "$MAN1DSTDIR" "$MAN7DSTDIR"
rm -f ${MANDIR}/cdist-reference.text
;;
*)
echo ''
echo 'Welcome to cdist!'
echo ''
echo 'Here are the possible targets:'
echo ''
echo ' man: Build manpages (requires Asciidoc)'
echo ' manhtml: Build html-manpages (requires Asciidoc)'
echo ' clean: Remove build stuff'
echo ''
echo ''
echo "Unknown target, \"$1\"" >&2
exit 1
;;
esac

5
conf/type/__apt_ppa/manifest
View file

@ -27,8 +27,3 @@ require="__package/python-software-properties" \
--source "$__type/files/remove-apt-repository" \
--mode 0755
# FIXME: rewrite to use $__self once that exists
# FIXME: only run if something changed. currently working arround this in
# __apt_update_index/gencode-remote
require="__apt_ppa/$__object_id" __apt_update_index

44
conf/type/__apt_update_index/man.text
View file

@ -1,44 +0,0 @@
cdist-type__apt_update_index(7)
==========================
Steven Armstrong <steven-cdist--@--armstrong.cc>
NAME
----
cdist-type__apt_update_index - resynchronize the apt package index
DESCRIPTION
-----------
This cdist type allows you to resynchronize the package index files from their
sources. In other words, it runs `apt-get update`.
FIXME: currently hard coded to work together with the __apt_ppa type
REQUIRED PARAMETERS
-------------------
None.
OPTIONAL PARAMETERS
-------------------
None.
EXAMPLES
--------
--------------------------------------------------------------------------------
__apt_update_index
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist-type(7)
COPYING
-------
Copyright \(C) 2011 Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

0
conf/type/__apt_update_index/singleton
View file

24
conf/type/__apt_update_index/gencode-remote → conf/type/__key_value/explorer/value
View file

@ -17,19 +17,17 @@
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
# Get the current value of key or __NOTSET__ if the key doesn't exist.
#
# BIG FAT FIXME: there must be a better way to do this!
tmpdir=$(mktemp -d)
cleanup() {
rm -rf "$tmpdir"
}
trap cleanup EXIT
key="$(cat "$__object/parameter/key")"
file="$(cat "$__object/parameter/file")"
delimiter="$(cat "$__object/parameter/delimiter")"
should="$tmpdir/should"
is="$tmpdir/is"
find "$__global/object/__apt_ppa" -path "*.cdist/parameter/state" | xargs cat > "$should"
find "$__global/object/__apt_ppa" -path "*.cdist/explorer/state" | xargs cat > "$is"
diff -ru "$is" "$should" || echo apt-get update
awk -F "$delimiter" '
BEGIN { found=0 }
/^'$key'/ { print $2; found=1 }
END { if (found) exit 0; else exit 1 }' "$file" \
|| echo "__NOTSET__"

52
conf/type/__key_value/gencode-remote Executable file
View file

@ -0,0 +1,52 @@
#!/bin/sh
#
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
value_is="$(cat "$__object/explorer/value")"
value_should="$(cat "$__object/parameter/value")"
key="$(cat "$__object/parameter/key")"
file="$(cat "$__object/parameter/file")"
delimiter="$(cat "$__object/parameter/delimiter")"
if [ "$value_is" != "$value_should" ]; then
case "$value_is" in
__NOTSET__)
# add key and value
echo "echo \"${key}${delimiter}${value_should}\" >> \"$file\""
;;
*)
if [ "$value_should" = '__NOTSET__' ]; then
# remove key and value
cat << DONE
sed -i '/^${key}/d' "$file"
DONE
else
# change value
cat << DONE
awk -F "$delimiter" '
/${key}${delimiter}*/{gsub(/$value_is/, $value_should)};{print}' "$file" > "${file}+" \
&& mv "${file}+" "$file"
DONE
fi
;;
esac
fi

59
conf/type/__key_value/man.text Normal file
View file

@ -0,0 +1,59 @@
cdist-type__key_value(7)
========================
Steven Armstrong <steven-cdist--@--armstrong.cc>
NAME
----
cdist-type__key_value - Change property values in files
DESCRIPTION
-----------
This cdist type allows you to change values in a key value based config
file.
REQUIRED PARAMETERS
-------------------
value::
The value for the key. Setting the value to __NOTSET__ will remove the key
from the file.
file::
The file to operate on.
delimiter::
The delimiter which seperates the key from the value.
OPTIONAL PARAMETERS
-------------------
key::
The key to change. Defaults to object_id.
EXAMPLES
--------
--------------------------------------------------------------------------------
# Set the maximum system user id
__key_value SYS_UID_MAX --file /etc/login.defs --value 666 --delimiter ' '
# Same with fancy id
__key_value my-fancy-id --file /etc/login.defs --key SYS_UID_MAX --value 666 \
--delimiter ' '
# Enable packet forwarding
__key_value net.ipv4.ip_forward --file /etc/sysctl.conf --value 1 \
--delimiter '='
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist-type(7)
COPYING
-------
Copyright \(C) 2011 Steven Armstrong. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

26
conf/type/__key_value/manifest Executable file
View file

@ -0,0 +1,26 @@
#!/bin/sh
#
# 2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
if [ -f "$__object/parameter/key" ]; then
key="$(cat "$__object/parameter/key")"
else
echo "$__object_id" > "$__object/parameter/key"
fi

1
conf/type/__key_value/parameter/optional Normal file
View file

@ -0,0 +1 @@
key

3
conf/type/__key_value/parameter/required Normal file
View file

@ -0,0 +1,3 @@
value
file
delimiter

6
doc/changelog
View file

@ -1,3 +1,9 @@
1.4.1: 2011-03-25
* New type __key_value (Steven Armstrong)
* New type __apt_ppa (Steven Armstrong)
* Documentation: Manpage generation cleanup
* Documentation: Manpage fix for __apt_ppa
1.4.0: 2011-03-24
* Add --recursive to __directory
* Move cdist generated stuff to .cdist of an object

19
doc/dev/todo/TAKEME Normal file
View file

@ -0,0 +1,19 @@
UNASSIGNED TODOS
----------------
The following list of todos has not been assigned to any developer.
Feel free to pick one!
CORE
----
Only build manpages if necessary for types as well as for the core!
TYPES
------
Types to be written/extended:
- __ssh-keys (host/user)
- Think about __service - necessary?
- __file_edit
- regexp replace (can probably cover all?)
-> aka sed.
- __cron

26
doc/dev/todo/niconext
View file

@ -1,3 +1,23 @@
Document redefine behaviour
release cdist 1.4.0
perhaps include __self
Core:
- support $__self = relative_type/object_id
- cache
Cache:
Assume you want to configure stuff one host ("monitor node"),
depending on the configuration of other hosts ("cluster nodes").
For instance, the monitor host would like to know,
which hosts are configured with the provider
"apache" and option --start true.
This requires the monitor node to be able to
query all other configured nodes. It can't
ask for all hosts, because cdist does not
know which hosts are configured or may exist.
Example implementation
If cdist keeps ("caches") the configuration of every
node it configures, each new node can query the
cache for existing nodes that acquired the given
configuration.

35
doc/dev/todo/post-1.3
View file

@ -1,35 +0,0 @@
Core:
- allow redefine object, if everything same
- support $__self = relative_type/object_id
- .cdist
- cache
- display changes if object cannot be merged to user
Cache:
Assume you want to configure stuff one host ("monitor node"),
depending on the configuration of other hosts ("cluster nodes").
For instance, the monitor host would like to know,
which hosts are configured with the provider
"apache" and option --start true.
This requires the monitor node to be able to
query all other configured nodes. It can't
ask for all hosts, because cdist does not
know which hosts are configured or may exist.
Example implementation
If cdist keeps ("caches") the configuration of every
node it configures, each new node can query the
cache for existing nodes that acquired the given
configuration.
Types to be written/extended:
- __ssh-keys (host/user)
- __service
- __file_edit
- regexp replace (can probably cover all?)
-> aka sed.
- __cron

2
doc/man/cdist-env.text
View file

@ -32,7 +32,7 @@ in csh variants (csh, tcsh):
eval `./bin/cdist-env`
--------------------------------------------------------------------------------
For bourne shell, the is also a shorter version:
For bourne shell, there is also a shorter version:
--------------------------------------------------------------------------------
. ./bin/cdist-env
--------------------------------------------------------------------------------

73
doc/man/cdist-hacker.text Normal file
View file

@ -0,0 +1,73 @@
cdist-hacker(7)
===============
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-hacker - How to get (stuff) into cdist
WELCOME
-------
Welcome dear hacker! I invite you to a tour of pointers to
get into the usable configuration mangament system, cdist.
The first thing to know is probably that cdist is brought to
you by people who care about how code looks like and who think
twice before merging or implementing a feature: Less features
with good usability are far better than the opposite.
UNDERSTANDING CDIST INTERNALS
-----------------------------
IF you are interested in how cdist internally works, you can open
bin/cdist-config and bin/cdist-deploy-to in your favorite editor and
read the scripts bin/cdist-deploy-to calls. The magnificent HACKERS_README
may be of great help as well.
CODING CONVENTIONS (CORE)
-------------------------
- All variables exported by cdist are prefixed with a double underscore (__)
- All cdist-internal variables are prefixed with __cdist_ and are generally not exported.
HOW TO SUBMIT STUFF FOR INCLUSION INTO UPSTREAM CDIST
-----------------------------------------------------
If you did some cool changes to cdist, which you value as a benefit for
everybody using cdist, you're welcome to propose inclusion into upstream.
There are though some requirements to ensure your changes don't break others
work nor kill the authors brain:
- Code submission must be done via git
- Code to be included should be branched of the upstream "master" branch
- Exception: Bugfixes to a version branch
- Code submissions must be in your master branch
- Exception: If you only want a comment on your code, but not an inclusion.
- Do not add conf/manifest/init - This file should only be touched in your
private branch!
As soon as your work meets these requirements, you can contact me
(IRC, Mailinglist, Phone, RFC 1149) and I'll check your code before
including it.
HOW TO SUBMIT A NEW TYPE
------------------------
Submitting a type works as described above, with the additional requirement
that a corresponding manpage named man.text in asciidoc format with
the manpage-name "cdist-type__NAME" is included in the type directory
AND asciidoc is able to compile it.
SEE ALSO
--------
- cdist(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

1
doc/man/cdist.text
View file

@ -32,6 +32,7 @@ SEE ALSO
- Website: http://www.nico.schottelius.org/cdist/[]
- cdist-best-practise(7)
- cdist-deploy-to(1)
- cdist-hacker(7)
- cdist-manifest(7)
- cdist-quickstart(1)
- cdist-type(7)