stephan/cdist
1
0
Fork 0
forked from ungleich-public/cdist
Code Issues Pull requests Projects Releases Wiki Activity

Merge branch 'master' into type__rbenv

Browse source
This commit is contained in:
Nico Schottelius 2014-02-14 20:13:18 +01:00
commit 490bad7b26
446 changed files with 8668 additions and 2109 deletions
Download patch file Download diff file Expand all files Collapse all files

BIN
docs/dev/factsheet.odt Normal file
View file

Binary file not shown.

BIN
docs/dev/factsheet.pdf Normal file
View file

Binary file not shown.

BIN
docs/dev/logs/2011-11-16.workflow-example.dia Normal file
View file

Binary file not shown.

BIN
docs/dev/logs/2011-11-16.workflow-example.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 20 KiB

BIN
docs/dev/logs/2011-11-17.workflow-example.dia Normal file
View file

Binary file not shown.

72
docs/dev/logs/2012-05-24.preos Normal file
View file

@ -0,0 +1,72 @@
Todo for preos:
get debian installer (?)
x86, amd64
configure sshd
add authorized_keys
output files
tftp: cuni: curl -s "http://http.us.debian.org/debian/dists/$version/main/installer-$arch/current/images/netboot/netboot.tar.gz" | tar xz
iso
http://wiki.debian.org/DebianInstaller/
--------------------------------------------------------------------------------
debootstrap:
[19:33] brief:hack% sudo debootstrap squeeze ./debian-squeeze
[19:30] brief:hack# du -sh .
213M .
install kernel
[19:35] brief:hack# chroot debian-squeeze/ apt-get -y install linux-image-amd64
[19:37] brief:debian-squeeze# ls boot/initrd*
boot/initrd.img-2.6.32-5-amd64
[19:37] brief:debian-squeeze# ls boot/vmlinuz*
boot/vmlinuz-2.6.32-5-amd64
install sshd
[19:37] brief:hack# chroot debian-squeeze/ apt-get -y --force-yes install openssh-server
- connect back?
- generate sshd keys?
--------------------------------------------------------------------------------
initramfs:
find . -print0 | bsdcpio $( (( QUIET )) && echo '--quiet' ) -R 0:0 -0oH newc | $COMPRESSION $COMPRESSION_OPTIONS > "$IMGPATH"
/init for booting
find . -print0 | cpio --null -ov --format=newc | gzip -9 > /boot/my-initramfs.cpio.gz
cpio -H newc -o
find . | cpio -H newc -o > ../initramfs.cpio # <-- this is the actual initramfs
[19:39] brief:debian-squeeze# find . | bsdcpio -H newc -o > ../initramfs.cpio
[19:43] brief:debian-squeeze# xz ../initramfs.cpio
--------------------------------------------------------------------------------
cdrom:
http://tldp.org/HOWTO/Bootdisk-HOWTO/cd-roms.html
--------------------------------------------------------------------------------
[19:34] brief:hack# chroot debian-squeeze/ apt-cache search kernel | grep linux-image
linux-image-2.6.32-5-amd64-dbg - Debugging infos for Linux 2.6.32-5-amd64
linux-image-2.6.32-5-amd64 - Linux 2.6.32 for 64-bit PCs
linux-image-2.6.32-5-openvz-amd64-dbg - Debugging infos for Linux 2.6.32-5-openvz-amd64
linux-image-2.6.32-5-openvz-amd64 - Linux 2.6.32 for 64-bit PCs, OpenVZ support
linux-image-2.6.32-5-vserver-amd64-dbg - Debugging infos for Linux 2.6.32-5-vserver-amd64
linux-image-2.6.32-5-vserver-amd64 - Linux 2.6.32 for 64-bit PCs, Linux-VServer support
linux-image-2.6.32-5-xen-amd64-dbg - Debugging infos for Linux 2.6.32-5-xen-amd64
linux-image-2.6.32-5-xen-amd64 - Linux 2.6.32 for 64-bit PCs, Xen dom0 support
linux-image-2.6-amd64 - Linux 2.6 for 64-bit PCs (meta-package)
linux-image-2.6-openvz-amd64 - Linux 2.6 for 64-bit PCs (meta-package), OpenVZ support
linux-image-2.6-vserver-amd64 - Linux 2.6 for 64-bit PCs (meta-package), Linux-VServer support
linux-image-2.6-xen-amd64 - Linux 2.6 for 64-bit PCs (meta-package), Xen dom0 support
linux-image-amd64 - Linux for 64-bit PCs (meta-package)
linux-image-openvz-amd64 - Linux for 64-bit PCs (meta-package), OpenVZ support
linux-image-vserver-amd64 - Linux for 64-bit PCs (meta-package), Linux-VServer support
linux-image-xen-amd64 - Linux for 64-bit PCs (meta-package), Xen dom0 support
[19:34] brief:hack#
--------------------------------------------------------------------------------

BIN
docs/dev/logs/2012-09-03.dep-ideas.xoj Normal file
View file

Binary file not shown.

BIN
docs/dev/logs/2012-11-02.cdist-vs-centralised-development.xoj Normal file
View file

Binary file not shown.

282
docs/dev/logs/2012-11-15.cdist-sexy-interaction.svg Normal file
View file

@ -0,0 +1,282 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="1052.3622"
height="744.09448"
id="svg2"
version="1.1"
inkscape:version="0.48.3.1 r9886"
sodipodi:docname="cdist-sexy-actions.svg"
inkscape:export-filename="/home/users/nico/cdist-sexy-actions.png"
inkscape:export-xdpi="90"
inkscape:export-ydpi="90">
<defs
id="defs4" />
<sodipodi:namedview
id="base"
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1.0"
inkscape:pageopacity="0.0"
inkscape:pageshadow="2"
inkscape:zoom="0.62488877"
inkscape:cx="526.18109"
inkscape:cy="410.90353"
inkscape:document-units="px"
inkscape:current-layer="layer1"
showgrid="true"
objecttolerance="20"
guidetolerance="20"
inkscape:window-width="1436"
inkscape:window-height="861"
inkscape:window-x="0"
inkscape:window-y="18"
inkscape:window-maximized="0"
gridtolerance="10" />
<metadata
id="metadata7">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title></dc:title>
</cc:Work>
</rdf:RDF>
</metadata>
<g
inkscape:label="Layer 1"
inkscape:groupmode="layer"
id="layer1"
transform="translate(0,-308.2677)">
<g
id="g3791"
transform="translate(-65.448375,393.5891)">
<rect
y="167.46855"
x="222.23357"
height="88.893425"
width="173.74623"
id="rect2985"
style="fill:#cdff13;fill-opacity:0.90416715;stroke:none" />
<text
sodipodi:linespacing="125%"
id="text3755"
y="228.0777"
x="260.61935"
style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
xml:space="preserve"><tspan
y="228.0777"
x="260.61935"
id="tspan3757"
sodipodi:role="line">cdist</tspan></text>
</g>
<g
id="g3802"
transform="translate(-88.702304,-97.993841)">
<rect
y="519.00165"
x="109.09647"
height="90.913727"
width="452.54834"
id="rect3796"
style="fill:#008000;fill-opacity:0.90416715;stroke:none" />
<text
sodipodi:linespacing="125%"
id="text3798"
y="575.57019"
x="171.72594"
style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
xml:space="preserve"><tspan
y="575.57019"
x="171.72594"
id="tspan3800"
sodipodi:role="line">configures hosts</tspan></text>
</g>
<g
id="g3791-5"
transform="translate(258.6201,492.81494)">
<g
id="g3834"
transform="translate(204.05081,-98.994949)">
<rect
style="fill:#cdff13;fill-opacity:0.90416715;stroke:none"
id="rect2985-2"
width="173.74623"
height="88.893425"
x="222.23357"
y="167.46855" />
<text
xml:space="preserve"
style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
x="264.65994"
y="224.03709"
id="text3755-9"
sodipodi:linespacing="125%"><tspan
sodipodi:role="line"
id="tspan3757-2"
x="264.65994"
y="224.03709">sexy</tspan></text>
</g>
</g>
<g
id="g3802-3"
transform="translate(436.48671,-101.85286)">
<g
id="g3866">
<rect
style="fill:#008000;fill-opacity:0.90416715;stroke:none"
id="rect3796-9"
width="452.54834"
height="90.913727"
x="109.09647"
y="519.00165" />
<text
xml:space="preserve"
style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
x="143.44167"
y="573.54987"
id="text3798-6"
sodipodi:linespacing="125%"><tspan
sodipodi:role="line"
id="tspan3800-5"
x="143.44167"
y="573.54987">manages inventory</tspan></text>
</g>
</g>
<g
id="g3802-9"
transform="matrix(0.96624748,0,0,1,-134.02038,188.43537)">
<g
id="g3980"
transform="translate(112.6206,22.403987)">
<rect
style="fill:#822a0e;fill-opacity:1;stroke:none"
id="rect3796-1"
width="319.27777"
height="146.92369"
x="115.72122"
y="536.6048" />
<text
xml:space="preserve"
style="font-size:40px;font-style:normal;font-weight:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
x="269.50381"
y="592.71771"
id="text3798-3"
sodipodi:linespacing="125%"><tspan
sodipodi:role="line"
id="tspan3800-6"
x="269.50381"
y="592.71771">installs hosts</tspan><tspan
sodipodi:role="line"
x="275.871"
y="642.71771"
id="tspan3968">(missing) </tspan></text>
</g>
</g>
<path
style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="m 330.53142,605.54234 354.37306,0.15493"
id="path3924"
inkscape:connector-type="polyline"
inkscape:connector-curvature="0"
inkscape:connection-start="#g3791"
inkscape:connection-start-point="d4"
inkscape:connection-end="#g3791-5"
inkscape:connection-end-point="d4" />
<text
xml:space="preserve"
style="font-size:27.59350204px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
x="447.33084"
y="637.85706"
id="text3926"
sodipodi:linespacing="125%"><tspan
sodipodi:role="line"
id="tspan3928"
x="447.33084"
y="637.85706">interact</tspan></text>
<path
style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="m 244.62052,561.05765 1.06374,-49.13612"
id="path3930"
inkscape:connector-type="polyline"
inkscape:connector-curvature="0"
inkscape:connection-start="#g3791"
inkscape:connection-start-point="d4"
inkscape:connection-end="#g3802"
inkscape:connection-end-point="d4" />
<path
style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;display:inline"
d="m 243.08193,649.95108 -1.26428,97.49307"
id="path3932"
inkscape:connector-type="polyline"
inkscape:connector-curvature="0"
inkscape:connection-start="#g3791"
inkscape:connection-start-point="d4"
inkscape:connection-end="#g3802-9"
inkscape:connection-end-point="d4" />
<path
style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="m 771.80236,561.28854 0.0297,-53.22603"
id="path3934"
inkscape:connector-type="polyline"
inkscape:connector-curvature="0"
inkscape:connection-start="#g3791-5"
inkscape:connection-start-point="d4"
inkscape:connection-end="#g3802-3"
inkscape:connection-end-point="d4" />
<g
id="g3802-9-7"
transform="matrix(0.96624748,0,0,1,323.63894,252.66181)">
<g
id="g3900-3"
transform="matrix(1.0748862,0,0,0.96932859,-4.8574514,97.533037)">
<g
id="g3970"
transform="translate(58.55042,-102.35709)">
<rect
y="519.00165"
x="158.40208"
height="144.96896"
width="431.08368"
id="rect3796-1-1"
style="fill:#822a0e;fill-opacity:1;stroke:none" />
<text
sodipodi:linespacing="125%"
id="text3798-3-7"
y="576.61359"
x="374.46384"
style="font-size:40px;font-style:normal;font-weight:normal;text-align:center;line-height:125%;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
xml:space="preserve"><tspan
y="576.61359"
x="374.46384"
id="tspan3800-6-3"
sodipodi:role="line">visualises inventory</tspan><tspan
id="tspan3966"
y="626.61359"
x="380.83102"
sodipodi:role="line">(missing) </tspan></text>
</g>
</g>
</g>
<path
style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="m 771.03726,650.18197 -1.54888,92.98943"
id="path3986"
inkscape:connector-type="polyline"
inkscape:connector-curvature="0"
inkscape:connection-start="#g3791-5"
inkscape:connection-start-point="d4"
inkscape:connection-end="#g3802-9-7"
inkscape:connection-end-point="d4" />
</g>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 10 KiB

97
docs/dev/logs/2012-12-11.dependencies
View file

@ -8,28 +8,95 @@ __git bar
__package git --state present
require="__git/foo" git bar:
require="__git/foo" __git bar:
__git bar
__git foo
__package git --state present
__package git --state present
__git foo
__package git --state present
__git foo <---|
__package git --state present ---|
-> detects circular dependency
--------------------------------------------------------------------------------
require="__apt_repository/somewhere-where-you-can-find-package-git __git/foo" __git bar
__git bar
__apt_repository somewhere-where-you-can-find-package-git
__git foo
__package git --state present
__package_apt git depends nachher auf __apt_repository
__package git --state present
__git foo <---|
__package git --state present ---|
__apt_repository somewhere-where-you-can-find-package-git
possible solutions:
- __package git does not depend on __git foo (clear dependency)
- because it DOES NOT depend on it!
- but we don't know whether this is always true :-/
- multiple instances of __package git exist, with
- different required BY
- different requirements
- define non inheritent dependencies (?)
- because __git bar really depends only on __git foo
- proposal: introduce require_non_recursive and require_recursive (previously: require)
- recording the source of the dependency and use it to assist resolving (?)
__package git:
inherited require: __git foo von __git bar
__git foo:
inherited autorequire: __package git durch Nutzung
- break out circular references (?)
- if either of both parties is only locked by the other, allow execution of this one?
--------------------------------------------------------------------------------
__package foo
__package_apt foo
__package bar
__package_apt bar
require="__package/foo" __package bar
__package bar
__package foo
__package_apt foo
__package_apt bar
__package foo
--------------------------------------------------------------------------------
__package abc
__package_apt abc
__type1 var1
__type2 FIX
__sometype def
__package abc
__package_apt abc
__type1 var2
__type2 FIX
--------------------------------------------------------------------------------
facts:
- use is different from require="", as use makes USED depend on parent deps
- use = called/defined in the manifest of a type
- it is currently not recorded, where an object gained its requirements and autorequirements
--------------------------------------------------------------------------------
requirements:
- a type should be a black box:
I can require an object and it is ensured,
everything it needs is executed before me.
--------------------------------------------------------------------------------
possible implementations
- requiring it should include everything it USES
--------------------------------------------------------------------------------
solutions:
__type1 DEPENDS but does not use __type2 FIX
--------------------------------------------------------------------------------
Change proposal:
@ -59,14 +126,14 @@ Order:
For __package:
__sometype def
__package abc
__sometype bar
__package foo
__package abc
__package_apt abc
__package foo
__package_apt foo
1) __package_apt/abc (leaf node)
1) __package_apt/foo (leaf node)
2) __package/abc (new leaf node)
2) __package/foo (new leaf node)
3) __sometype/def (new leaf node)
3) __sometype/bar (new leaf node)

BIN
docs/dev/logs/2013-01-03.dependencies-visualised.xoj Normal file
View file

Binary file not shown.

20
docs/dev/logs/2013-01-20.notifications Normal file
View file

@ -0,0 +1,20 @@
Allow cross-type communication
Sending notifications is possible from
- manifest
- gencode-local
- gencode-remote
Sending a notification from an object means writing to the file "notifications" into
its object:
echo mytest >> "$__object/notifications" # a type reports something
Reading / Reacting on notifications works by accessing the file
referred to be "$__notifications". All notifications are prefixed with
the object name ($__object_name) and are appended into this file.
To find out, whether a file was copied, run:
grep __file/etc/passwd:copy "$__notifications"

49
docs/dev/logs/2013-01-20.triggers Normal file
View file

@ -0,0 +1,49 @@
An alternative / complementary approach to notifications: triggers (or actions?)
A type may support various actions by creating files in its subdirectory
"actions". Other types can trigger an action of a different type or object
by calling them (indirectly?):
if grep "__file/etc/nginx/conf.d/.*:copy" "$__notifications"; then
# Call action from a type
cdist trigger __nginx/reload
fi
Not sure whether this approach (calling "actions" of other types) is sane,
as nginx should probably better know if it should be restarted "itself".
--------------------------------------------------------------------------------
Alternate approach:
__nginx_vhost www.some-domain.ch --custom << eof
some custom code for __nginx_vhost inclusion
eof
__nginx_vhost:
manifest:
# __nginx_vhost requires __nginx: creates directories
require"$__object_name" __nginx --require-only
# Do WE or __file ... depend on nginx?
cdist require __nginx
# Create file that contains the giving code
__file /etc/nginx/conf.d/www.some-domain.ch
require="__nginx" __file /etc/nginx/conf.d/www.some-domain.ch
__nginx:
manifest:
__package nginx --state present
__file some-custom-files
gencode-remote:
if first_install or file changed:

34
docs/dev/logs/2013-02-05.debugging-wrong-singleton-type-parameter Normal file
View file

@ -0,0 +1,34 @@
Traceback (most recent call last):
File "/home/users/nico/p/cdist/cdist/bin/../scripts/cdist", line 230, in <module>
commandline()
File "/home/users/nico/p/cdist/cdist/bin/../scripts/cdist", line 104, in commandline
args.func(args)
File "/home/users/nico/p/cdist/cdist/bin/../scripts/cdist", line 107, in config
configinstall(args, mode=cdist.config.Config)
File "/home/users/nico/p/cdist/cdist/bin/../scripts/cdist", line 143, in configinstall
configinstall_onehost(host, args, mode, parallel=False)
File "/home/users/nico/p/cdist/cdist/bin/../scripts/cdist", line 180, in configinstall_onehost
c.deploy_and_cleanup()
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/config_install.py", line 74, in deploy_and_cleanup
self.deploy_to()
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/config_install.py", line 68, in deploy_to
self.stage_prepare()
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/config_install.py", line 91, in stage_prepare
self.context.local.type_path):
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/core/cdist_object.py", line 80, in list_objects
yield cls(cdist.core.CdistType(type_base_path, type_name), object_base_path, object_id=object_id)
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/core/cdist_object.py", line 65, in __init__
self.validate_object_id()
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/core/cdist_object.py", line 130, in validate_object_id
(self.cdist_type.name, self.parameters))
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/util/fsproperty.py", line 210, in __get__
return self._get_attribute(instance, owner)
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/util/fsproperty.py", line 202, in _get_attribute
path = self._get_path(instance)
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/util/fsproperty.py", line 190, in _get_path
path = path(instance)
File "/home/users/nico/oeffentlich/rechner/projekte/cdist/cdist/cdist/core/cdist_object.py", line 192, in <lambda>
parameters = fsproperty.DirectoryDictProperty(lambda obj: os.path.join(obj.base_path, obj.parameter_path))
AttributeError: 'CdistObject' object has no attribute 'parameter_path'
[22:37] brief:~%

15
docs/dev/logs/2013-02-05.weird-notsingleton-type-error Normal file
View file

@ -0,0 +1,15 @@
Hard to find the source bug/problem:
DEBUG: solr.petspremium.de: (emulator) __file//etc/solr/solr.xml: Finished __file/etc/solr/solr.xml/.cdist {'mode': '0644', 'source': '/home/users/nico/.tmp/tmpn27s24/out/conf/type/__petspremium_solr/files/solr/solr.xml'}
+ for file in '$(find . -type f | sed '\''s,^./,,'\'')'
+ dfile=/etc/solr/web.xml
+ reqdir=/etc/solr
+ require=__directory/etc/solr
+ __file /etc/solr/web.xml --source /home/users/nico/.tmp/tmpn27s24/out/conf/type/__petspremium_solr/files/solr/web.xml --mode 0644
DEBUG: solr.petspremium.de: (emulator): /home/users/nico/.tmp/tmpn27s24/out/bin/__file: Namespace(mode='0644', object_id=['/etc/solr/web.xml'], source='/home/users/nico/.tmp/tmpn27s24/out/conf/type/__petspremium_solr/files/solr/web.xml')
DEBUG: solr.petspremium.de: (emulator) __file//etc/solr/web.xml: Recording requirement: __directory/etc/solr
DEBUG: solr.petspremium.de: (emulator) __file//etc/solr/web.xml: Finished __file/etc/solr/web.xml/.cdist {'source': '/home/users/nico/.tmp/tmpn27s24/out/conf/type/__petspremium_solr/files/solr/web.xml', 'mode': '0644'}
ERROR: solr.petspremium.de: Type __directory requires object id (is not a singleton type)
INFO: Total processing time for 1 host(s): 9.756716251373291
ERROR: Failed to deploy to the following hosts: solr.petspremium.de

30
docs/dev/logs/2013-04-03.dependency-discussion Normal file
View file

@ -0,0 +1,30 @@
Steven, Nico
Discussion raised due to proposal from Arkaitz Jimenez
--------------------------------------------------------------------------------
Proposal changes back to cdist behaviour as of 2011 (see commit 61b7b68).
Change would introduce:
- no direct stage based running
- stages only in object (not globally)
- cannot build full dependency list before beginning
- Thus wildcard requirements (require="__file/*") don't work anymore
Accepting this or similar approaches means:
- Drop wildcard requirements (is undocumented anyway)
- Type execution is closed (again)
Furthermore/other points:
- Change cdist to continue run as long as possible
- Don't stop if an object fails
- Record failure, print at the end (and exit non zero)
- Logging
- Catch output of manifest, gencode, code, do not display directly
- Print at the end
- Prefix with hostname as usual!

BIN
docs/dev/logs/2013-04-08.execution-graph.xoj Normal file
View file

Binary file not shown.

77
docs/dev/logs/2013-04-10.discussion Normal file
View file

@ -0,0 +1,77 @@
Steven, Nico (ETH office)
- Try out patch for dependency resolver changing from [nico]
- Add tests
- Cleanup code:
- remove all old resolver parts (including tests!)
- remve wildcard matching pattern code
- Cache: [nobody]
- Should cache be usable by types?
- Should all run outputs be stored?
- Different caches for install and config
- Replace fsproperties with cconfig [steven]
- Maybe support "rerun from previous version (cache)"? [nobody]
- need to include initial manifest(s!)
- copy/link types
- save remote-{exec,copy} parameters (copy or save argument list)
- cdist replay / oldconfig ?
- Support diffing two configurations [nobody]
- cdist diff ?
- Nested Types [both]
- Motivation:
- Put everything related into one directory
- Have a look at it when Arkaitz pushes out pull request
- Implementations:
1) Arkaitz
Folder structure Call Object
__package/ __package abc __package/abc
__package/type/pkg __package.pkg abc __package.pkg/abc
__package/type/pkg/type/green __package.pkg.green abc __package.pkg.green/abc
...
__package.pkg __package.pkg abc __package.pkg/abc
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
=> Need to forbid types with "." in the name!
2) Steven (earlier version)
Folder structure Call Object
__package/.type __package abc __package/abc
__package/pkg/.type __package.pkg abc __package.pkg/abc
__package/pkg/green/.type __package.pkg.green abc __package.pkg.green/abc
- Clashes:
- if __<type>.<name> and __<type> and subtype <name> exist both (in both implementations)
- Install [nobody]
- Merge into master?
- Needs some cleanups
- PreOS [nobody]
- cdist preos / preos-generate
--output=<dir-or-filename>
--arch=[i386|amd64|arm??]
--type=[usb, cdrom/iso, floppy, pxe]
--other-params (?)
- Maybe implement using cdist config indirectly and a type __preos
- Can be:
- Internally only (devs)
- Usable by end users
- Requirements:
- git
- buildchain
- toolchain for target arch
- ...

44
docs/dev/logs/2013-04-12.execution-order Normal file
View file

@ -0,0 +1,44 @@
Old:
- global explores (all)
- initial manifest
- for each object
execute type explorers
execute manifest
continue until all objects (including newly created)
have their type explorers/manifests run
- build dependency tree
- for each object
execute gencode-*
execute code-*
New:
- run all global explorers
- run initial manifest
creates zero or more cdist_objects
- for each cdist_object
if not cdist_object.has_unfullfilled_requirements:
execute type explorers
execute manifest
may create new objects, resulting in autorequirements
# Gained requirements during manifest run
if object.has_auto_requirements():
continue
cdist_object.execute gencode-*
cdist_object.execute code-*
Requirements / Test cases for requirments / resolver:
- omnipotence
-
--------------------------------------------------------------------------------
ERROR: localhost: The following objects could not be resolved: __cdistmarker/singleton requires autorequires ; __directory/etc/sudoers.d requires autorequires ; __file/etc/sudoers.d/nico requires __directory/etc/sudoers.d autorequires ; __file/etc/motd requires autorequires ; __package_pacman/atop requires autorequires ; __package_pacman/screen requires autorequires ; __package_pacman/strace requires autorequires ; __package_pacman/vim requires autorequires ; __package_pacman/zsh requires autorequires ; __package_pacman/lftp requires autorequires ; __package_pacman/nmap requires autorequires ; __package_pacman/ntp requires autorequires ; __package_pacman/rsync requires autorequires ; __package_pacman/rtorrent requires autorequires ; __package_pacman/wget requires autorequires ; __package_pacman/nload requires autorequires ; __package_pacman/iftop requires autorequires ; __package_pacman/mosh requires autorequires ; __package_pacman/git requires autorequires ; __package_pacman/mercurial requires autorequires ; __package_pacman/netcat requires autorequires ; __package_pacman/python-virtualenv requires autorequires ; __package_pacman/wireshark-cli requires autorequires ; __package_pacman/sudo requires autorequires
INFO: Total processing time for 1 host(s): 32.30426597595215
ERROR: Failed to deploy to the following hosts: localhost

340
docs/dev/logs/2013-05-04.ssh Normal file
View file

@ -0,0 +1,340 @@
- analysis of ssh connections for callback
SSH_CLIENT='::1 38502 22'
SSH_CONNECTION='::1 38502 ::1 22'
-> callback possible to source host
[ target host ] <--------------|
| |
| |
| |
| trigger | configuration
| |
v |
[ configuration host ] ----|
- dynamic port allocation for tunneling
[1:37] bento:~% ssh -R 0:localhost:22 localhost
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Allocated port 53161 for remote forward to localhost:22
SSH_AUTH_SOCK=/tmp/ssh-zDCWbUVcUK/agent.30749
SSH_CLIENT='::1 38587 22'
SSH_CONNECTION='::1 38587 ::1 22'
SSH_TTY=/dev/pts/21
- ssh_config:
DynamicForward
LocalForward
RemoteForward
- testing
[1:52] bento:cdist% netstat -anp | grep 56844
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 127.0.0.1:56844 0.0.0.0:* LISTEN -
tcp6 0 0 ::1:56844 :::* LISTEN -
[1:53] bento:cdist%
[1:48] bento:~% ssh -R 0:localhost:22 localhost
Allocated port 56844 for remote forward to localhost:22
...
- chatting
01:42 -!- Irssi: Join to #openssh was synced in 0 secs
01:42 < telmich> good evening
01:43 < telmich> I am trying to make use of remote port forwarding using dynamic port
allocation (port=0) -- I am wondering if there is an easy way to
access the port number on the remote side easily?
01:44 < telmich> background for this question is: I'd like to allow various clients to
login to a configuration server, which then configures the clients by
using the tunnel the client provides for the server to ssh back into
02:07 < BasketCase> telmich: afaik you need to use a tool like ss/netstat/lsof to see what port it has open
- ssh debug
[11:37] bento:~% ssh -R 0:localhost:22 localhost
Allocated port 33562 for remote forward to localhost:22
.. . .x+=:. s
dF @88> z` ^% :8
'88bu. %8P . <k .88
. '*88888bu . .@8Ned8" :888ooo
.udR88N ^"*8888N .@88u .@^%8888" -*8888888
<888'888k beWE "888L ''888E` x88: `)8b. 8888
9888 'Y" 888E 888E 888E 8888N=*8888 8888
9888 888E 888E 888E %8" R88 8888
9888 888E 888F 888E @8Wou 9% .8888Lu=
?8888u../ .888N..888 888& .888888P` ^%888*
"8888P' `"888*"" R888" ` ^"F 'Y"
"P' "" ""
Welcome to a cdist automated system!
Last login: Sat May 4 01:52:46 2013 from localhost.localdomain
debug1: PAM: reinitializing credentials
debug1: permanently_set_uid: 0/0
Environment:
USER=root
LOGNAME=root
HOME=/root
PATH=/usr/bin:/bin:/usr/sbin:/sbin
MAIL=/var/spool/mail/root
SHELL=/bin/bash
SSH_CLIENT=::1 57848 22
SSH_CONNECTION=::1 57848 ::1 22
SSH_TTY=/dev/pts/32
TERM=rxvt-unicode
XDG_SESSION_ID=1
XDG_RUNTIME_DIR=/run/user/1000
XDG_SEAT=seat0
XDG_VTNR=1
SSH_AUTH_SOCK=/tmp/ssh-6j0elukLHA/agent.17260
[root@bento ~]#
[root@bento nico]# /usr/sbin/sshd -D -d
debug1: sshd version OpenSSH_6.2, OpenSSL 1.0.1e 11 Feb 2013
debug1: read PEM private key done: type RSA
debug1: private host key: #0 type 1 RSA
debug1: read PEM private key done: type DSA
debug1: private host key: #1 type 2 DSA
debug1: read PEM private key done: type ECDSA
debug1: private host key: #2 type 3 ECDSA
debug1: rexec_argv[0]='/usr/sbin/sshd'
debug1: rexec_argv[1]='-D'
debug1: rexec_argv[2]='-d'
Set /proc/self/oom_score_adj from 0 to -1000
debug1: Bind to port 22 on 0.0.0.0.
Server listening on 0.0.0.0 port 22.
debug1: Bind to port 22 on ::.
Server listening on :: port 22.
debug1: Server will not fork when running in debugging mode.
debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8
debug1: inetd sockets after dupping: 3, 3
Connection from ::1 port 57848
debug1: Client protocol version 2.0; client software version OpenSSH_6.2
debug1: match: OpenSSH_6.2 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: permanently_set_uid: 99/99 [preauth]
debug1: list_hostkey_types: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256 [preauth]
debug1: SSH2_MSG_KEXINIT sent [preauth]
debug1: SSH2_MSG_KEXINIT received [preauth]
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com zlib@openssh.com [preauth]
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com zlib@openssh.com [preauth]
debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth]
debug1: SSH2_MSG_NEWKEYS sent [preauth]
debug1: expecting SSH2_MSG_NEWKEYS [preauth]
debug1: SSH2_MSG_NEWKEYS received [preauth]
debug1: KEX done [preauth]
debug1: userauth-request for user root service ssh-connection method none [preauth]
debug1: attempt 0 failures 0 [preauth]
debug1: PAM: initializing for "root"
debug1: PAM: setting PAM_RHOST to "localhost.localdomain"
debug1: PAM: setting PAM_TTY to "ssh"
debug1: userauth-request for user root service ssh-connection method publickey [preauth]
debug1: attempt 1 failures 0 [preauth]
debug1: test whether pkalg/pkblob are acceptable [preauth]
debug1: temporarily_use_uid: 0/0 (e=0/0)
debug1: trying public key file /root/.ssh/authorized_keys
debug1: fd 4 clearing O_NONBLOCK
debug1: matching key found: file /root/.ssh/authorized_keys, line 2
Found matching RSA key: 2e:1b:3f:10:01:1d:21:6c:6c:1e:3d:a9:33:ba:3c:f7
debug1: restore_uid: 0/0
Postponed publickey for root from ::1 port 57848 ssh2 [preauth]
debug1: userauth-request for user root service ssh-connection method publickey [preauth]
debug1: attempt 2 failures 0 [preauth]
debug1: temporarily_use_uid: 0/0 (e=0/0)
debug1: trying public key file /root/.ssh/authorized_keys
debug1: fd 4 clearing O_NONBLOCK
debug1: matching key found: file /root/.ssh/authorized_keys, line 2
Found matching RSA key: 2e:1b:3f:10:01:1d:21:6c:6c:1e:3d:a9:33:ba:3c:f7
debug1: restore_uid: 0/0
debug1: ssh_rsa_verify: signature correct
debug1: do_pam_account: called
Accepted publickey for root from ::1 port 57848 ssh2
debug1: monitor_child_preauth: root has been authenticated by privileged process
debug1: Enabling compression at level 6. [preauth]
debug1: monitor_read_log: child log fd closed
debug1: PAM: establishing credentials
debug1: Entering interactive session for SSH2.
debug1: server_init_dispatch_20
debug1: server_input_global_request: rtype tcpip-forward want_reply 1
debug1: server_input_global_request: tcpip-forward listen localhost port 0
debug1: Local forwarding listening on ::1 port 0.
debug1: Allocated listen port 33562
debug1: channel 0: new [port listener]
debug1: Local forwarding listening on 127.0.0.1 port 33562.
debug1: channel 1: new [port listener]
debug1: server_input_channel_open: ctype session rchan 0 win 1048576 max 16384
debug1: input_session_request
debug1: channel 2: new [server-session]
debug1: session_new: session 0
debug1: session_open: channel 2
debug1: session_open: session 0: link with channel 2
debug1: server_input_channel_open: confirm session
debug1: server_input_global_request: rtype no-more-sessions@openssh.com want_reply 0
debug1: server_input_channel_req: channel 2 request auth-agent-req@openssh.com reply 0
debug1: session_by_channel: session 0 channel 2
debug1: session_input_channel_req: session 0 req auth-agent-req@openssh.com
debug1: temporarily_use_uid: 0/0 (e=0/0)
debug1: restore_uid: 0/0
debug1: channel 3: new [auth socket]
debug1: server_input_channel_req: channel 2 request pty-req reply 1
debug1: session_by_channel: session 0 channel 2
debug1: session_input_channel_req: session 0 req pty-req
debug1: Allocating pty.
debug1: session_pty_req: session 0 alloc /dev/pts/32
debug1: server_input_channel_req: channel 2 request shell reply 1
debug1: session_by_channel: session 0 channel 2
debug1: session_input_channel_req: session 0 req shell
debug1: Setting controlling tty using TIOCSCTTY.
--------------------------------------------------------------------------------
debug1: server_input_global_request: rtype tcpip-forward want_reply 1
debug1: server_input_global_request: tcpip-forward listen localhost port 0
debug1: Local forwarding listening on ::1 port 0.
debug1: Allocated listen port 33562
debug1: channel 0: new [port listener]
debug1: Local forwarding listening on 127.0.0.1 port 33562.
[11:49] bento:openssh-6.2p1% grep "Allocated listen port" -r .
./channels.c: debug("Allocated listen port %d",
[11:49] bento:openssh-6.2p1%
--------------------------------------------------------------------------------
[11:54] bento:~% ssh -R 0:localhost:22 -R 0:192.168.1.1:33 localhost
Allocated port 48392 for remote forward to localhost:22
Allocated port 37515 for remote forward to 192.168.1.1:33
debug1: server_input_global_request: rtype tcpip-forward want_reply 1
debug1: server_input_global_request: tcpip-forward listen localhost port 0
debug1: Local forwarding listening on ::1 port 0.
debug1: Allocated listen port 48392
debug1: channel 0: new [port listener]
debug1: Local forwarding listening on 127.0.0.1 port 48392.
debug1: channel 1: new [port listener]
debug1: server_input_global_request: rtype tcpip-forward want_reply 1
debug1: server_input_global_request: tcpip-forward listen localhost port 0
debug1: Local forwarding listening on ::1 port 0.
debug1: Allocated listen port 37515
debug1: channel 2: new [port listener]
debug1: Local forwarding listening on 127.0.0.1 port 37515.
debug1: channel 3: new [port listener]
debug1: server_input_channel_open: ctype session rchan 0 win 1048576 max 16384
debug1: input_session_request
debug1: channel 4: new [server-session]
debug1: session_new: session 0
debug1: session_open: channel 4
debug1: session_open: session 0: link with channel 4
debug1: Local forwarding listening on ::1 port 5555.
debug1: channel 0: new [port listener]
debug1: Local forwarding listening on 127.0.0.1 port 5555.
debug1: channel 1: new [port listener]
debug1: server_input_global_request: rtype tcpip-forward want_reply 1
debug1: server_input_global_request: tcpip-forward listen localhost port 4444
debug1: Local forwarding listening on ::1 port 4444.
debug1: channel 2: new [port listener]
debug1: Local forwarding listening on 127.0.0.1 port 4444.
debug1: channel 3: new [port listener]
debug1: server_input_channel_open: ctype session rchan 0 win 1048576 max 16384
debug1: input_session_request
debug1: channel 4: new [server-session]
debug1: session_new: session 0
debug1: session_open: channel 4
--------------------------------------------------------------------------------
[12:06] bento:openssh-6.2p1% grep SSH_CONNECTION -r *
audit-bsm.c: case SSH_CONNECTION_CLOSE:
audit.c: {SSH_CONNECTION_CLOSE, "CONNECTION_CLOSE"},
audit.c: {SSH_CONNECTION_ABANDON, "CONNECTION_ABANDON"},
audit.h: SSH_CONNECTION_CLOSE, /* closed after attempting auth or session */
audit.h: SSH_CONNECTION_ABANDON, /* closed without completing auth */
audit-linux.c: case SSH_CONNECTION_CLOSE:
monitor.c: case SSH_CONNECTION_CLOSE:
regress/proxy-connect.sh: SSH_CONNECTION=`${SSH} -$p -F $OBJ/ssh_proxy 999.999.999.999 'echo $SSH_CONNECTION'`
regress/proxy-connect.sh: if [ "$SSH_CONNECTION" != "UNKNOWN 65535 UNKNOWN 65535" ]; then
regress/proxy-connect.sh: fail "bad SSH_CONNECTION"
session.c: child_set_env(&env, &envsize, "SSH_CONNECTION", buf);
sftp-server.c: if ((cp = getenv("SSH_CONNECTION")) != NULL) {
sftp-server.c: error("Malformed SSH_CONNECTION variable: \"%s\"",
sftp-server.c: getenv("SSH_CONNECTION"));
ssh.0: SSH_CONNECTION Identifies the client and server ends of the
ssh.1:.It Ev SSH_CONNECTION
sshd.c: PRIVSEP(audit_event(SSH_CONNECTION_CLOSE));
sshd.c: audit_event(SSH_CONNECTION_ABANDON);
[12:06] bento:openssh-6.2p1%
--------------------------------------------------------------------------------
debug1: Remote connections from LOCALHOST:5555 forwarded to local address localhost:22
--------------------------------------------------------------------------------
[12:42] bento:openssh-6.2p1% grep tcpip-forward *
channels.c: packet_put_cstring("tcpip-forward");
channels.c: packet_put_cstring("cancel-tcpip-forward");
Binary file channels.o matches
grep: contrib: Is a directory
Binary file libssh.a matches
grep: openbsd-compat: Is a directory
grep: regress: Is a directory
grep: scard: Is a directory
serverloop.c: if (strcmp(rtype, "tcpip-forward") == 0) {
serverloop.c: debug("server_input_global_request: tcpip-forward listen %s port %d",
serverloop.c: } else if (strcmp(rtype, "cancel-tcpip-forward") == 0) {
serverloop.c: debug("%s: cancel-tcpip-forward addr %s port %d", __func__,
Binary file serverloop.o matches
Binary file ssh matches
Binary file sshd matches
Binary file ssh-keyscan matches
Binary file ssh-keysign matches
[12:42] bento:openssh-6.2p1%
--------------------------------------------------------------------------------
Channel information for (remote) forwarding:
c = channel_new("port listener", type, sock, sock, -1,
CHAN_TCP_WINDOW_DEFAULT, CHAN_TCP_PACKET_DEFAULT,
0, "port listener", 1);
c->path = xstrdup(host);
c->host_port = port_to_connect;
c->listening_addr = addr == NULL ? NULL : xstrdup(addr);
if (listen_port == 0 && allocated_listen_port != NULL &&
!(datafellows & SSH_BUG_DYNAMIC_RPORT))
c->listening_port = *allocated_listen_port;
else
c->listening_port = listen_port;
--------------------------------------------------------------------------------
Code handling remote forwarding in the client:
- ssh_init_forwarding
- channel_request_remote_forwarding
Sends hostname + port for ssh1 only - not send in ssh2
Code handling forwarding / listening in the server:
- channel_new: creates channels, 2 per listener (ipv4/ipv6)
- channels_alloc contains number of channels
- server_input_global_request
Reads only listen port, not hostname/port to connect to
- channel_setup_remote_fwd_listener
- channel_setup_remote_fwd_listener
Code handling environment variables:
- child_set_env
1236 child_set_env(&env, &envsize, "SSH_CONNECTION", buf);

40
docs/dev/logs/2013-05-17.ssh-callback-socat Normal file
View file

@ -0,0 +1,40 @@
start ssh
to controlhost,
bind other side to
localhost:22
targethost ------> ssh ------> controlhost
|
|
socat: connect stdin/stdout to ?
start cdist with port information
added
Use
socat
--------------------------------------------------------------------------------
TCP:<host>:<port>
Connects to <port> [TCP service] on <host> [IP address] using TCP/IP version 4 or 6 depending on address specifi‐
cation, name resolution, or option pf.
Option groups: FD,SOCKET,IP4,IP6,TCP,RETRY
Useful options: crnl, bind, pf, connect-timeout, tos, mtudiscover, mss, nodelay, nonblock, sourceport, retry,
readbytes
See also: TCP4, TCP6, TCP-LISTEN, UDP, SCTP-CONNECT, UNIX-CONNECT
forever
--------------------------------------------------------------------------------
[root@nico-dev-vm-snr01 yum.repos.d]# ps aux | grep socat
nico 25035 0.0 0.0 41640 1524 ? Ss 13:27 0:00 socat - TCP-LISTEN:1234
root 25037 0.0 0.0 103240 836 pts/1 S+ 13:27 0:00 grep socat
[root@nico-dev-vm-snr01 yum.repos.d]#
--------------------------------------------------------------------------------

38
docs/dev/logs/2013-07-12.release Normal file
View file

@ -0,0 +1,38 @@
- setup release date in docs/changelog to today manually
- checkout master branch
[
x check if date is correct in docs/changelog
x ensure all unittests work
- requires (wrong/outdated) versionfile!
x compile manpages
x compile speeches
]
[
x add manpages to website repo
x add speeches to website repo
x rsync cdist docs to website repo & add to website repo
x create blog entry & add to website repo
]
x upload website
x fix latest link for manpages
x send mail to mailinglist -> also requires git tag & git release
x should also require web-release including blog!
- create PKGBUILD for archlinux release
x create git tag / read description
t if necessary create version branch
x change to version branch and merge tag!
x update git repos
x update website from repo
x create release on freecode
x create versionfile
x make pypi release
x make archlinux release
manual last steps:
- announce on linkedin
- announce on twitter

56
docs/dev/logs/2013-07-25.source-error-does-not-stop-cdist Normal file
View file

@ -0,0 +1,56 @@
Symptom:
running something in a manifest and that fails does not exist
the cdist run
Analysis:
Find out what the shell does:
[23:56] bento:testshell% cat a.sh
# source something that fails
. b.sh
[23:57] bento:testshell% cat b.sh
nosuchcommand
[23:57] bento:testshell% sh -e a.sh
a.sh: 2: .: b.sh: not found
[23:57] bento:testshell% echo $?
2
-> exit 2 -> looks good
Find out what the python does:
[23:57] bento:testshell% python3
Python 3.3.2 (default, May 21 2013, 15:40:45)
[GCC 4.8.0 20130502 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.check_call(["/bin/sh", "-e", "a.sh"])
a.sh: 2: .: b.sh: not found
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.3/subprocess.py", line 544, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/sh', '-e', 'a.sh']' returned non-zero exit status 2
>>>
Conclusion:
Manifests that execute (!) other shell scripts does
not necessarily give the -e flag to the other script
-> called script can have failures, but exit 0
if something the last thing executed does exit 0!
Solution:
Instead of doing stuff like
"$__manifest/special"
use
sh -e "$__manifest/special"
or source the script:
. "$__manifest/special"
(runs the script in the same namespace/process as everything in the
calling script)

2
docs/dev/logs/2013-08-07.shell Normal file
View file

@ -0,0 +1,2 @@
What about having a cdist shell to have a shell with all available types?
Let's give it a try!

28
docs/dev/logs/2013-08-12.release Normal file
View file

@ -0,0 +1,28 @@
- already on 2.3.0-1 during release
- user bug: there should be no changes / commits during a release process
hard linking docs/man/man7/cdist-type__user.7 -> cdist-2.3.0-1-g8192c2c/docs/man/man7
hard linking docs/man/man7/cdist-type__user.html -> cdist-2.3.0-1-g8192c2c/docs/man/man7
hard linking docs/man/man7/cdist-type__user_groups.7 -> cdist-2.3.0-1-g8192c2c/docs/man/man7
hard linking docs/man/man7/cdist-type__user_groups.html -> cdist-2.3.0-1-g8192c2c/docs/man/man7
hard linking scripts/cdist -> cdist-2.3.0-1-g8192c2c/scripts
creating dist
Creating tar archive
removing 'cdist-2.3.0-1-g8192c2c' (and everything under it)
running upload
Submitting dist/cdist-2.3.0-1-g8192c2c.tar.gz to http://pypi.python.org/pypi
Server response (200): OK
touch .lock-pypi
./PKGBUILD.in 2.3.0
==> Retrieving sources...
-> Downloading cdist-2.3.0.tar.gz...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 404 Not Found
==> ERROR: Failure while downloading cdist-2.3.0.tar.gz
Aborting...
make: *** [PKGBUILD] Error 1
[12:38] bento:cdist%

17
docs/dev/logs/2013-08-18.cache-enhancement Normal file
View file

@ -0,0 +1,17 @@
- always save cache = outdir
- even if run aborts (for debugging)
- add a state flag
- save cache in a date based directory
- also add support for a per-host pidfile
- allow user to specify cache dir - to give
full flexibility
- drop context - it is a very small unecessary wrapper
- maye introduce cdist.log instead!
- replace out_path with out_base
- directory under which all the subdirectories are
created
-> by default ~/.cdist/run
-> out_base_path
- drop support for deprecated environment variables
__cdist_out_dir
__cdist_remote_out_dir

BIN
docs/dev/logs/2013-08-27-notifications.xoj Normal file
View file

Binary file not shown.

5
docs/dev/logs/2013-08-28.release Normal file
View file

@ -0,0 +1,5 @@
- release process releases pypi from something
that is git describe based, not changelog based...
- git describe should equal changelog, but may be
inconsistent due to branch merging!

34
docs/dev/logs/2013-09-05.test-cp Normal file
View file

@ -0,0 +1,34 @@
Test copy copys symlinks - making real files would be better
Test how to use cp:
[12:54] bento:~% cd test
[12:54] bento:test% ln -s /etc/passwd
[12:54] bento:test% cd ..
[12:54] bento:~% cp -r test test2
[12:54] bento:~% ls -lh test2/
total 4.0K
lrwxrwxrwx 1 nico nico 11 Sep 5 12:54 passwd -> /etc/passwd
[12:54] bento:~% rm -rf test2/
--------------------------------------------------------------------------------
[12:54] bento:~% ls -lh test2/
total 4.0K
lrwxrwxrwx 1 nico nico 11 Sep 5 12:54 passwd -> /etc/passwd
[12:54] bento:~% rm -rf test2/
[12:54] bento:~% cp -r --dereference test test2
[12:56] bento:~% ls -l test2/
total 4
-rw------- 1 nico nico 960 Sep 5 12:56 passwd
[12:56] bento:~%
--------------------------------------------------------------------------------
[13:04] bento:cdist% git describe
2.3.2
[13:09] bento:cdist% vi MANIFEST.in
[13:09] bento:cdist% vi MANIFEST
[13:09] bento:cdist% vi setup.py
[13:09] bento:cdist% cat cdist/version.py
VERSION = "2.3.1-34-g7acf041"
[13:10] bento:cdist%

13
docs/dev/logs/2013-10-03.ossawards/infos Normal file
View file

@ -0,0 +1,13 @@
Required for the ossawards until 2013-10-06:
- all source code
- licenses GPLv3
- installation instructions,
- On Linux do the following:
- pip install
-
- necessary documents and
- a demo video onto our web hard.
- installation
- cdist via cdist
- presentation
- build from existing ones (?)

6
docs/dev/logs/2013-10-29.__line Normal file
View file

@ -0,0 +1,6 @@
- fix handling of fixed strings
- ensure special characters are not interpreted
[12:18] bento:~% cat /etc/bash.bashrc
cat: /etc/bash.bashrc: Permission denied
[12:19] bento:~%

50
docs/dev/logs/2013-11-25.notifications Normal file
View file

@ -0,0 +1,50 @@
Follow up from 2013-01-20:
- (re-)create message file per object?
- yes, but do not necessarily save in object space
- save $anywhere
- object_run
- current notifications are imported into a file available at $__messages_in
- after object run, everything that has been written to $__messages_out is merged into the $__messages file
- functions:
self.explorer.run_global_explorers(self.local.global_explorer_out_path)
self.manifest.run_initial_manifest(self.local.initial_manifest)
self.local.run_script(initial_manifest, env=self.env_initial_manifest(initial_manifest))
self.explorer.run_type_explorers(cdist_object)
self.manifest.run_type_manifest(cdist_object)
self.local.run_script(type_manifest, env=self.env_type_manifest(cdist_object))
self.code.run_gencode_local(cdist_object)
self.local.run_script(script, env=env, return_output=True)
self.code.run_gencode_remote(cdist_object)
self.local.run_script(script, env=env, return_output=True)
- message support in ...
- initialmanifest - yes
- explorer - no
- only locally - yes
- how to use notification / messaging in cdist
- can be used in all local scripts:
- initial manifest
- type manifest
- type gencode-*
- order of object exeution is random or as you requested using require=""
- example use:
__file/gencode-local:
if [ "$local_cksum" != "$remote_cksum" ]; then
echo "$__remote_copy" "$source" "${__target_host}:${destination}"
echo "copy" >> "$__messages_out"
fi
__nginx/manifest:
__file /etc/nginx/sites-enabled/myfile --source "$__type/files/nginx-config"
__nginx/gencode-remote:
if grep -q "__file/etc/nginx/sites-enabled/myfile:copy" "$__messages_in"; then
echo /etc/init.d/nginx restart
fi

6
docs/dev/logs/2013-12-12.discussion Normal file
View file

@ -0,0 +1,6 @@
With Steven
- Implement environments
- for configuring "anything" including switches
- can disable / use other global explorers
- 98% of our framework is generic and can be used for any applikation

44
docs/dev/logs/2014-01-20.environments Normal file
View file

@ -0,0 +1,44 @@
raw quote from irc
16:00 < sar> telmich: btw, ich denke nicht dass man install schon zu gross bewerben
sollte
16:00 < telmich> sar: ack
16:00 < sar> telmich: imho sollten wir erst die cdist environments implementieren,
install waere dann eines davon
16:00 < sar> config ein anderes
16:01 < sar> foobar noch ein anderes
16:01 < sar> es macht einfach keinen sinn auf type ebene install vs nicht-install zu
unterscheiden
16:02 < telmich> sar: environments sind bei mir noch nicht ganz im gehirn (ganicht?)
angelangt - hast du (nochmal?) kurz eine idee, was du damit meinst?
16:02 < sar> telmich: wenn man cdist anschaut, dann macht es eigentlich folgendes:
16:03 < sar> - definiere objekte mit hilfe von types
16:03 < sar> - deps zwischen objekten
16:03 < sar> - queue von objekten abarbeiten und auf $etwas anwenden
16:03 < sar> das ist alles
16:04 < sar> telmich: das ist eigentlich ziemlich generisch
16:04 < sar> telmich: fuer mich wuerde es sich hier anbieten das auch so zu
abstrahieren
16:05 < sar> telmich: ein environment (nenn das mal so weil kein besserer name zzt)
koennte das wie $objekt auf $etwas bestimmen
16:05 < sar> telmich: und auch was fuer types es in diesem environment gibt
16:06 < telmich> sar: klingt gut
16:06 < sar> telmich: e.g. es gibt ein environment fuer config -> was wir jetzt haben
16:06 < sar> eins fuer install -> += was im install branch ist (nur die types), den
python code brauchts nacher nicht mehr
16:07 < sar> eins fuer cisco-switch -> hat types um mit cisco zu spielen
16:07 < sar> usw
16:07 < sar> ein environment hat auch eigene remote-{exec,copy} scripte
16:08 < sar> und vielleicht globale explorer, vielleicht auch nicht
16:08 < sar> ein enviroment ist ein cconfig style directory
16:09 < sar> wo man cdist drueber laufen laesst
16:09 < sar> so was in der art
16:13 < telmich> sar: hmmja...klingt gut
16:15 < telmich> vielleicht etwas für cdist 4 oder cdist 5 :-)
16:15 < telmich> aber ich denke auf jeden fall als grundgedanke behaltbar
16:16 < telmich> ok für dich, wenn ich den chat ins docs/dev/logs kopiere als
erinnerungs
16:16 < telmich> s/s$/?/?
16:16 < telmich> s/?$//
16:20 < sar> klar

86
docs/dev/logs/2014-02-13.discussion Normal file
View file

@ -0,0 +1,86 @@
With Steven
t marker .cdist breaks
- use random marker that starts with .cdist-
- has fixed number of following characters (like 6 or 10)
- write marker name to $__global/marker
- export $__global/marker path as $__marker
- document variable in cdist-reference
- also document the pattern how the marker is built
so that other people may be able to dig into the structure
from outside
t save method
- in $__global/method
- values
- config
- install
- document path and description in cdist-reference
t save whole runtime in cache
- missing items
- initial manifest may be specified on commandline
- always save the initial manifest to $__global/initial-manifest
- currently it is a lost tempfile
- remote exec / remote copy
- save to $__global/remote_exec
- save to $__global/remote_copy
- stdout and stderr of everything
- need to implement Steven's patch of stderr/stdout capturing
- exit code of cdist
- if it is complete, we can use it for replay / reconfigure
- new idea: replay / reconfig / reinstall
- --from-cache?
t stderr/stdout
- capture all messages
- prefix with target_host
- implementation exists in one of Steven's branches
- ping steven for updated pull request
x on error dump all information about the failing object
- where created
- stderr
- stdout
- parameter (+values)
- everything known [tm]
t multiple versions of cache
- see #298
t absolute path of types, explorer
- resolve instead of using the temporary link name
- #305
t report command
- from cache?
- #306
t add session to "run directories"
- instead of /var/lib/cdist (remote)
- instead of static dir in cache
- same id remote and local
- maybe timestamp
- in or excluding the pid of cdist?
- cache
- also save when cdist fails
- save exit code
- be able to restore config
- new command: cdist clean-cache
- --since
- --keep-versions --keep-lala $num
- cdist 4.0.0pre2
- cleanup in preos
- logging for types
cdist log ...?
- cdist logserver
- $__global/log.socket
- fifo?
echo into logpipe?