nico/www.nico.schottelius.org
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

add blog entry that describes how to recursively transfer files

Browse source 
Signed-off-by: Nico Schottelius <nico@brief.schottelius.org>
This commit is contained in:
Nico Schottelius 2012-11-22 20:21:28 +01:00
parent 5a3b791a6e
commit 64561acccb
1 changed files with 88 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

88
blog/cdist-transfer-files-recursively.mdwn Normal file
View file

@ -0,0 +1,88 @@
[[!meta title="Cdist: How to copy a folder recursively"]]
## Introduction
This article describes one solution to transfer a folder
and all of its contents recursively with [[cdist homepage|software/cdist]]
to target hosts. I am motivated to do so, because I want to have one
central place to configure the tftproot that I may use on a variety
of KVM hosts.
Traditionally, it is not an easy job to handle recursive transfer correctly
and efficiently in a configuration management system. Using a sophisticated
tool like [rsync](http://rsync.samba.org/) or
[unison](http://www.cis.upenn.edu/~bcpierce/unison/) makes life way easier.
## Copying the files recursively
Cdist knows about the types **\_\_file** and **\_\_directory** for
file transfer and directory management.
The type **\_\_nico\_tftp\_root**,
which can be found in the
[cdist-nico git repository](http://git.schottelius.org/?p=cdist-nico)
(below **cdist/conf/type**) recursively copies all files it has to
the folder **/home/service/tftp**. Only when a file is changed, it
is transferred again (the **\_\_file** type takes care of this).
## The manifest
In cdist, a manifest of a type defines, which other types to use.
A manifest file is essentially shell code, that can call other
cdist types.
To accomplish the task, first of all the base directory is created
on the remote site:
basedir=/home/services/tftp
__directory "$basedir" --mode 0755
Afterwards, I change into the source directory
and find all files. Cdist exports the
variable "__type" to access the folder in which the type is stored.
cd "$__type/files"
for file in $(find . | sed 's;./;;' | grep -v '^\.$' ); do
The grep command is needed, to skip the current directory, that is returned
by find.
Now, for every file I determine the remote file name. Furthermore
dependencies to the required directories are setup:
You can **require** another type to be run before a type, by setting
up the **require** environment variable (this will be changed in cdist
2.1. and replaced in 2.2, but there is still some time until this is released).
The remote name is constructed by this line:
name="$basedir/$file"
And the requirement is setup using this line:
# Require the previous directory in the path
export require="__directory/${name%/*}"
The shell (!) knows about string manipulation: ${variablename%/*} replaces
the shortest matching suffix that equals "/*". And thus the previous
statement removes the last part of the path (also known as dirname).
If the file found by find is a file, we call the \_\_file type,
if the file is actually a directory, the \_\_directory type is called:
if [ -d "$file" ]; then
__directory "$name" --mode 0755
else
__file "$basedir/$file" --source "$__type/files/$file" \
--mode 0644
fi
done
And that's it - a full recursive copy with just a bunch of lines.
## Further Reading
* [[cdist|software/cdist]]
* [cdist-nico git repository](http://git.schottelius.org/?p=cdist-nico)
* [manifest of __nico_tftp_root](http://git.schottelius.org/?p=cdist-nico;a=blob;f=cdist/conf/type/__nico_tftp_root/manifest;h=b312210d878b30e5871751d62cea14172f63c756;hb=HEAD)
[[!tag cdist config unix]]