saz/ccollect
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Some documentation polishment

Browse source
This commit is contained in:
Nico Schottelius 2006-02-06 00:46:50 +01:00 committed by Nico Schottelius
parent decae37d8d
commit 68c83cf50b
2 changed files with 35 additions and 18 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

13
doc/ccollect.html
View file

@ -287,8 +287,8 @@ your farm of webservers <strong>to</strong> a backup host somewhere else. One of
your webservers gets compromised, then your backup server will be compromised, your webservers gets compromised, then your backup server will be compromised,
too. Think of it the other way round: The backup server (now behind a too. Think of it the other way round: The backup server (now behind a
firewall using NAT and strong firewall rules) connects to the firewall using NAT and strong firewall rules) connects to the
webservers and pulls the data to it. If someone gets access to the webservers and pulls the data <strong>from</strong> them. If someone gets access to one
webserver, the person will perhaps not even see your machine. If of the webservers, the person will perhaps not even see your machine. If
he/she sees that there are connections from a host to the compromised he/she sees that there are connections from a host to the compromised
machine, he/she will not be able to login to the backup machine. machine, he/she will not be able to login to the backup machine.
All other backups are still secure.</p> All other backups are still secure.</p>
@ -392,7 +392,7 @@ This number defines how many versions of this intervall to keep.</p>
<em>/etc/ccollect/defaults/<tt>pre_exec</tt></em> (same with <tt>post_exec</tt>), <tt>ccollect</tt> <em>/etc/ccollect/defaults/<tt>pre_exec</tt></em> (same with <tt>post_exec</tt>), <tt>ccollect</tt>
will start <tt>pre_exec</tt> before the whole backup process and will start <tt>pre_exec</tt> before the whole backup process and
<tt>post_exec</tt> after backup of all sources is done.</p> <tt>post_exec</tt> after backup of all sources is done.</p>
<p>The following Example describes how to report free disk space in <p>The following example describes how to report free disk space in
human readable format before and after the whole backup process:</p> human readable format before and after the whole backup process:</p>
<div class="listingblock"> <div class="listingblock">
<div class="content"> <div class="content">
@ -555,6 +555,9 @@ seperated.</p>
<pre><tt> [11:36] zaphodbeeblebrox:ccollect-0.2% ls -l conf/sources/testsource2/destination <pre><tt> [11:36] zaphodbeeblebrox:ccollect-0.2% ls -l conf/sources/testsource2/destination
lrwxrwxrwx 1 nico users 20 2005-11-17 16:44 conf/sources/testsource2/destination -&gt; /home/nico/backupdir</tt></pre> lrwxrwxrwx 1 nico users 20 2005-11-17 16:44 conf/sources/testsource2/destination -&gt; /home/nico/backupdir</tt></pre>
</div></div> </div></div>
<p>To speak truth, this is not fully correct. <tt>ccollect</tt> will also backup
your data, if <tt>destination</tt> is a directory. But do you really want to have
a backup below /etc?</p>
<h4>4.3.7. Detailled description of "intervalls/"</h4> <h4>4.3.7. Detailled description of "intervalls/"</h4>
<p>When you create a subdirectory <tt>intervalls/</tt> within your source configuration <p>When you create a subdirectory <tt>intervalls/</tt> within your source configuration
directory, you can specify individiual intervalls for this specific source. directory, you can specify individiual intervalls for this specific source.
@ -828,13 +831,13 @@ du (GNU coreutils) 5.93
1.5G hydrogenium/durcheinander.2006-01-25-23:18.31328 1.5G hydrogenium/durcheinander.2006-01-25-23:18.31328
200M hydrogenium/durcheinander.2006-01-26-00:11.3332</tt></pre> 200M hydrogenium/durcheinander.2006-01-26-00:11.3332</tt></pre>
</div></div> </div></div>
<p>In the second report the sizes include the space the inodes of <p>In the second report (without -l) the sizes include the space the inodes of
the hardlinks allocate.</p> the hardlinks allocate.</p>
</div> </div>
<div id="footer"> <div id="footer">
<div id="footer-text"> <div id="footer-text">
Version 0.3.1<br /> Version 0.3.1<br />
Last updated 05-Feb-2006 18:56:42 CEST Last updated 06-Feb-2006 00:46:09 CEST
</div> </div>
</div> </div>
</body> </body>

40
doc/ccollect.text
View file

@ -26,8 +26,8 @@ your farm of webservers *to* a backup host somewhere else. One of
your webservers gets compromised, then your backup server will be compromised, your webservers gets compromised, then your backup server will be compromised,
too. Think of it the other way round: The backup server (now behind a too. Think of it the other way round: The backup server (now behind a
firewall using NAT and strong firewall rules) connects to the firewall using NAT and strong firewall rules) connects to the
webservers and pulls the data *from* them. If someone gets access to the webservers and pulls the data *from* them. If someone gets access to one
webserver, the person will perhaps not even see your machine. If of the webservers, the person will perhaps not even see your machine. If
he/she sees that there are connections from a host to the compromised he/she sees that there are connections from a host to the compromised
machine, he/she will not be able to login to the backup machine. machine, he/she will not be able to login to the backup machine.
All other backups are still secure. All other backups are still secure.
@ -38,7 +38,6 @@ Requirements
Installing ccollect Installing ccollect
~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~
For the installation, you need at least For the installation, you need at least
- either `cp` and `chmod` or `install` - either `cp` and `chmod` or `install`
@ -115,15 +114,15 @@ Example:
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
This means to keep 28 daily backups, 12 monthly backups and 4 weekly. This means to keep 28 daily backups, 12 monthly backups and 4 weekly.
General pre- and post-execution General pre- and post-execution
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you add '$CCOLLECT_CONF/defaults/`pre_exec`' or If you add '$CCOLLECT_CONF/defaults/`pre_exec`' or
'/etc/ccollect/defaults/`pre_exec`' (same with `post_exec`), `ccollect` '/etc/ccollect/defaults/`pre_exec`' (same with `post_exec`), `ccollect`
will start `pre_exec` before the whole backup process and will start `pre_exec` before the whole backup process and
`post_exec` after backup of all sources is done. `post_exec` after backup of all sources is done.
The following Example describes how to report free disk space in The following example describes how to report free disk space in
human readable format before and after the whole backup process: human readable format before and after the whole backup process:
------------------------------------------------------------------------- -------------------------------------------------------------------------
[13:00] hydrogenium:~# mkdir -p /etc/ccollect/defaults/ [13:00] hydrogenium:~# mkdir -p /etc/ccollect/defaults/
@ -134,6 +133,7 @@ human readable format before and after the whole backup process:
[13:01] hydrogenium:~# ln -s /etc/ccollect/defaults/pre_exec /etc/ccollect/defaults/post_exec [13:01] hydrogenium:~# ln -s /etc/ccollect/defaults/pre_exec /etc/ccollect/defaults/post_exec
------------------------------------------------------------------------- -------------------------------------------------------------------------
Source configuration Source configuration
~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~
Each source configuration exists below '$CCOLLECT_CONF/sources/$name' or Each source configuration exists below '$CCOLLECT_CONF/sources/$name' or
@ -158,6 +158,7 @@ Additionally a source may have the following files:
- `pre_exec` program to execute before backuping this source - `pre_exec` program to execute before backuping this source
- `post_exec` program to execute after backuping this source - `post_exec` program to execute after backuping this source
Example: Example:
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
[10:47] zaphodbeeblebrox:ccollect-0.2% ls -l conf/sources/testsource2 [10:47] zaphodbeeblebrox:ccollect-0.2% ls -l conf/sources/testsource2
@ -179,6 +180,7 @@ Example:
/home/nico/vpn /home/nico/vpn
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
Detailled description of "source" Detailled description of "source"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
`source` describes a `rsync` compatible source (one line only). `source` describes a `rsync` compatible source (one line only).
@ -187,17 +189,20 @@ For instance 'backup_user@foreign_host:/home/server/video'.
To use the `rsync` protocol without the `ssh`-tunnel, use To use the `rsync` protocol without the `ssh`-tunnel, use
'rsync::USER@HOST/SRC'. For more information have a look at `rsync`(1). 'rsync::USER@HOST/SRC'. For more information have a look at `rsync`(1).
Detailled description of "verbose" Detailled description of "verbose"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
`verbose` tells `ccollect` that the log should contain verbose messages. `verbose` tells `ccollect` that the log should contain verbose messages.
If this file exists in the source specification *-v* will be passed to `rsync`. If this file exists in the source specification *-v* will be passed to `rsync`.
Example: Example:
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
[11:35] zaphodbeeblebrox:ccollect-0.2% touch conf/sources/testsource1/verbose [11:35] zaphodbeeblebrox:ccollect-0.2% touch conf/sources/testsource1/verbose
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
Detailled description of "very_verbose" Detailled description of "very_verbose"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
`very_verbose` tells `ccollect` that it should log very verbose. `very_verbose` tells `ccollect` that it should log very verbose.
@ -205,11 +210,13 @@ Detailled description of "very_verbose"
If this file exists in the source specification *-v* will be passed to If this file exists in the source specification *-v* will be passed to
`rsync`, `cp`, `rm` and `mkdir`. `rsync`, `cp`, `rm` and `mkdir`.
Example: Example:
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
[23:67] nohost:~% touch conf/sources/testsource1/very_verbose [23:67] nohost:~% touch conf/sources/testsource1/very_verbose
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
Detailled description of "summary" Detailled description of "summary"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -268,12 +275,17 @@ Detailled description of "destination"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
`destination` must be a link to the destination directory. `destination` must be a link to the destination directory.
Example: Example:
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
[11:36] zaphodbeeblebrox:ccollect-0.2% ls -l conf/sources/testsource2/destination [11:36] zaphodbeeblebrox:ccollect-0.2% ls -l conf/sources/testsource2/destination
lrwxrwxrwx 1 nico users 20 2005-11-17 16:44 conf/sources/testsource2/destination -> /home/nico/backupdir lrwxrwxrwx 1 nico users 20 2005-11-17 16:44 conf/sources/testsource2/destination -> /home/nico/backupdir
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
To speak truth, this is not fully correct. `ccollect` will also backup
your data, if `destination` is a directory. But do you really want to have
a backup below /etc?
Detailled description of "intervalls/" Detailled description of "intervalls/"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -281,6 +293,7 @@ When you create a subdirectory `intervalls/` within your source configuration
directory, you can specify individiual intervalls for this specific source. directory, you can specify individiual intervalls for this specific source.
Each file below this directory describes an intervall. Each file below this directory describes an intervall.
Example: Example:
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
[11:37] zaphodbeeblebrox:ccollect-0.2% ls -l conf/sources/testsource2/intervalls/ [11:37] zaphodbeeblebrox:ccollect-0.2% ls -l conf/sources/testsource2/intervalls/
@ -292,30 +305,32 @@ Example:
20 20
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
Detailled description of "rsync_options" Detailled description of "rsync_options"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When you create the file `rsync_options` below your source configuration, When you create the file `rsync_options` below your source configuration,
all the parameters found in this file will be passed to rsync. This all the parameters found in this file will be passed to rsync. This
way you can pass additional options to rsync. For instance you can tell rsync way you can pass additional options to rsync. For instance you can tell rsync
to show progress ("--progress") or which -password-file ("--password-file") to show progress ("--progress") or which -password-file ("--password-file")
to use for automatic backup over the rsync-protocol. to use for automatic backup over the rsync-protocol.
Example: Example:
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
[23:42] hydrogenium:ccollect-0.2% cat conf/sources/test_rsync/rsync_options [23:42] hydrogenium:ccollect-0.2% cat conf/sources/test_rsync/rsync_options
--password-file=/home/user/backup/protected_password_file --password-file=/home/user/backup/protected_password_file
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
Detailled description of "pre_exec" and "post_exec" Detailled description of "pre_exec" and "post_exec"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When you create `pre_exec` and / or `post_exec` below your source When you create `pre_exec` and / or `post_exec` below your source
configuration, `ccollect` will execute this command before, configuration, `ccollect` will execute this command before,
respective after doing the backup for *this specific* source. respective after doing the backup for *this specific* source.
If you want to have pre-/post-exec before and after *all* If you want to have pre-/post-exec before and after *all*
backups, see above for general configuration. backups, see above for general configuration.
Example: Example:
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
[13:09] hydrogenium:ccollect-0.3% cat conf/sources/with_exec/pre_exec [13:09] hydrogenium:ccollect-0.3% cat conf/sources/with_exec/pre_exec
@ -330,12 +345,12 @@ df -h
df -h df -h
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
Hints Hints
----- -----
Using rsync protocol without ssh Using rsync protocol without ssh
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When you have a computer with little computing power, it may be useful to use When you have a computer with little computing power, it may be useful to use
rsync without ssh, directly using the rsync protocol rsync without ssh, directly using the rsync protocol
(specify 'user@host::share' in `source`). You may wish to use (specify 'user@host::share' in `source`). You may wish to use
@ -358,7 +373,6 @@ This hint was reported by Daniel Aubry.
Not-excluding top-level directories Not-excluding top-level directories
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When you exclude "/proc" or "/mnt" from your backup, you may run into When you exclude "/proc" or "/mnt" from your backup, you may run into
trouble when you restore your backup. When you use "/proc/\*" or "/mnt/\*" trouble when you restore your backup. When you use "/proc/\*" or "/mnt/\*"
instead `ccollect` will backup empty directories. instead `ccollect` will backup empty directories.
@ -374,11 +388,11 @@ This hint was reported by Marcus Wagner.
Re-using already created rsync-backups Re-using already created rsync-backups
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you used `rsync` directly before you use `ccollect`, you can If you used `rsync` directly before you use `ccollect`, you can
use this old backup as initial backup for `ccollect`: You use this old backup as initial backup for `ccollect`: You
simply move it into a subdirectory named "'intervall'.0". simply move it into a subdirectory named "'intervall'.0".
Example: Example:
------------------------------------------------------------------------------- -------------------------------------------------------------------------------
backup:/home/backup/web1# ls backup:/home/backup/web1# ls
@ -424,7 +438,6 @@ F.A.Q.
What happens, if one backup is broken or empty? What happens, if one backup is broken or empty?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Let us assume, that one backup failed (connection broke or hard disk had Let us assume, that one backup failed (connection broke or hard disk had
some failures). So we've one backup in our history, which is incomplete. some failures). So we've one backup in our history, which is incomplete.
@ -433,7 +446,6 @@ The next time you use `ccollect`, it will transfer the missing files
When backing up from localhost the destination is also included. Is this a bug? When backing up from localhost the destination is also included. Is this a bug?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
No. `ccollect` passes your source definition directly to `rsync`. It No. `ccollect` passes your source definition directly to `rsync`. It
does not try to analyze it. So it actually does not know if a source does not try to analyze it. So it actually does not know if a source
comes from local harddisk or from a remote server. And it does not want comes from local harddisk or from a remote server. And it does not want
@ -479,6 +491,7 @@ srwali01:/etc/ccollect/sources/local-root# ccollect.sh taeglich local-root
|-> 0 backup(s) already exist, keeping 28 backup(s). |-> 0 backup(s) already exist, keeping 28 backup(s).
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
After that, I added some more sources: After that, I added some more sources:
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
srwali01:~# cd /etc/ccollect/sources srwali01:~# cd /etc/ccollect/sources
@ -508,6 +521,7 @@ srwali01:/etc/ccollect/sources/srwali03# ln -s /mnt/hdbackup/srwali03 destinatio
srwali01:/etc/ccollect/sources/srwali03# mkdir /mnt/hdbackup/srwali03 srwali01:/etc/ccollect/sources/srwali03# mkdir /mnt/hdbackup/srwali03
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
Using hard-links requires less disk space Using hard-links requires less disk space
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -569,5 +583,5 @@ du (GNU coreutils) 5.93
200M hydrogenium/durcheinander.2006-01-26-00:11.3332 200M hydrogenium/durcheinander.2006-01-26-00:11.3332
------------------------------------------------------------------------- -------------------------------------------------------------------------
In the second report the sizes include the space the inodes of In the second report (without -l) the sizes include the space the inodes of
the hardlinks allocate. the hardlinks allocate.