User may optionally create a file rsync_failure_codes with a
newline-separated list of rsync return codes that are to be
regarded as complete failure.
If rsync exits with such a code, then the backup will be marked
for deletion during the next ccollect run (if delete_incomplete).
I added documentation for this feature in doc/ccollect.text
In my experience (yours may differ), two rsync exit codes that
belong in this file are 12 and 255. I have seen 12 result from
ssh errors and 255 result from a kernel module conflict. In both
cases, the resulting backups were empty. Without the
rsync_failure_codes option, such errors cause good backups to be
deleted, as per c_interval, leaving behind the new empty backups.
In the long run, we may want a different and more comprehensive
method for analyzing rsync errors. In the short run, I find this
option necessary.
1). On systems I tried, delete_incomplete failed because the line:
< pcmd rm $VVERBOSE -rf "${ddir}/${realincomplete}" || \
should read:
> pcmd rm $VVERBOSE -rf "${realincomplete}" || \
(Is this true of all systems?)
2). The marker file was not deleted. Code was added to delete it.
3). The delete_incomplete code was simplified.
Previously, there was a $VERBOSE script variable that was set according to
the "-v" command line option and then reset to null within each source
sub-shell. With no loss of functionality, this patch removes all references
to that variable.
This makes the script 13 lines shorter.
According to the documentation, "if [the very_verbose] file exists in the
source specification -v will be passed to rsync, rm and mkdir." Previously,
the -v option was passed only to rsync. This patch passes it to rm and mkdir
as well.
Actually, as per the behavior of the previous version, it is verbose that
sends the -v option to rsync while very_verbose sends it the -vv option. I
left it this way because this behavior seems reasonable. Maybe the
documentation should be corrected on this point.
1). If an option doesn't exist in a source directory, check the defaults directory.
2). For every option, create a corresponding "no_" option so that a source directory
can override an option set in defaults.
(i.patch)
Based on patches by John Lawless <jll2_8854b@redwoodscientific.com>.
Skipped the sort changing part (from -tc to -t)
c.patch:
--- ccollect-0.7.1-b.sh 2009-05-24 21:32:00.000000000 -0700
+++ ccollect-0.7.1-c.sh 2009-05-24 21:39:43.000000000 -0700
@@ -40,10 +40,13 @@
VERSION=0.7.1
RELEASE="2009-02-02"
HALF_VERSION="ccollect ${VERSION}"
FULL_VERSION="ccollect ${VERSION} (${RELEASE})"
+#TSORT="tc" ; NEWER="cnewer"
+TSORT="t" ; NEWER="newer"
+
#
# CDATE: how we use it for naming of the archives
# DDATE: how the user should see it in our output (DISPLAY)
#
CDATE="date +%Y%m%d-%H%M"
@@ -513,14 +516,14 @@
#
# Check for backup directory to clone from: Always clone from the latest one!
#
- # Use ls -1c instead of -1t, because last modification maybe the same on all
- # and metadate update (-c) is updated by rsync locally.
- #
- last_dir="$(pcmd ls -tcp1 "${ddir}" | grep '/$' | head -n 1)" || \
+ # Depending on your file system, you may want to sort on:
+ # 1. mtime (modification time) with TSORT=t, or
+ # 2. ctime (last change time, usually) with TSORT=tc
+ last_dir="$(pcmd ls -${TSORT}p1 "${ddir}" | grep '/$' | head -n 1)" || \
_exit_err "Failed to list contents of ${ddir}."
#
# clone from old backup, if existing
#
d.patch:
--- ccollect-0.7.1-c.sh 2009-05-24 21:39:43.000000000 -0700
+++ ccollect-0.7.1-d.sh 2009-05-24 21:47:09.000000000 -0700
@@ -492,12 +492,12 @@
if [ "${count}" -ge "${c_interval}" ]; then
substract=$((${c_interval} - 1))
remove=$((${count} - ${substract}))
_techo "Removing ${remove} backup(s)..."
- pcmd ls -p1 "$ddir" | grep "^${INTERVAL}\..*/\$" | \
- sort -n | head -n "${remove}" > "${TMP}" || \
+ pcmd ls -${TSORT}p1r "$ddir" | grep "^${INTERVAL}\..*/\$" | \
+ head -n "${remove}" > "${TMP}" || \
_exit_err "Listing old backups failed"
i=0
while read to_remove; do
eval remove_$i=\"${to_remove}\"
Signed-off-by: Nico Schottelius <nico@ikn.schottelius.org>
> # Verify source is up and accepting connections before deleting any old backups
> rsync "$source" >/dev/null || _exit_err "Source ${source} is not readable. Skipping."
I think that this quick test is a much better than, say, pinging
the source in a pre-exec script: this tests not only that the
source is up and connected to the net, it also verifies (1) that
ssh is up and accepting our key (if we are using ssh), and (2) that
the source directory is mounted (if it needs to be mounted) and
readable.
Bug description:
From: Tiziano Müller <dev-zero@g.o>
Subject: Notiz: Probleme mit Ampersand in source
Date: Tue, 07 Oct 2008 16:11:10 +0000
Hoi Nico
Kleine Notiz wie im IRC besprochen:
- Hat es in /etc/ccollect/sources/$BACKUPNAME/source ein Ampersand muss
dieses Escaped werden, da sonst folgender Fehler auftritt:
*snip*
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03: Beginning to backup,
this may take some time...
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03:
Creating /home/backup/volumes/Forschung&Entwicklung/daily.20081005-0310.15173 ...
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03: Transferring
files...
[volumes_ForschungEntwicklung] bash: Entwicklung/*: No such file or
directory
[volumes_ForschungEntwicklung] rsync: connection unexpectedly closed (4
bytes received so far) [receiver]
[volumes_ForschungEntwicklung] rsync error: error in rsync protocol data
stream (code 12) at io.c(635) [receiver=3.0.2]
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03: Finished backup
(rsync return code: 12).
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03: Warning: rsync
exited non-zero, the backup may be broken (see rsync errors).
*snip*
- ein "&" im Namen fürs Backup,
also: /etc/ccollect/sources/volumes_Forschung&Entwicklung/ wird beim
Anzeigen (wie oben zu sehen) verschluckt.
Cheers,
Tiziano
Signed-off-by: Nico Schottelius <nico@ikn.schottelius.org>
1. I got a syntax error in ccollect.sh once in a while, i think at
rollover. I changed the line
j=$((j+1))
to
j=$(($j+1))
(twice) and the error went away.
