1). On systems I tried, delete_incomplete failed because the line:
< pcmd rm $VVERBOSE -rf "${ddir}/${realincomplete}" || \
should read:
> pcmd rm $VVERBOSE -rf "${realincomplete}" || \
(Is this true of all systems?)
2). The marker file was not deleted. Code was added to delete it.
3). The delete_incomplete code was simplified.
Previously, there was a $VERBOSE script variable that was set according to
the "-v" command line option and then reset to null within each source
sub-shell. With no loss of functionality, this patch removes all references
to that variable.
This makes the script 13 lines shorter.
According to the documentation, "if [the very_verbose] file exists in the
source specification -v will be passed to rsync, rm and mkdir." Previously,
the -v option was passed only to rsync. This patch passes it to rm and mkdir
as well.
Actually, as per the behavior of the previous version, it is verbose that
sends the -v option to rsync while very_verbose sends it the -vv option. I
left it this way because this behavior seems reasonable. Maybe the
documentation should be corrected on this point.
1). If an option doesn't exist in a source directory, check the defaults directory.
2). For every option, create a corresponding "no_" option so that a source directory
can override an option set in defaults.
(i.patch)
I added some patches from John with an E-Mail address he
does not to be public on the internet.
I did not ask him before whether this is fine, so I screwed
up (similar to the description in git-tag(1)).
Thus I replaced his e-mail and would like you to accept
this forced push and remove old trees on the net.
Thanks,
Nico
Signed-off-by: Nico Schottelius <nico@ikn.schottelius.org>
Based on patches by John Lawless <jlawless@redwoodscientific.com>.
Skipped the sort changing part (from -tc to -t)
c.patch:
--- ccollect-0.7.1-b.sh 2009-05-24 21:32:00.000000000 -0700
+++ ccollect-0.7.1-c.sh 2009-05-24 21:39:43.000000000 -0700
@@ -40,10 +40,13 @@
VERSION=0.7.1
RELEASE="2009-02-02"
HALF_VERSION="ccollect ${VERSION}"
FULL_VERSION="ccollect ${VERSION} (${RELEASE})"
+#TSORT="tc" ; NEWER="cnewer"
+TSORT="t" ; NEWER="newer"
+
#
# CDATE: how we use it for naming of the archives
# DDATE: how the user should see it in our output (DISPLAY)
#
CDATE="date +%Y%m%d-%H%M"
@@ -513,14 +516,14 @@
#
# Check for backup directory to clone from: Always clone from the latest one!
#
- # Use ls -1c instead of -1t, because last modification maybe the same on all
- # and metadate update (-c) is updated by rsync locally.
- #
- last_dir="$(pcmd ls -tcp1 "${ddir}" | grep '/$' | head -n 1)" || \
+ # Depending on your file system, you may want to sort on:
+ # 1. mtime (modification time) with TSORT=t, or
+ # 2. ctime (last change time, usually) with TSORT=tc
+ last_dir="$(pcmd ls -${TSORT}p1 "${ddir}" | grep '/$' | head -n 1)" || \
_exit_err "Failed to list contents of ${ddir}."
#
# clone from old backup, if existing
#
d.patch:
--- ccollect-0.7.1-c.sh 2009-05-24 21:39:43.000000000 -0700
+++ ccollect-0.7.1-d.sh 2009-05-24 21:47:09.000000000 -0700
@@ -492,12 +492,12 @@
if [ "${count}" -ge "${c_interval}" ]; then
substract=$((${c_interval} - 1))
remove=$((${count} - ${substract}))
_techo "Removing ${remove} backup(s)..."
- pcmd ls -p1 "$ddir" | grep "^${INTERVAL}\..*/\$" | \
- sort -n | head -n "${remove}" > "${TMP}" || \
+ pcmd ls -${TSORT}p1r "$ddir" | grep "^${INTERVAL}\..*/\$" | \
+ head -n "${remove}" > "${TMP}" || \
_exit_err "Listing old backups failed"
i=0
while read to_remove; do
eval remove_$i=\"${to_remove}\"
Signed-off-by: Nico Schottelius <nico@ikn.schottelius.org>
Based on patches by John Lawless <jll2_8854b@redwoodscientific.com>.
Skipped the sort changing part (from -tc to -t)
c.patch:
--- ccollect-0.7.1-b.sh 2009-05-24 21:32:00.000000000 -0700
+++ ccollect-0.7.1-c.sh 2009-05-24 21:39:43.000000000 -0700
@@ -40,10 +40,13 @@
VERSION=0.7.1
RELEASE="2009-02-02"
HALF_VERSION="ccollect ${VERSION}"
FULL_VERSION="ccollect ${VERSION} (${RELEASE})"
+#TSORT="tc" ; NEWER="cnewer"
+TSORT="t" ; NEWER="newer"
+
#
# CDATE: how we use it for naming of the archives
# DDATE: how the user should see it in our output (DISPLAY)
#
CDATE="date +%Y%m%d-%H%M"
@@ -513,14 +516,14 @@
#
# Check for backup directory to clone from: Always clone from the latest one!
#
- # Use ls -1c instead of -1t, because last modification maybe the same on all
- # and metadate update (-c) is updated by rsync locally.
- #
- last_dir="$(pcmd ls -tcp1 "${ddir}" | grep '/$' | head -n 1)" || \
+ # Depending on your file system, you may want to sort on:
+ # 1. mtime (modification time) with TSORT=t, or
+ # 2. ctime (last change time, usually) with TSORT=tc
+ last_dir="$(pcmd ls -${TSORT}p1 "${ddir}" | grep '/$' | head -n 1)" || \
_exit_err "Failed to list contents of ${ddir}."
#
# clone from old backup, if existing
#
d.patch:
--- ccollect-0.7.1-c.sh 2009-05-24 21:39:43.000000000 -0700
+++ ccollect-0.7.1-d.sh 2009-05-24 21:47:09.000000000 -0700
@@ -492,12 +492,12 @@
if [ "${count}" -ge "${c_interval}" ]; then
substract=$((${c_interval} - 1))
remove=$((${count} - ${substract}))
_techo "Removing ${remove} backup(s)..."
- pcmd ls -p1 "$ddir" | grep "^${INTERVAL}\..*/\$" | \
- sort -n | head -n "${remove}" > "${TMP}" || \
+ pcmd ls -${TSORT}p1r "$ddir" | grep "^${INTERVAL}\..*/\$" | \
+ head -n "${remove}" > "${TMP}" || \
_exit_err "Listing old backups failed"
i=0
while read to_remove; do
eval remove_$i=\"${to_remove}\"
Signed-off-by: Nico Schottelius <nico@ikn.schottelius.org>
> # Verify source is up and accepting connections before deleting any old backups
> rsync "$source" >/dev/null || _exit_err "Source ${source} is not readable. Skipping."
I think that this quick test is a much better than, say, pinging
the source in a pre-exec script: this tests not only that the
source is up and connected to the net, it also verifies (1) that
ssh is up and accepting our key (if we are using ssh), and (2) that
the source directory is mounted (if it needs to be mounted) and
readable.
> # Verify source is up and accepting connections before deleting any old backups
> rsync "$source" >/dev/null || _exit_err "Source ${source} is not readable. Skipping."
I think that this quick test is a much better than, say, pinging
the source in a pre-exec script: this tests not only that the
source is up and connected to the net, it also verifies (1) that
ssh is up and accepting our key (if we are using ssh), and (2) that
the source directory is mounted (if it needs to be mounted) and
readable.
Bug description:
From: Tiziano Müller <dev-zero@g.o>
Subject: Notiz: Probleme mit Ampersand in source
Date: Tue, 07 Oct 2008 16:11:10 +0000
Hoi Nico
Kleine Notiz wie im IRC besprochen:
- Hat es in /etc/ccollect/sources/$BACKUPNAME/source ein Ampersand muss
dieses Escaped werden, da sonst folgender Fehler auftritt:
*snip*
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03: Beginning to backup,
this may take some time...
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03:
Creating /home/backup/volumes/Forschung&Entwicklung/daily.20081005-0310.15173 ...
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03: Transferring
files...
[volumes_ForschungEntwicklung] bash: Entwicklung/*: No such file or
directory
[volumes_ForschungEntwicklung] rsync: connection unexpectedly closed (4
bytes received so far) [receiver]
[volumes_ForschungEntwicklung] rsync error: error in rsync protocol data
stream (code 12) at io.c(635) [receiver=3.0.2]
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03: Finished backup
(rsync return code: 12).
[volumes_ForschungEntwicklung] 2008-10-05-03:10:03: Warning: rsync
exited non-zero, the backup may be broken (see rsync errors).
*snip*
- ein "&" im Namen fürs Backup,
also: /etc/ccollect/sources/volumes_Forschung&Entwicklung/ wird beim
Anzeigen (wie oben zu sehen) verschluckt.
Cheers,
Tiziano
Signed-off-by: Nico Schottelius <nico@ikn.schottelius.org>
1. I got a syntax error in ccollect.sh once in a while, i think at
rollover. I changed the line
j=$((j+1))
to
j=$(($j+1))
(twice) and the error went away.
