[SOLVED] crontab rsync not working

If none of the more specific forums is the right place to ask

[SOLVED] crontab rsync not working

Postby MeleeIkon » 2017-12-29 19:43

I have a debian 9 vm, quad core, 8GB Ram. I run the Unifi Video software on it. I have 60GB boot & 1TB Data VHD's and then I have a 2TB NFS drive that runs over a VPN tunnel (offsite). Now I have a simple rsync script that I call from crontab through flock.
Code: Select all
*\5 * * * *     /usr/bin/flock -n /var/lock/cameracopy.lck -c "/usr/local/bin/cameracopy.sh"


Yes, I know flock doesn't work well with NFS but it doesn't know it is NFS, I'm not calling an NFS store through flock, The system is handling that so it shouldn't matter. to flock, its just another directory.

The script is:
#!/bin/bash
echo "Time: $(date -Iseconds). RSync Starting."$'\r' >> /mnt/backup/cronjob.log
rsync -rltzhu --log-file=/mnt/backup/cambackup.log /cameras/videos /mnt/backup
echo "Time: $(date -Iseconds). RSync Finished."$'\r' >> /mnt/backup/cronjob.log
echo $'\r' >> /mnt/backup/cronjob.log


Well the lock file is created, but never gets deleted and I keep getting multiple RSync Starting in the log.
Code: Select all
Time: 2017-12-29T15:20:01-05:00. RSync Starting.
Time: 2017-12-29T15:21:01-05:00. RSync Starting.
Time: 2017-12-29T15:22:01-05:00. RSync Starting.
Time: 2017-12-29T15:23:01-05:00. RSync Starting.
Time: 2017-12-29T15:24:01-05:00. RSync Starting.
Time: 2017-12-29T15:25:01-05:00. RSync Starting.
Time: 2017-12-29T15:26:01-05:00. RSync Starting.
Time: 2017-12-29T15:27:01-05:00. RSync Starting.
Time: 2017-12-29T15:28:01-05:00. RSync Starting.
Time: 2017-12-29T15:29:01-05:00. RSync Starting.


However the cambackup.log shows no changes since 7am this morning when I ran it manually (the script works, but never writes RSync Finished, ever- even manually). I've even gone in and deleted the lock file and it just gets recreated.
Last edited by MeleeIkon on 2017-12-30 22:04, edited 1 time in total.
MeleeIkon
 
Posts: 5
Joined: 2017-12-29 19:26

Re: crontab rsync not working

Postby dilberts_left_nut » 2017-12-29 19:59

Full paths are your friends.
AdrianTM wrote:There's no hacker in my grandma...
User avatar
dilberts_left_nut
 
Posts: 4884
Joined: 2009-10-05 07:54
Location: enzed

Re: crontab rsync not working

Postby MeleeIkon » 2017-12-29 20:25

I have full paths, everywhere... What am I missing or not seeing?
MeleeIkon
 
Posts: 5
Joined: 2017-12-29 19:26

Re: crontab rsync not working

Postby dilberts_left_nut » 2017-12-29 20:35

Not in your script.

Your cron line is also messed up, so is running every minute. See https://linux.die.net/man/5/crontab

No clue about flock - seems overly complicated for your needs - why not just test for existence of an already running script instance, with pgrep or such?

Does it work if you just run the script?
AdrianTM wrote:There's no hacker in my grandma...
User avatar
dilberts_left_nut
 
Posts: 4884
Joined: 2009-10-05 07:54
Location: enzed

Re: crontab rsync not working

Postby MeleeIkon » 2017-12-29 21:28

D'oh! / vs \

*/5 * * * *

Overly complicated, how so? one line is better than the if statement and if I get my whacks wrong, just imagine how I could mess that up! What was the saying? With C, you can shoot yourself in the foot, but in C++ you blow off your whole darn leg.

And with pgrep, I would still need wc -l, so isn't that just a fancy ps aux | grep foo | grep -v grep | wc -l.

Seems just another way to skin a cat to me.

oh yes, there is dirt older than me.
MeleeIkon
 
Posts: 5
Joined: 2017-12-29 19:26

Re: crontab rsync not working

Postby dilberts_left_nut » 2017-12-29 22:01

I'd just use something like (not tested)
Code: Select all
*/5 * * * *     /usr/bin/pgrep cameracopy.sh || /usr/local/bin/cameracopy.sh

It will only execute the script if pgrep doesn't find it running already.
AdrianTM wrote:There's no hacker in my grandma...
User avatar
dilberts_left_nut
 
Posts: 4884
Joined: 2009-10-05 07:54
Location: enzed

Re: crontab rsync not working

Postby MeleeIkon » 2017-12-30 19:17

Well, the best thing to do is test it. I'm spooling up another VM on my home lab and I have a Synology which can do NFS. Will check back later.
MeleeIkon
 
Posts: 5
Joined: 2017-12-29 19:26

Re: crontab rsync not working

Postby MeleeIkon » 2017-12-30 21:45

MeleeIkon wrote:Well, the best thing to do is test it. I'm spooling up another VM on my home lab and I have a Synology which can do NFS. Will check back later.


OK. I have my test set up. I have my script, crontest.sh
Code: Select all
#!/bin/bash
echo "Time: $(date -Iseconds). RSync Starting."$'\r' >> /mnt/nas/cronjob.log
/usr/bin/rsync -rltzhu --log-file=/mnt/nas/cambackup.log /mnt/test /mnt/nas
echo "Time: $(date -Iseconds). RSync Finished."$'\r' >> /mnt/nas/cronjob.log
echo $'\r' >> /mnt/nas/cronjob.log


My nas:
10.25.10.16:/volume1/crontest 4.0T 391M 4.0T 1% /mnt/nas


And Crontab:
Code: Select all
*/5 * * * *     /usr/bin/pgrep crontest.sh || /usr/local/bin/crontest.sh


It starts... but...
Time: 2017-12-30T17:25:38-05:00. RSync Starting.
Time: 2017-12-30T17:25:48-05:00. RSync Finished.

Time: 2017-12-30T17:30:00-05:00. RSync Starting.
Time: 2017-12-30T17:30:59-05:00. RSync Finished.


It seems as if the contents of my downloads folder is not enough to make it take longer than 5 minutes. I'm dumping some more stuff into it now.

EDIT: I think it works...
Time: 2017-12-30T17:40:01-05:00. RSync Starting.
Time: 2017-12-30T17:47:23-05:00. RSync Finished.

Time: 2017-12-30T17:50:02-05:00. RSync Starting.


It exceeded 5 minutes after I dumped 18GB into it.
MeleeIkon
 
Posts: 5
Joined: 2017-12-29 19:26


Return to General Questions

Who is online

Users browsing this forum: No registered users and 5 guests

fashionable