Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

[SOLVED] crontab rsync not working

If none of the specific sub-forums seem right for your thread, ask here.
Post Reply
Message
Author
MeleeIkon
Posts: 5
Joined: 2017-12-29 19:26

[SOLVED] crontab rsync not working

#1 Post by MeleeIkon »

I have a debian 9 vm, quad core, 8GB Ram. I run the Unifi Video software on it. I have 60GB boot & 1TB Data VHD's and then I have a 2TB NFS drive that runs over a VPN tunnel (offsite). Now I have a simple rsync script that I call from crontab through flock.

Code: Select all

*\5 * * * *     /usr/bin/flock -n /var/lock/cameracopy.lck -c "/usr/local/bin/cameracopy.sh"
Yes, I know flock doesn't work well with NFS but it doesn't know it is NFS, I'm not calling an NFS store through flock, The system is handling that so it shouldn't matter. to flock, its just another directory.

The script is:
#!/bin/bash
echo "Time: $(date -Iseconds). RSync Starting."$'\r' >> /mnt/backup/cronjob.log
rsync -rltzhu --log-file=/mnt/backup/cambackup.log /cameras/videos /mnt/backup
echo "Time: $(date -Iseconds). RSync Finished."$'\r' >> /mnt/backup/cronjob.log
echo $'\r' >> /mnt/backup/cronjob.log
Well the lock file is created, but never gets deleted and I keep getting multiple RSync Starting in the log.

Code: Select all

Time: 2017-12-29T15:20:01-05:00. RSync Starting.
Time: 2017-12-29T15:21:01-05:00. RSync Starting.
Time: 2017-12-29T15:22:01-05:00. RSync Starting.
Time: 2017-12-29T15:23:01-05:00. RSync Starting.
Time: 2017-12-29T15:24:01-05:00. RSync Starting.
Time: 2017-12-29T15:25:01-05:00. RSync Starting.
Time: 2017-12-29T15:26:01-05:00. RSync Starting.
Time: 2017-12-29T15:27:01-05:00. RSync Starting.
Time: 2017-12-29T15:28:01-05:00. RSync Starting.
Time: 2017-12-29T15:29:01-05:00. RSync Starting.
However the cambackup.log shows no changes since 7am this morning when I ran it manually (the script works, but never writes RSync Finished, ever- even manually). I've even gone in and deleted the lock file and it just gets recreated.
Last edited by MeleeIkon on 2017-12-30 22:04, edited 1 time in total.

User avatar
dilberts_left_nut
Administrator
Administrator
Posts: 5346
Joined: 2009-10-05 07:54
Location: enzed
Has thanked: 12 times
Been thanked: 66 times

Re: crontab rsync not working

#2 Post by dilberts_left_nut »

Full paths are your friends.
AdrianTM wrote:There's no hacker in my grandma...

MeleeIkon
Posts: 5
Joined: 2017-12-29 19:26

Re: crontab rsync not working

#3 Post by MeleeIkon »

I have full paths, everywhere... What am I missing or not seeing?

User avatar
dilberts_left_nut
Administrator
Administrator
Posts: 5346
Joined: 2009-10-05 07:54
Location: enzed
Has thanked: 12 times
Been thanked: 66 times

Re: crontab rsync not working

#4 Post by dilberts_left_nut »

Not in your script.

Your cron line is also messed up, so is running every minute. See https://linux.die.net/man/5/crontab

No clue about flock - seems overly complicated for your needs - why not just test for existence of an already running script instance, with pgrep or such?

Does it work if you just run the script?
AdrianTM wrote:There's no hacker in my grandma...

MeleeIkon
Posts: 5
Joined: 2017-12-29 19:26

Re: crontab rsync not working

#5 Post by MeleeIkon »

D'oh! / vs \

*/5 * * * *

Overly complicated, how so? one line is better than the if statement and if I get my whacks wrong, just imagine how I could mess that up! What was the saying? With C, you can shoot yourself in the foot, but in C++ you blow off your whole darn leg.

And with pgrep, I would still need wc -l, so isn't that just a fancy ps aux | grep foo | grep -v grep | wc -l.

Seems just another way to skin a cat to me.

oh yes, there is dirt older than me.

User avatar
dilberts_left_nut
Administrator
Administrator
Posts: 5346
Joined: 2009-10-05 07:54
Location: enzed
Has thanked: 12 times
Been thanked: 66 times

Re: crontab rsync not working

#6 Post by dilberts_left_nut »

I'd just use something like (not tested)

Code: Select all

*/5 * * * *     /usr/bin/pgrep cameracopy.sh || /usr/local/bin/cameracopy.sh
It will only execute the script if pgrep doesn't find it running already.
AdrianTM wrote:There's no hacker in my grandma...

MeleeIkon
Posts: 5
Joined: 2017-12-29 19:26

Re: crontab rsync not working

#7 Post by MeleeIkon »

Well, the best thing to do is test it. I'm spooling up another VM on my home lab and I have a Synology which can do NFS. Will check back later.

MeleeIkon
Posts: 5
Joined: 2017-12-29 19:26

Re: crontab rsync not working

#8 Post by MeleeIkon »

MeleeIkon wrote:Well, the best thing to do is test it. I'm spooling up another VM on my home lab and I have a Synology which can do NFS. Will check back later.
OK. I have my test set up. I have my script, crontest.sh

Code: Select all

#!/bin/bash
echo "Time: $(date -Iseconds). RSync Starting."$'\r' >> /mnt/nas/cronjob.log
/usr/bin/rsync -rltzhu --log-file=/mnt/nas/cambackup.log /mnt/test /mnt/nas
echo "Time: $(date -Iseconds). RSync Finished."$'\r' >> /mnt/nas/cronjob.log
echo $'\r' >> /mnt/nas/cronjob.log
My nas:
10.25.10.16:/volume1/crontest 4.0T 391M 4.0T 1% /mnt/nas
And Crontab:

Code: Select all

*/5 * * * *     /usr/bin/pgrep crontest.sh || /usr/local/bin/crontest.sh
It starts... but...
Time: 2017-12-30T17:25:38-05:00. RSync Starting.
Time: 2017-12-30T17:25:48-05:00. RSync Finished.

Time: 2017-12-30T17:30:00-05:00. RSync Starting.
Time: 2017-12-30T17:30:59-05:00. RSync Finished.
It seems as if the contents of my downloads folder is not enough to make it take longer than 5 minutes. I'm dumping some more stuff into it now.

EDIT: I think it works...
Time: 2017-12-30T17:40:01-05:00. RSync Starting.
Time: 2017-12-30T17:47:23-05:00. RSync Finished.

Time: 2017-12-30T17:50:02-05:00. RSync Starting.
It exceeded 5 minutes after I dumped 18GB into it.

Post Reply