Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Scripts to automate jigdo download

Share your HowTo, Documentation, Tips and Tricks. Not for support questions!.
Post Reply
Message
Author
timbgo
Posts: 265
Joined: 2013-04-14 12:17

Scripts to automate jigdo download

#1 Post by timbgo »

Scripts to automate jigdo download
============================
First of all:
These scripts work for me, and allow me to, say, go to sleep while they take care to replace old iso with the new, and unmount the old iso and free space, because I, as right now, am running jigdo download with all the iso jigdo and template files for my amd64 machine, and the sources...
...and, again, it works for me.
Sure if you're not amd64, modify the scripts to meet your own arch.
But above all, DON'T USE this if you will blame me for if anything goes wrong!
Responsability will only be your own!

Prerequisite: Old Jigdo downloaded ISO files from:

http://cdimage.debian.org/cdimage/weekl ... jigdo-dvd/

Last week's or last month's.
Or this week's if you will download those this week for the first time, but, sure, it will only make sense for you then to use this script next week.
No point using this script without old ISO files.

Create a directory Old in this directory where you will run these my scripts from.
$ mkdir Old
and move or copy from where you keep them, all the old ISO files into that directory.

Now here the scripts.

Put these in a separate user (yout) writeable directory. You will be running them from there.
They are:
jigdo-dl_01-jigdo-templ-t-s.sh
jigdo-dl_02-mnt-t-s.sh
jigdo-dl_03-dLo-SUMS.sh
jigdo-dl_04-dLo-t-s.sh


jigdo-dl_01-jigdo-templ-t-s.sh

Code: Select all

wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-1.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-1.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-2.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-2.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-3.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-3.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-4.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-4.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-5.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-5.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-6.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-6.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-7.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-7.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-8.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-8.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-9.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-9.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-10.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-10.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-11.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-11.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-12.jigdo http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/debian-testing-amd64-DVD-12.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-1.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-1.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-2.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-1.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-2.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-2.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-3.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-3.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-4.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-4.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-5.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-5.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-6.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-6.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-7.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-7.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-8.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-8.template
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-9.jigdo http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/debian-testing-source-DVD-9.template
----------------------------------------------------------------------------

jigdo-dl_02-mnt-t-s.sh

Code: Select all

mkdir -p d-1 ; mount Old/debian-testing-amd64-DVD-1.iso d-1
mkdir -p d-2 ; mount Old/debian-testing-amd64-DVD-2.iso d-2
mkdir -p d-3 ; mount Old/debian-testing-amd64-DVD-3.iso d-3
mkdir -p d-4 ; mount Old/debian-testing-amd64-DVD-4.iso d-4
mkdir -p d-5 ; mount Old/debian-testing-amd64-DVD-5.iso d-5
mkdir -p d-6 ; mount Old/debian-testing-amd64-DVD-6.iso d-6
mkdir -p d-7 ; mount Old/debian-testing-amd64-DVD-7.iso d-7
mkdir -p d-8 ; mount Old/debian-testing-amd64-DVD-8.iso d-8
mkdir -p d-9 ; mount Old/debian-testing-amd64-DVD-9.iso d-9
mkdir -p d-10 ; mount Old/debian-testing-amd64-DVD-10.iso d-10
mkdir -p d-11 ; mount Old/debian-testing-amd64-DVD-11.iso d-11
mkdir -p d-12 ; mount Old/debian-testing-amd64-DVD-12.iso d-12
mkdir -p d-13 ; mount Old/debian-testing-source-DVD-1.iso d-13
mkdir -p d-14 ; mount Old/debian-testing-source-DVD-2.iso d-14
mkdir -p d-15 ; mount Old/debian-testing-source-DVD-3.iso d-15
mkdir -p d-16 ; mount Old/debian-testing-source-DVD-4.iso d-16
mkdir -p d-17 ; mount Old/debian-testing-source-DVD-5.iso d-17
mkdir -p d-18 ; mount Old/debian-testing-source-DVD-6.iso d-18
mkdir -p d-19 ; mount Old/debian-testing-source-DVD-7.iso d-19
mkdir -p d-20 ; mount Old/debian-testing-source-DVD-8.iso d-20
mkdir -p d-21 ; mount Old/debian-testing-source-DVD-9.iso d-21
jigdo-dl_03-dLo-SUMS.sh

Code: Select all

wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/MD5SUMS
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/SHA1SUMS
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/SHA256SUMS
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/amd64/jigdo-dvd/SHA512SUMS
mv -iv MD5SUMS    MD5SUMS-t
mv -iv SHA1SUMS   SHA1SUMS-t
mv -iv SHA256SUMS SHA256SUMS-t
mv -iv SHA512SUMS SHA512SUMS-t
  
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/MD5SUMS
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/SHA1SUMS
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/SHA256SUMS
wget -nc http://cdimage.debian.org/cdimage/weekly-builds/source/jigdo-dvd/SHA512SUMS
mv -iv MD5SUMS    MD5SUMS-s
mv -iv SHA1SUMS   SHA1SUMS-s
mv -iv SHA256SUMS SHA256SUMS-s
mv -iv SHA512SUMS SHA512SUMS-s
jigdo-dl_04-dLo-t-s.sh
jigdo-lite --scan d-1/ debian-testing-amd64-DVD-1.jigdo ; \
jigdo-lite --noask --scan d-2/ debian-testing-amd64-DVD-2.jigdo ; \
jigdo-lite --noask --scan d-3/ debian-testing-amd64-DVD-3.jigdo ; \
jigdo-lite --noask --scan d-4/ debian-testing-amd64-DVD-4.jigdo ; \
jigdo-lite --noask --scan d-5/ debian-testing-amd64-DVD-5.jigdo ; \
jigdo-lite --noask --scan d-6/ debian-testing-amd64-DVD-6.jigdo ; \
jigdo-lite --noask --scan d-7/ debian-testing-amd64-DVD-7.jigdo ; \
jigdo-lite --noask --scan d-8/ debian-testing-amd64-DVD-8.jigdo ; \
jigdo-lite --noask --scan d-9/ debian-testing-amd64-DVD-9.jigdo ; \
jigdo-lite --noask --scan d-10/ debian-testing-amd64-DVD-10.jigdo ; \
jigdo-lite --noask --scan d-11/ debian-testing-amd64-DVD-11.jigdo ; \
jigdo-lite --noask --scan d-12/ debian-testing-amd64-DVD-12.jigdo
jigdo-lite --noask --scan d-13/ debian-testing-source-DVD-1.jigdo ; \
jigdo-lite --noask --scan d-14/ debian-testing-source-DVD-2.jigdo ; \
jigdo-lite --noask --scan d-15/ debian-testing-source-DVD-3.jigdo ; \
jigdo-lite --noask --scan d-16/ debian-testing-source-DVD-4.jigdo ; \
jigdo-lite --noask --scan d-17/ debian-testing-source-DVD-5.jigdo ; \
jigdo-lite --noask --scan d-18/ debian-testing-source-DVD-6.jigdo ; \
jigdo-lite --noask --scan d-19/ debian-testing-source-DVD-7.jigdo ; \
jigdo-lite --noask --scan d-20/ debian-testing-source-DVD-8.jigdo ; \
jigdo-lite --noask --scan d-21/ debian-testing-source-DVD-9.jigdo ; \
dl in the names of the script is for d(own)l(oad), mnt for mount, t for testing s for source. Not important, but helps understand.

Now I have a partition

Code: Select all

$ df -h
/dev/my_device   123G  111G  6.0G  95% /Cmn
$
of which in my Old directory I got:

Code: Select all

mr@naibd9:/Cmn/Debian$ ls -lh Old
total 86G
-rw-r--r-- 1 mr mr 4.4G Jan  6 15:31 debian-testing-amd64-DVD-10.iso
-rw-r--r-- 1 mr mr 4.4G Jan  2 21:33 debian-testing-amd64-DVD-11.iso
-rw-r--r-- 1 mr mr 446M Jan  2 21:43 debian-testing-amd64-DVD-12.iso
-rw-r--r-- 1 mr mr 3.8G Jan  6 11:37 debian-testing-amd64-DVD-1.iso
-rw-r--r-- 1 mr mr 4.3G Jan  6 11:52 debian-testing-amd64-DVD-2.iso
-rw-r--r-- 1 mr mr 4.4G Jan  6 12:10 debian-testing-amd64-DVD-3.iso
-rw-r--r-- 1 mr mr 4.4G Jan  6 12:29 debian-testing-amd64-DVD-4.iso
-rw-r--r-- 1 mr mr 4.4G Jan  6 12:42 debian-testing-amd64-DVD-5.iso
-rw-r--r-- 1 mr mr 4.4G Jan  6 12:57 debian-testing-amd64-DVD-6.iso
-rw-r--r-- 1 mr mr 4.4G Jan  6 13:20 debian-testing-amd64-DVD-7.iso
-rw-r--r-- 1 mr mr 4.4G Jan  6 14:06 debian-testing-amd64-DVD-8.iso
-rw-r--r-- 1 mr mr 4.4G Jan  6 14:42 debian-testing-amd64-DVD-9.iso
-rw-r--r-- 1 mr mr 4.4G Jan  2 22:02 debian-testing-source-DVD-1.iso
-rw-r--r-- 1 mr mr 4.4G Jan  2 22:24 debian-testing-source-DVD-2.iso
-rw-r--r-- 1 mr mr 4.4G Jan  2 22:47 debian-testing-source-DVD-3.iso
-rw-r--r-- 1 mr mr 4.4G Jan  2 23:14 debian-testing-source-DVD-4.iso
-rw-r--r-- 1 mr mr 4.4G Jan  2 23:53 debian-testing-source-DVD-5.iso
-rw-r--r-- 1 mr mr 4.4G Jan  3 00:37 debian-testing-source-DVD-6.iso
-rw-r--r-- 1 mr mr 4.4G Jan  3 01:18 debian-testing-source-DVD-7.iso
-rw-r--r-- 1 mr mr 4.4G Jan  3 02:32 debian-testing-source-DVD-8.iso
-rw-r--r-- 1 mr mr 3.1G Jan  3 02:55 debian-testing-source-DVD-9.iso
mr@naibd9:/Cmn/Debian$
and just imagine how I had to wake over it and delete and move and unmount etc. and how meny times, to replace all the old with the new.
Not anymore.
But I haven't yet given you the script that does the moving and unmounting and deleting at the right time, while I sleep!
Wait, it's two more scripts Here they come.

iso_check-del_00.sh

Code: Select all

#!/bin/bash
arrayMD5=(`cat MD5SUMS-bare`)
arrayDebNames=(`cat debian-DVD-list`)
for ((i=0; i <= "20" ; i++ )) 
do
 echo "$i" 
echo ${arrayMD5[i]} ;
 iR=${arrayMD5[i]}
echo "\$iR" ;
echo "$iR" ;
 j="${arrayDebNames[i]}"  ;
echo "\$j" ;
echo "$j" ;
 jR=`md5sum $j|cut -d ' ' -f 1` ;
echo "\$jR" ;
echo "$jR" ;
# read FAKE ;
if [[ $iR == $jR ]] ;
then 
k=`echo $i+1|bc`
echo "\$k" ;
echo "$k" ;
 echo ;
 echo ;
 echo ;
 echo ;
 echo ;
 echo ;
 mv -v ${arrayDebNames[i]} Old/ ; umount -v d-$k ;
 rmdir -v d-$k ;
 df -h . ;
 echo "Sleeping 2 minutes, usually even more to next mv..."
 echo ;
 echo ;
 echo ;
 echo ;
 echo ;
 sleep 120 ; 
fi
df -h . ;
# read FAKE ;
done
echo "============================================================"
That one is still in the same directory which you created as explained, with all the other scripts so far.

Now the final script.

Place this one in /usr/local/bin/
It could be done otherwise, but it's ok this way.

Oh, if I forget to write this where appropriate, let me tell you here, all the scripts certainly must be

Code: Select all

$ chmod 755 the_script
But the following one must be so as root, and created in, or anyway placed in /usr/locl/bin/ by root.

EDIT Fri Mar 14 09:56:16 GMT 2014 START
This below is not the run_CMD_00.sh! And then the whole idea will not work.
I have noticed this this very moment. Correction follows in today's post further down.
EDIT END
run_CMD_00.sh

Code: Select all

#!/bin/bash
arrayMD5=(`cat MD5SUMS-bare`)
arrayDebNames=(`cat debian-DVD-list`)
for ((i=0; i <= "20" ; i++ )) 
do
 echo "$i" 
echo ${arrayMD5[i]} ;
 iR=${arrayMD5[i]}
echo "\$iR" ;
echo "$iR" ;
 j="${arrayDebNames[i]}"  ;
echo "\$j" ;
echo "$j" ;
 jR=`md5sum $j|cut -d ' ' -f 1` ;
echo "\$jR" ;
echo "$jR" ;
# read FAKE ;
if [[ $iR == $jR ]] ;
then 
k=`echo $i+1|bc`
echo "\$k" ;
echo "$k" ;
 echo ;
 echo ;
 echo ;
 echo ;
 echo ;
 echo ;
 mv -v ${arrayDebNames[i]} Old/ ; umount -v d-$k ;
 rmdir -v d-$k ;
 df -h . ;
 echo "Sleeping 2 minutes, usually even more to next mv..."
 echo ;
 echo ;
 echo ;
 echo ;
 echo ;
 sleep 120 ; 
fi
df -h . ;
# read FAKE ;
done
echo "============================================================"
EDIT START
Mon Jan 6 16:10:50 UTC 2014
Oh, no, I forgot more stuff.
The script needs two files to be in that directory you created as explained.
And pls see the post after next that I will write next, for how to crete those files.
They are, names hardwired in the script, debian-DVD-list and MD5SUMS-bare
and, maybe one more thing, if you change the number of ISO that you download, than the line:
for ((i=0; i <= "20" ; i++ ))
needs to reflect.that. 20 is for 21 files, 0 to 20.
EDIT END

Now I guess you need to peruse the scripts and see that they do simple tasks mostly.
I don't think there's much explanation on what
jigdo-dl_01-jigdo-templ-t-s.sh
jigdo-dl_02-mnt-t-s.sh
jigdo-dl_03-dLo-SUMS.sh

do.
Also
jigdo-dl_04-dLo-t-s.sh
is easy to understand.

Ah, I forgot:

https://wiki.debian.org/jigdo

Read there if you haven't yet, and, of course, install the jigdo-lite !

$ man jigdo-lite

And the moving and deleting and unmounting is done with:
run_CMD_00.sh
given above.

EDIT START Tue Jan 7 02:12:44 UTC 2014
"(and as root)" was needed to be added, mounting and things... can't do common user
EDIT END
In this way (and as root):

Code: Select all

$ run_CMD_00.sh 10000 60 ./iso_check-del_00.sh
It doesn't have to be 1000, and it doesn't have to be 60, but that's my measure.

I got to rush now. I hope someone can even improve on these scripts.
Pls, this is really not much, but consider them under GPLv3 or later licence. And don't disregard that I came up with this idea.
Miroslav Rovis
Zagreb, Croatia,
http://www.CroatiaFidelis.hr
Last edited by timbgo on 2014-03-14 09:59, edited 4 times in total.
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#2 Post by timbgo »

I know it is simpler for the newbies if I uploaded the scripts on my site, and I will probably do it, but I don't know when and I can promise nothing.
I have two or so days ago been hacked, through my mobile (Google Android, may God pay you, Google the NSA's associate who sell our date to the NSA and to whoever pays, whichever country secret servece or other subjects. Money that grows in people's hearts suffocates and strangle those hearts, mend your ways!)...
And I am still dazed.
It was on my Gentoo GNU Linux and surely my GPG keys have been compromised. Can't used them.
The new has been revoked. The old that I made expired three months ago, I believe I don't need to revoke, but if anyone more knowledgeable than me knows better, pls. tell me...
I belive with GPG keys I can not transition anymore at all, such as it is explained here:
https://www.apache.org/dev/key-transition.html
right?
Sorry for the digression!
Miroslav Rovis,
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#3 Post by timbgo »

The files that the names of are hardwired in the script, as I just wrote in the edit in the first post of this tip, are, for today's jigdo download for amd64 and source, these:

debian-DVD-list

Code: Select all

debian-testing-amd64-DVD-1.iso
debian-testing-amd64-DVD-2.iso
debian-testing-amd64-DVD-3.iso
debian-testing-amd64-DVD-4.iso
debian-testing-amd64-DVD-5.iso
debian-testing-amd64-DVD-6.iso
debian-testing-amd64-DVD-7.iso
debian-testing-amd64-DVD-8.iso
debian-testing-amd64-DVD-9.iso
debian-testing-amd64-DVD-10.iso
debian-testing-amd64-DVD-11.iso
debian-testing-amd64-DVD-12.iso
debian-testing-source-DVD-1.iso
debian-testing-source-DVD-2.iso
debian-testing-source-DVD-3.iso
debian-testing-source-DVD-4.iso
debian-testing-source-DVD-5.iso
debian-testing-source-DVD-6.iso
debian-testing-source-DVD-7.iso
debian-testing-source-DVD-8.iso
debian-testing-source-DVD-9.iso
and MD5SUMS-bare

Code: Select all

a48989385de66c9b7fa179efde2719d7
d2addd6f0b7b66f327266b4641e72fd2
32e1837190f1395d0ec8af8250fde893
5501fea3c6189f7a8c49b39826d12533
f7782069a2252d1fd341615a52897426
f9ca0b5ab1042fa734ef4f43b7e8ff7f
0ea96445aff6bd25e6f28b7083a42854
d1484e033dd422dc2bafcea9c7919526
2d80fed4e2202a14eed5bcef2ccce01c
c1fb28f2ea198f96260c420161b72422
e39e421ffec161dabb42c6b3c23658f9
0a3241457510a8dffa2134c33cfac551
c8a349c78850c7bf92654cfc4affb676
d8c3aff348c365c2a3b18f08db9e7967
b68ba47555ed7e98ad40f489c9f4a627
a374f096e93d0c1c40b20af4112b13b9
6a1d70d7a6041478a74c8975c357b706
2590a8a35b0a19f6bbe3b7b4e792dbf6
1e0521b87d24247fa57471f186f43cdf
979223bb927db6bd13658bc9dbf5895d
7c9722f5bc1ae969121e638d20dcf8c6
Of course we could have used say SHA256SUMS, or any other of the sums, instead.
They are derived from the downloaded MD5SUMS but I edit it with vim or emacs to get it in the order real decimal, not 1, 10, 11, 12, 2 as it is in the downloaded file, but 1, 2, 3, 4, 5, 6 ,7, 8, 9, 10, 11, 12 and then the 9 source MD5SUMS.
No more time.
I'll be back to see if this works for other people. Will try and not be off for very long now. But, as Spender, young man, but my great hero, said somewhere, [ paraphrasing] the Linux audience is not very perceptive, and (I add) it's not very greatful...
Miroslav Rovis,
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#4 Post by timbgo »

The jigdo downloads, the MD5SUMS for the AMD64 testing branch, which I renamed MD5SUMS-t I sign to identify it with pretty much a unique amongst all the numbers in the world, lest tremendous power to counterfeit it were used, only that much the MD5 is broken:

MD5SUMS-t.asc:

Code: Select all

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAABCgAGBQJTA5EwAAoJEOqYhIhPuvCuQiEQAIJokM34W3u9IQJadTqA4V+I
jdEF4nzrTf1mGQE2jegOB8D6z45AHt92niAeNGdaovvHsSqhx6+B0kdj11kYpOMK
jyQNvsM5rI4okG++exPnm4wBmJqYzVEOOdoRshu4VUl9kzF31MVxHbXlOhUc9O2a
Vlm8oqgnEgknauYxtGAfu/qcpj1yhD5MVE2gvewNpoE04sPGkE29CzIfisyZlPh+
IKKdM5wMSuRoHnjZho9w6TecVwmGy2C3lyemNCQRCvQLQrS7Qm+jJMKrrS4TfHlu
sW41QcnONZAZHYwDiQJrhMz4lfmG5FVLIo6PjUUo5I3ve+pHXHz0izqbQeKRodcy
m4h1nEKVvn/Wz/UvGLbe+pL0LKVAcKlxBeEVp+vpiSZZFGSzyM+eT6BBILWdL6v+
PpwuCFEeVqcs7s/hbrGEuPxbrSqEaDYQT0E8t+hld8+c3khnQRReQSgMZ5pSu1rR
qUaEwOwHk8xZYRvhBBy0M2O3KuDVSp33xLVwRtknyY1T2Ik3HU7ZuW5tSnmIMUFh
Aogloobazql984TtRlXrisUkicZU8ivRHBc333JDeeydRxfoRJ4Ar2TCTCAHKOzs
8KDvBblJoIBGLH1aKk3emcrIf1jNhd1etq4cir2tBkNHXEHbK1rQZzhJE6RSke69
36YXRyNuoVEQLwcAHlHt
=A8eM
-----END PGP SIGNATURE-----
And jigdo downloads, the MD5SUMS for the sources branch, which I renamed MD5SUMS-s I sign:

MD5SUMS-s.asc:

Code: Select all

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAABCgAGBQJTA5EiAAoJEOqYhIhPuvCuSaUP/2HvVGRyI8FHLems87Orvww/
jknOwfyNS+ftr+Uol+sF15pN+v4o2+KvGeClsSqkPuwmOSYsCKX8CohPFy35Rm4i
CZmpzS8LsUct5GSHyY4Kg7jgdhTftVq2ajjUf5RGG0F+ph4FxiM8BBVxs7WoFUDo
q80jfXiSyJy/m22ez5imlcOTTnqn1eFYP7wA2V3IabPNwsODSD/3mhTPMOp9EF+w
U4VrJLAd/NaabAeXQkwiL+yJkdFw2SIjKTkKF6oL4XXKQzXmlrzeVJh6JSJBNHpX
gtVf/oR+2UqJwcphGdh1TeiwdfdZY1WxYCk45VQ+s8U46M2SyEXFrz7K6kK01ZsF
Veukp3TyJ1q1PBf5Oj9YewLwBa0CEhmuW8i1bv+bLEVAVgY5vANn9pQXoI3efJf1
SWThpUY2GoAwkEhgiLat+EgdYMfOiMeJpCnOgGQw8U0TCMK7zaTwsTOCkupX0ZUI
tUSJs1fDC8SDhpwXFkKtOCXKdjOrQAJ2UWg5WoA2X0+9VUWobe7zyW8D/684xECT
R2f6PfdJOb9j4YN7gUJsWE4I0Uodt1RIk+FNdvKCVhHi7+AnpfI0vCts45ryOiSU
glGC/OX8lVPrWYuuh3FfXhl3+caL64vWiL+qAri9T+6RuLzWN/1JQsnjeY6fAgVe
WWvpnmzAgnAByzV+Ze2D
=5oGw
-----END PGP SIGNATURE-----
And those of you who are patching the pristine kernel with Grsecurity, can se the sed command to use today and in the future, that I am posting today on the:
Grsecurity/Pax installation on Debian GNU/Linux
http://forums.debian.net/viewtopic.php?f=16&t=108616
Tips page,

Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#5 Post by timbgo »

There is instead of run_CMD_00.sh another instance of iso_check-del_00.sh apparently, on quick glance.
I plan to post signed versions, God permitting, so there might soon be doubts on this method, but in case the gentle reader needs the real run_CMD_00.sh before that time (times are never certain with me), here the run_CMD_00.sh that workd for me right now:

Code: Select all

#!/bin/bash

LIMIT=$1

for ((i=1; i<=LIMIT; i++))
do

$4 $3
sleep $2
echo " "

echo " "
echo "---------------------------------------------------------------------------"


done

exit 0
Just the arguments are not sorted. This (which you can find furthe --and earlier--- above):
run_CMD_00.sh 10000 60 ./iso_check-del_00.sh
would not work.
This:
run_CMD_00.sh 10000 60 anything ./iso_check-del_00.sh
should work.
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#6 Post by timbgo »

Here are the scripts that I use so I, whom the, well arguably criminals, in power in my country keep on slow connection (thanks Iskon, that's my provider, Iskon, Zagreb, Croatia, thanks a lot!)...

So here are the scripts that I use so I can go to sleep and not worry to wake up and see that my system has run out of space for download, or the next ISO is not being downloaded... of the Jigdo ISOs testing and source, from:
http://cdimage.debian.org/cdimage/weekl ... jigdo-dvd/
http://cdimage.debian.org/cdimage/weekl ... jigdo-dvd/

Here trhe scripts:

http://www.croatiafidelis.hr/gnu/deb/jigdo/

Verify the sigs.

chmod 755 both the scripts and choose XXX_01.sh, better.
Move into /usr/local/bin the run_CMD_01.sh
mv -iv run_CMD_01.sh /usr/local/bin
Do *not* move the iso_check-del_01.sh[/code] into there, but, I think I explained earlier, so read (this is the top of page):
http://forums.debian.net/viewtopic.php?f=16&t=110503
if you haven't yet...
...And run:

Code: Select all

run_CMD_01.sh 10000 30 ./iso_check-del_01.sh
And, don't know if very soon or who knows when, I intend to make these scripts so one can choose to download just the testing branch, because most people don't need the sources...

Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#7 Post by timbgo »

Here the complete set of scripts that I used just yesterday. Previous posts apply for more explanation.

http://croatiafidelis.hr/gnu/deb/jigdo/

The listing:

Code: Select all

me@mybox$ ls -l
total 44
-rwxr-xr-x 1 mr mr  733 Jan  6 13:11 iso_check-del_00.sh
-rwxr-xr-x 1 mr mr  778 Mar 14 11:23 iso_check-del_01.sh
-rwxr-xr-x 1 mr mr 4790 Mar 14 08:22 jigdo-dl_01-jigdo-templ-t-s.sh
-rwxr-xr-x 1 mr mr 1360 Mar 14 08:21 jigdo-dl_02-mnt-t-s.sh
-rwxr-xr-x 1 mr mr  910 Dec 15 03:31 jigdo-dl_03-dLo-SUMS.sh
-rwxr-xr-x 1 mr mr 1507 Mar 14 08:25 jigdo-dl_04-dLo-t-s.sh
-rwxr-xr-x 1 mr mr  186 Oct  7 15:12 run_CMD_00.sh
-rwxr-xr-x 1 mr mr  183 Mar 14 11:25 run_CMD_01.sh
-rw-r--r-- 1 mr mr  697 Mar 15 08:13 SUMS
-rw-r--r-- 1 mr mr  819 Mar 15 08:13 SUMS.sig
me@mybox$ 
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#8 Post by timbgo »

Pls. use:

http://croatiafidelis.hr/gnu/deb/jigdo/

for the current, latest edition of the set of scripts.

But for the users comfort, pls. here's the link:

http://www.croatiafidelis.hr/gnu/deb/ji ... pts.tar.gz

In case anything happens, pls. share it with one another. It's little but useful, and I am happy to have contributed, however verly little, to the GNUer and free-er world,

Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#9 Post by timbgo »

I studied the whole weekend the several git things.
(Well it wasn't the first weekend dedicated to it, but this time I made it... No, it's not the github which is hard, it's the git)

And I think I made a fine repo, for a first git repo of mine:

miroR/jigdo-automate-scripts

https://github.com/miroR/jigdo-automate-scripts

IMPORTANT: if anyome, and there are people in Debian who could make it in a breeze (but, OK they work on so much more important things...), would like to improve the scripts, I am willing to apply their patches. Just let me say upfront. I still need to learn a lot about those, and I am often not available for days and weeks. But I will apply my best understanding once I am available to view any possible contributions.

Sure, now that the sums are digitally signed, it would be nice to include the checking, and actually improve the scrips further.
I'll try.
Can't promise, but will be trying,next.

Miroslav Rovis,
http://www.CroatiaFidelis.hr
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#10 Post by timbgo »

It's signing in git that I stumbled upon, because I don't want to use gpg-agent and they seem to be forcing it upon users.
That was short explanation.
Longer coming in a separate thread.
EDIT START Here:
gpg-agent now forced upon users of GnuPG
http://forums.debian.net/viewtopic.php?f=3&t=114427
EDIT END

So, pls. try the new version, surely no guarranties, I am only now going to test it to download the new Jigdo Testing AMD64 DVDs.
http://croatiafidelis.hr/gnu/deb/jigdo/ ... e-scripts/
read carefully at the top what I wrote a while ago.
Don't use the github today, it is only the old.version there.

Miroslav Rovis
http://www.CroatiaFidelis.hr
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

timbgo
Posts: 265
Joined: 2013-04-14 12:17

Re: Scripts to automate jigdo download

#11 Post by timbgo »

I seem to have finally learned (after some research each time; I use git off and on only), and the:

https://github.com/miroR/jigdo-automate-scripts

is now PGP signed, the tags.

Just use the latest. Beginners should be able to use it too, although they would need to really apply, read the man pages and other.

Miroslav Rovis
Zagreb, Croatia
www.CroatiaFidelis.hr
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr
Anyone can dismiss these: kernel hooks for rootkits
linux capabilities for intrusion?

Post Reply