Puzzle for Programmers

Need help with C, C++, perl, python, etc?

Puzzle for Programmers

Postby Blyiss » 2007-02-23 21:40

Hello, guys. I need a script for moving files.
There are a lot of examples and tutorials but I can't find anything like this:
I have a folder "DIR" that contains subfolders "DIR/a", "DIR/b" and "DIR/c". And each subfolder contains files ",1", ",2" and ",3".
Question: how can I move these files to the subfolder DIR/x in such a manner that files would be renamed and lined up in the following order: "1, 2, 3, 4, 5, 6, 7, 8, 9", if the folder DIR/x is empty. And if it already contains some files, the order would continue starting from the last name of the file like : "10, 11, 12 and so on" ?

Will be very grateful for any help.
Blyiss
 
Posts: 585
Joined: 2007-02-10 19:47
Location: Yakutia

Postby Grifter » 2007-02-24 01:29

better explanation would be nice
Eagles may soar, but weasels don't get sucked into jet engines...
Grifter
 
Posts: 1572
Joined: 2006-05-04 07:53
Location: Svea Rike

Postby Blyiss » 2007-02-24 02:49

Grifter wrote:better explanation would be nice

OK, I will try. As you know, I am playing with nmh, which is a flavor of mh emails managing program - Message Handling System. When I delete messages in mh, they are not actually deleted but renamed by adding a "," character at the beginning of the file. But mh thinks they are gone and doesn't list them any more. So there is no possibility to view them again in MH, until your rename them back. MH files have very simple names: "1", "2", "3" and so on. When they are deleted, they become ",1", ",2", ",3" and so on. And such a thing happens to any folder like "Inbox", "Outbox","MY Favourite Emails", etc. It is not a big deal to remove these files permanently using a simple one-line bash script. But I want to rename them back so that they become visible again and move to the "Trash" folder, where I can keep them and do to them whatever I want. And here appear two problems:

1. What pattern can be applied for renaming the files with such short names, and
2. How to move the files with the same names from multiple folders to a single one. They will be overwritten, I guess.

I believe there should be a way like first collecting files names somewhere in the memory and then renaming them in order. For example, if I have file ",1" from "Inbox", file ",1" from "Outbox" and file ",1" from "Drafts", they would become "1", "2", "3" and go to the "Trash" folder. But there's another problem. If the "Trash" folder already contains files "1", "2", "3", they will be overwritten. That's why it would be nicer to check this folder first and, if it is not empty, to start renaming files beginning not from "1" but the name or number of the last file like "3" in this case so that the final result for six files would be "1","2","3","4","5","6", containing in the "Trash"
Where can I find info about how to do this?
Last edited by Blyiss on 2007-02-24 04:54, edited 1 time in total.
Blyiss
 
Posts: 585
Joined: 2007-02-10 19:47
Location: Yakutia

Postby Grifter » 2007-02-24 03:29

can you run

head -n 1 ,1

(this would be on a ,1 file of course)
perhaps a mail can be renamed to a part of its header, that might organize a little, but i'm unsure if they will be able to be read by... well whatever you use to read them, try to copy a ,1 file to tempname, and see if you can view the mail when it's called tempname, or if they absolutely _have_ to be named a digit
Eagles may soar, but weasels don't get sucked into jet engines...
Grifter
 
Posts: 1572
Joined: 2006-05-04 07:53
Location: Svea Rike

Postby Grifter » 2007-02-24 03:32

also check if they can have leading zeroes, like be called 00001 instead of 1
Eagles may soar, but weasels don't get sucked into jet engines...
Grifter
 
Posts: 1572
Joined: 2006-05-04 07:53
Location: Svea Rike

Postby Grifter » 2007-02-24 03:52

if you can have leading zeroes, it would be rather simple to make a move script, but if you can't it's going to be a little harder due to how ls sorts files ( 1 10 11 2 3 4 5 etc)

assuming you can have leading zeroes, and we decide to set 5 leading zeroes infront of mails (a theoretical max of 99 999 mails before the script falls apart) it would be simple to set a variable like this,

VARCOUNT=`ls /path/to/zeromails | tail -n 1`

now we'd have the number of the latest mail we added, then we simply add 1 to the var, create a loop where we use find to find all the ,x files, and in each iteration we we move the ,x file to /path/to/zeromails renaming it to the VARCOUNT variable, and then add 1 to the variable again, and close the loop, then it'll go for each iteration adding all the ,x mails as one number higher
Eagles may soar, but weasels don't get sucked into jet engines...
Grifter
 
Posts: 1572
Joined: 2006-05-04 07:53
Location: Svea Rike

Postby Grifter » 2007-02-24 03:55

of course, if we _can't_ have leading zeroes it's going to take a little more overhead, because when we create the variable we're going to have to put in multiple checks along the way, to first check if there are any filenames with say 5 digits, if not move down to 4 digits, if not move down to 3 digits, if we get a bite here, then we'll have to find the highest 3 digit number which is essentially the same, but as you can see it's a bit more overhead
Eagles may soar, but weasels don't get sucked into jet engines...
Grifter
 
Posts: 1572
Joined: 2006-05-04 07:53
Location: Svea Rike

Postby Blyiss » 2007-02-24 04:41

Sorry, I am not very experience in bash scripting. It is interesting, of course. Maybe in the future I will find some time to improve my bash understanding.

I checked. Digits are OK. MH reads any sort of digits in any order including any number of leading zeros. But characters don't work. MH doesn't accept characters.
Code: Select all
: ~/Mail/spam
$ head -n 1 ,1
Return-Path: <jna@ono.com>

What does it mean? Some email address.
Blyiss
 
Posts: 585
Joined: 2007-02-10 19:47
Location: Yakutia

Postby Blyiss » 2007-02-24 15:55

The guys from Linux forum say that bash cannot cope with such a task. They advise to use Perl instead. I'll take a look what Perl can do.
Blyiss
 
Posts: 585
Joined: 2007-02-10 19:47
Location: Yakutia

Postby drl » 2007-02-24 17:36

Hi, Blyiss.

Following what I understood your requirements to be, the result of running a small demo script are these lines:
Code: Select all
% ./s1

 Current state:
drafts:
,1

inbox:
,1

outbox:
,01  ,02

trash:


 Final contents of trash:
1  2  3  4

Here is the script that does this:
Code: Select all
#!/bin/sh

# @(#) s1       Demonstrate move and rename.

# Provide audit trail for learning and debugging.
# Interchange the two debug lines to toggle printing on and off.

debug="echo"
debug=":"

# Set up the test situation.
# For production, comment out the remove and create calls.

./remove
./create

echo
DIRS="drafts inbox outbox"
echo " Current state:"
ls -R $DIRS trash

echo
$debug " audit trail of operations:"
$debug

for d in $DIRS
do
  for file in $(ls $d)
  do
    $debug " Working on $d/$file"
    n=${file#,}
    $debug " extracted :$n: from :$file: in :$d:"
    m=$( expr $n + 0 )
    $debug " transformed $n into $m"
    xform=$m
    if [ -f trash/$xform ]
    then
      highest=$( expr $highest + 1 )
      $debug " set highest number to :$highest:"
      bump=$highest
      mv $d/$file trash/$bump
      $debug " moved (bumped) $d/$file to trash/$bump"
    else
      # echo mv $d/$xform trash
      mv $d/$file trash/$xform
      $debug " moved $d/$file to trash/$xform"
      highest=$xform
      $debug " set highest number to :$highest:"
    fi
  done
done

echo
echo " Final contents of trash:"
ls trash

exit 0

An important part of this is "normalizing" the numeric name so that leading zeroes are eliminated. Use as you will; as usual, no warranty ... cheers, drl
["Sure, I can help you with that." -- USBank voice recognition system.
( Mn, 2.6.x, AMD-64 3000+, ASUS A8V Deluxe, 3 GB, SATA + IDE, NV34 )
Debian Wiki | Packages | Backports | Netinstall
User avatar
drl
 
Posts: 434
Joined: 2006-09-20 02:27
Location: Saint Paul, Minnesota, USA

Postby Grifter » 2007-02-24 18:03

Blyiss wrote:The guys from Linux forum say that bash cannot cope with such a task. They advise to use Perl instead. I'll take a look what Perl can do.


Ech, people only say such a thing if they themselves don't know how to do it

drl - darn you beat me to it, but oh well, here's my version too, and while I don't think it will set fire to your house or impregnate your cat, who can be sure:

Code: Select all
#!/bin/sh

# Set this dir to be the location of where you want
# the files moved, be aware that this should be the
# dump for all deleted mails, it should not be an
# existing mailbox that has to compete with 'real'
# mails
UN=/changeme

# Set this dir to be the root of the mailstructure
# you want to search for ",filenames"
DEUX=/changeme

# Check for the latest mail, stripping the padding zeroes
# (let doesn't preserve the padding zeroes, so we need to
# re-pad them later on)
VARCOUNT=`ls $UN | sed -r 's/(0*)(.*)/\2/' | tail -n 1`

# Check if it's the first run, if not we'll let the var
# iterate higher, so the next filename won't overwrite
# the previous mail
if [ -z $VARCOUNT ]; then
   VARCOUNT=00001
   else
   let VARCOUNT+=1
fi

# Just making sure something isn't funky
if [ -f $VARCOUNT ]; then
        echo "ARRGH! File exists when it shouldn't! Funky Alert!"
        exit 1
fi

# Here we start the loop to find ",filenames", when it finds one
# it will test how many characters the current count has, first
# testing for 5 characters where no padding is needed, if the
# var doesn't have 5 chars it will test for 4, and pad one zero
# infront of the count, otherwise it will continue down the list,
# adding extra zeroes as needed, to make it 5 characters
for i in $( find $DEUX -name ',*'); do
        if [ -n "`echo $VARCOUNT | grep .....`" ]; then mv ${DEUX}/${i} ${UN}/${VARCOUNT}
        elif [ -n "`echo $VARCOUNT | grep ....`" ]; then mv ${DEUX}/${i} ${UN}/0${VARCOUNT}
        elif [ -n "`echo $VARCOUNT | grep ...`" ]; then mv ${DEUX}/${i} ${UN}/00${VARCOUNT}
        elif [ -n "`echo $VARCOUNT | grep ..`" ]; then mv ${DEUX}/${i} ${UN}/000${VARCOUNT}
        elif [ -n "`echo $VARCOUNT | grep .`" ]; then mv ${DEUX}/${i} ${UN}/0000${VARCOUNT}
        fi

# ",file" has been renamed to VARCOUNT with leading zeroes if needed, now we
# increase the count by one to prepare for the next iteration of the loop
let VARCOUNT+=1

# Closing loop
done

Eagles may soar, but weasels don't get sucked into jet engines...
Grifter
 
Posts: 1572
Joined: 2006-05-04 07:53
Location: Svea Rike

Postby Blyiss » 2007-02-24 19:02

Thank you, drl, for the script. It works great!
At first it complained that "ls: command not found", so I added:
Code: Select all
PATH=/usr/local/bin:/bin:/sbin:/usr/bin:/usr/sbin
MAILDIR=$HOME/Mail

And it went very well except that it moved to trash all the files it found:
(those "non-numeric arguments" are left-overs from the previous experiments)
Code: Select all
Current state:
drafts:
?1  #1#  ,2

inbox:
,00000000000000000000000001  11  ,14  *17  *19  *20  ,22  *23  26   ,27  3  7
00000000000004               12  ,15  18   2    21   23   ,24  ,26  28   5  ,8
100                          13  ,16  ,19  ,20  *21  ,23  25   27   ,28  6  9

outbox:
1   11  13  15  17  19  20  22  3  5  7  9
10  12  14  16  18  2   21  23  4  6  8

trash:
1

trash:
1

expr: non-numeric argument
expr: non-numeric argument
mv: cannot stat `inbox/100': No such file or directory
expr: non-numeric argument
expr: non-numeric argument
expr: non-numeric argument
expr: non-numeric argument
expr: non-numeric argument
expr: non-numeric argument
mv: `trash/?1' and `trash/?1' are the same file
mv: `trash/11' and `trash/11' are the same file
mv: `trash/21' and `trash/21' are the same file
mv: `trash/31' and `trash/31' are the same file
expr: non-numeric argument
mv: `trash/#1#' and `trash/#1#' are the same file
expr: non-numeric argument
mv: `trash/*17' and `trash/*17' are the same file
expr: non-numeric argument
mv: `trash/*19' and `trash/*19' are the same file
expr: non-numeric argument
mv: `trash/*20' and `trash/*20' are the same file
expr: non-numeric argument
mv: `trash/*21' and `trash/*21' are the same file
expr: non-numeric argument
mv: `trash/*23' and `trash/*23' are the same file

 Final contents of trash:
1  ?1  #1#  10  11  12  13  14  15  16  *17  *19  2  *20  *21  *23  3

Some emails disappeared. The amount of emails in the trash folder and the sum of emails from inbox , outbox, and drafts is not equal.
Here is one more example:
Code: Select all
 Current state:
drafts:

inbox:
,1  ,2

outbox:
1  2  3  4

trash:

trash:


 Final contents of trash:
10  11  12  7  8  9

Its OK here, except that it grabbed all the emails it found.
Can it be because I commented out ./remove and ./create files?
I will tray it a bit later again when I collect enough of emails
Blyiss
 
Posts: 585
Joined: 2007-02-10 19:47
Location: Yakutia

Postby drl » 2007-02-24 19:37

Hi, Blyiss.

I intended it to be for demonstration. For production, one would add lots of protection.

No, the create/remove commands (scripts) would not cause that. The script assumes that you have only files of the name format "comma followed by decimal numbers".

To cause the extra files to be ignored, you could replace the line:
Code: Select all
for file in $(ls $d)

with
Code: Select all
for file in $( ls -1 $d/ | grep '^,[0-9][0-9]*$' )


Then you would see something like this, noting that I added a few junk file names to draft and outbox:
Code: Select all
% ./s1

 Current state:
drafts:
#1#  ,1  ,44fox  ?1

inbox:
,1

outbox:
,01  ,02  ,hello,world  list,3

trash:


 Final contents of trash:
1  2  3  4

Best wishes ... cheers, drl
["Sure, I can help you with that." -- USBank voice recognition system.
( Mn, 2.6.x, AMD-64 3000+, ASUS A8V Deluxe, 3 GB, SATA + IDE, NV34 )
Debian Wiki | Packages | Backports | Netinstall
User avatar
drl
 
Posts: 434
Joined: 2006-09-20 02:27
Location: Saint Paul, Minnesota, USA

Postby Blyiss » 2007-02-24 20:22

Than you, Grifter, for a great script!
But if the trash folder is not in the mailbox, I won't be able to see its content from MH. I tried anyway:
Code: Select all
 rex: ~/Mail
$ ./movetotrash2
mv: cannot stat `/home/rex/Mail//home/rex/Mail/spam/,1': No such file or directory
mv: cannot stat `/home/rex/Mail//home/rex/Mail/inbox/,12': No such file or directory
mv: cannot stat `/home/rex/Mail//home/rex/Mail/inbox/,11': No such file or directory
mv: cannot stat `/home/rex/Mail//home/rex/Mail/inbox/,10': No such file or directory

It seems that it tries to count the files that don't exist.
Blyiss
 
Posts: 585
Joined: 2007-02-10 19:47
Location: Yakutia

Postby Grifter » 2007-02-24 20:42

it looks like it does the path twice, it could be because i got overintentive; in the if lines originally i had the lines be

then mv $i ${UN}/${VARCOUNT}

but right before posting i thought, oh hmm, well maybe i can foolproof it by putting the the $DEUX path before $i, try changing the ${DEUX}/${i} to just $i, in all those if lines, and that should clear it up
Eagles may soar, but weasels don't get sucked into jet engines...
Grifter
 
Posts: 1572
Joined: 2006-05-04 07:53
Location: Svea Rike

Next

Return to Programming

Who is online

Users browsing this forum: No registered users and 5 guests

fashionable