Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

command issued from within bash script not working

Programming languages, Coding, Executables, Package Creation, and Scripting.
Message
Author
User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

command issued from within bash script not working

#1 Post by PsySc0rpi0n »

Hello again.

I have been working in a simple script to run a command in an automated fashion.

This script is supposed to build the command that can take a significant amount of parameters, append quite some special chars an values to it and then issue it in terminal.
I managed to make the script to build the command I intend to run in terminal. The command is correctly built,I can assure that because I did a "printf" of the command built by the script, ran it directly in the terminal and it worked smoothly.

So, are there any common/known reasons for a command not run from inside a bash script? If needed, I'll paste here the piece of code that builds the command and will also paste the command built and ready to run directly in terminal.

Thanks
Psy

User avatar
ruwolf
Posts: 618
Joined: 2008-02-18 05:04
Location: Banovce nad Bebravou
Has thanked: 34 times
Been thanked: 26 times

Re: command issued from within bash script not working

#2 Post by ruwolf »

I do not know such program (which cannot be run from bash script).
BTW, there are 2 types of commands: built-in utilities (in shell) and stand-alone programs.

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#3 Post by PsySc0rpi0n »

ruwolf wrote:I do not know such program (which cannot be run from bash script).
BTW, there are 2 types of commands: built-in utilities (in shell) and stand-alone programs.
I don't say it cannot run, but for some reason, it is not running. The command, instead of executing, returns it's own menu help as if the command syntax is wrong, but it is not because I added a line to "echo" the built command and copied it to the terminal and ran it and it worked.

I think this might be related somehow with bash and the program itself, which is a bridge to a JSON application to run RPC commands.
To be more specific, I'm trying to run a bitcoin core command. This command is bitcoin-cli with a bunch of parameters that will communicate with an already running bitcoin daemon which then sends the command to a JSON server and returns the response to the daemon which will in turn send it to this bitcoin-cli program that I'm using from within my script!

To make it simpler, the workflow is like:

bitcoin-cli sends an "user-friendly RPC" to bitcoind daemon.
bitcoind ddaemon converts this command into a JSON RPC and sends it to the JSON server
server responds to bitcoind daemon
bitcoind deamon converts this JSON response into a JSON object properly formatted and sends it to bitcoin-cli
bitcoin-cli shows that JSON object in terminal with the result of the JSON RPC.

Somewhere in the middle, my script, which simply builds this command that bitcoin-cli sends to bitcoind daemon, probably have some issue that prevents the command to be correctly sent from bitcoin-cli to bitcoind.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#4 Post by Dai_trying »

I would prefer Python for such a task, one guide that help in your case would be https://upcoder.com/7/bitcoin-rpc-from-python/ or if you want a youtube guide you could try https://www.youtube.com/watch?v=3xAO4K7FC6E. I don't use bitcoin-cli so cannot test it but I would think it would be easier for you once you grasp the basics of Python.

v&n
Posts: 624
Joined: 2015-02-04 02:57

Re: command issued from within bash script not working

#5 Post by v&n »

PsySc0rpi0n wrote:If needed, I'll paste here the piece of code that builds the command and will also paste the command built and ready to run directly in terminal.
I believe that should be the first step when asking for help on a script.

User avatar
pylkko
Posts: 1802
Joined: 2014-11-06 19:02

Re: command issued from within bash script not working

#6 Post by pylkko »

you need to show the script, otherwise it's just guessing. Also, you need to tell us how it is being launched. You progbably mean that you have commands that you run manually with your user and they work, but they do not in the script. If so, then are you running it under another user, with different privileges and environment?

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#7 Post by PsySc0rpi0n »

Dai_trying wrote:I would prefer Python for such a task, one guide that help in your case would be https://upcoder.com/7/bitcoin-rpc-from-python/ or if you want a youtube guide you could try https://www.youtube.com/watch?v=3xAO4K7FC6E. I don't use bitcoin-cli so cannot test it but I would think it would be easier for you once you grasp the basics of Python.
Well, I started this with bash script an I already have other 4 functionalities working. So, it would be throwing work out to the basket can if I just choose to change now! At this point this is the last functionality I need to get working. So, for now, I'm sticking to bash script mainly because it's kind of the correct choice to run commands in terminal an automate tasks.

But thank you for the links. They might be useful in the future if I see I can't do something with bash!

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#8 Post by PsySc0rpi0n »

Code: Select all

send_to_mult_addr(){
   load_addr_data

   val=$(printf "%.9f" $(echo $(bitcoin-cli -testnet getbalance) / "$num_addr" | bc -l))

   printf 'Value to send: %0.8f BTC\n' "$val"
   while true
   do
       echo "Confirm with YES or cancel with NO (caps matter)"
       read -r -p '> ' opt
       case $opt in
           "YES") count=0
                  com_params="\"\" \"{"
                  for i in "${addr_arr[@]}"
                  do
                     ((count++))
                     com_params+="\\\"$i\\\""
                     if [ "$count" -lt "$num_addr" ]; then
                        com_params+=":$val,"
                     else
                        com_params+=":$val"
                     fi
                  done
                  com_params+="}\" 6 \"Periodic payments\" \"["

                  count=0
                  for i in "${addr_arr[@]}"
                  do
                     ((count++))
                     if [ "$count" -lt "$num_addr" ]; then
                        com_params+="\\\"$i\\\","
                     else
                        com_params+="\\\"$i\\\"]\""
                     fi
                     done
                     com_params+=" true 6 CONSERVATIVE"
                     echo "bitcoin-cli -testnet sendmany "$com_params""
                     #bitcoin-cli -testnet sendmany "$com_params"
                     printf 'TxID: %s\n' "$?"
                     return 1
                     ;;
           "NO") echo "Action cancelled!"
                 return 2
                 ;;
       esac
   done
}
Basically this function builds a bitcoin-cli command to send bitcoin to BTC addresses dynamically, depending on how many addresses it loads from a file.

The line of code that holds the complete command is:

Code: Select all

bitcoin-cli -testnet sendmany "$com_params"
So, to test the built comman, I "echo'ed" it and then I ran the script so that the built command is printed in the terminal. Then I copied the entire command and ran it directly terminal. And it worked correctly. However, if I run the script and execute the command from within the script, the output is the menu help for the "sendmany" option. It just says "Error code: -1" and the menu help.

So, knowing that the command works correctly if issued directly from the command line, what could be going on?
I attached an image where the error message can be seen. Not that it helps much, but....

Thanks
Psy

Edited;
I can't attach the file... It says the maximum number of attachments was achieved. Although it was only a single image file!

User avatar
Bloom
df -h | grep > 90TiB
df -h | grep > 90TiB
Posts: 503
Joined: 2017-11-11 12:23
Been thanked: 26 times

Re: command issued from within bash script not working

#9 Post by Bloom »

From what I can see, you defined a big Bash function and then didn't call it. The result from this would be no output.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#10 Post by Dai_trying »

Not sure if this is the case with your script or not but I recall having similar issue with a script and it was caused by the command list being seen as a single item, a bit like using quotes with parameters when calling a script to allow spaces in text etc.

To demonstrate what I mean use a simple script like this

Code: Select all

#!/bin/bash

echo "Parameter 1 = $1"
echo "Parameter 2 = $2"
example1

Code: Select all

dai@trying:~$ ./test_script some text to say
Parameter 1 = some
Parameter 2 = text
example 2

Code: Select all

dai@trying:~$ ./test_script "some text" "to say"
Parameter 1 = some text
Parameter 2 = to say
example 3 (what I think is happening with your command)

Code: Select all

dai@trying:~$ ./test_script "some text to say"
Parameter 1 = some text to say
Parameter 2 = 
So what you might need to do is separate the different parameters required and use a different variable name for each and apply them individually, something like this

Code: Select all

    bitcoin-cli -testnet sendmany "$com_params_1" "$com_params_2" "$com_params_3"
Obviously the amount of parameters required by the command will dictate how you will need to assign the relevant data.

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

command issued from within bash script not working

#11 Post by PsySc0rpi0n »

And how am I supposed to add the "", {}, [], :, \ and the , ?

Sent from my SM-G935F using Tapatalk

User avatar
ruwolf
Posts: 618
Joined: 2008-02-18 05:04
Location: Banovce nad Bebravou
Has thanked: 34 times
Been thanked: 26 times

Re: command issued from within bash script not working

#12 Post by ruwolf »

When you give string in bash command argument to apostrophes (“'”, ASCII code 0x27), it will be not interpreted, it will by used literality as you has written it.
E.g.:

Code: Select all

echo '"", {}, [], :, \ \n \r ? '
will write it as exact copy:

Code: Select all

"", {}, [], :, \ \n \r ?
Another method for disabling interpretation of special character is writing it after back-slash (“\”, ASCII code 0x5C)
You can find fore information in manual of bash, section Quoting.

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#13 Post by PsySc0rpi0n »

ruwolf wrote:When you give string in bash command argument to apostrophes (“'”, ASCII code 0x27), it will be not interpreted, it will by used literality as you has written it.
E.g.:

Code: Select all

echo '"", {}, [], :, \ \n \r ? '
will write it as exact copy:

Code: Select all

"", {}, [], :, \ \n \r ?
Another method for disabling interpretation of special character is writing it after back-slash (“\”, ASCII code 0x5C)
You can find fore information in manual of bash, section Quoting.
I'm not sure I explained myself accurately.

If I just use the escape char to escape them, won't they be seen as a parameter themselves? I don't know if I try to add the special chars to the parameters themselves, it will probably turn into errors.

I guess I'll have to try.
Probably trying to use 'eval' won't be of much help either, right?

I think I'll try something like:

param1="\"\"", because I need this to be seen by the bitcoin-cli app exactly as doubles quotes which means a first empty parameter.

Then I need to open the list of those long strings with an "{. I don't know if I can pass this as 'param2'. I'll need to close this later.

Then I'll need to dynamically append the long strings, according to what is read from a file and that is already stored in an array in memory.

What I want to say is that I cannot add the special chars statically because I always need to append strings dinamically. Or maybe I'm not figuring out the whole picture!

I'll give it a few tries when I get home!
Thanks
Psy

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#14 Post by Dai_trying »

One more thing to think about, would it be easier (programmatically) to loop through your list and send each pair individually, I realise this would take the script longer to complete and would also result in making a new connection as it sends each each one (pair), but it might be easier to program for now.

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#15 Post by PsySc0rpi0n »

I don't think this can be done like this. Or at least I have no idea how it can be done!

I can't build the command just by doing something like:

Code: Select all

bitcoin-cli -testnet senmany $param_1 $param_2 $param_n
due at least 2 reasons.

First, I on't know how would I add the in between commas, brackets, spaces, etc.

The syntax is as follows:

Code: Select all

bitcoin-cli sendmany "" "{\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX\":0.01,\"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz\":0.02}" 1 "" "[\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX\",\"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz\"]"
where the first pair of double quotes is a parameter. then starts a block with ' "{' and closes with the matching '}" '. Inside this block there are 2 parameters or sub parameters which are the pairs

Code: Select all

\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX\":0.01
an are separated with a comma. Then comes the 1 which is another parameter. Then another pair of doubles quotes that may (or not) be empty, and so one. So these special chars that are intended to separate items in a list or to separate parameters, must be appended but not as parameters, I guess.

An 2nd, I would completely be unable to add more parameters if the number of parameters in the file changes.
I mean, if I try to write the script by manually writing each param as I did above, how would I add more parameters dynamically if the number of parameters in the file changes?

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#16 Post by Dai_trying »

You are confusing parameters with quantity of transactions, a parameter is information passed to a command, if you look at the api reference for bitcoin you will see that you have to submit parameters with the command (up to 8 for the sendmany option) so the number of parameters is static, it is the quantity of entries inside of each parameter that changes.

Looking through the api reference (https://bitcoin.org/en/developer-refere ... -core-apis) it would seem like you could use the SendToAddress to send an individual transaction, and it looks like it might be simpler to build the command. To do this you would read each line from your file and build the command from that one line, send it (using SendToAddress and then move to the next line etc, etc, etc...

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#17 Post by PsySc0rpi0n »

Dai_trying wrote:You are confusing parameters with quantity of transactions, a parameter is information passed to a command, if you look at the api reference for bitcoin you will see that you have to submit parameters with the command (up to 8 for the sendmany option) so the number of parameters is static, it is the quantity of entries inside of each parameter that changes.

Looking through the api reference (https://bitcoin.org/en/developer-refere ... -core-apis) it would seem like you could use the SendToAddress to send an individual transaction, and it looks like it might be simpler to build the command. To do this you would read each line from your file and build the command from that one line, send it (using SendToAddress and then move to the next line etc, etc, etc...
The quantity of parameters may change. Some of them are optional. But to make things simpler, I just use them all to avoid having to deal with checking if mandatory and optional are preset or not.

I understand I might have misused the correct words. Anyway, what I mean is that I need to find a way of dynamically insert the items inside those 2 parameters which are lists of items.
And also, I need to figure out out to distinguish parameters from wrapping brackets, braces, colons and etc.

What about "eval" command? Would it be useful here?

I don't have problems with "sendtoaddress". That one is working nice. The thing with using "sendmany" is that it allows to pay way less in fees for each transaction. So, I have all the interest in using this command over the other. The other is only to be used in exceptions.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#18 Post by Dai_trying »

PsySc0rpi0n wrote:I need to figure out out to distinguish parameters from wrapping brackets, braces, colons and etc.
I can see how it could be difficult to fully comprehend these things and I cannot say I do myself really, but this is one of the reasons I said I would use Python for this kind of script as you can easily create json "objects" which take care of formatting for you, I will take a look and see if I can put something together when I have more time but am a little limited at the moment.
PsySc0rpi0n wrote:What about "eval" command? Would it be useful here?
I really try to avoid using eval, I have used it in the past but only because I was pushed at the time, and eventually i found another way but even then I'm not sure it would work.
PsySc0rpi0n wrote:I don't have problems with "sendtoaddress". That one is working nice. The thing with using "sendmany" is that it allows to pay way less in fees for each transaction. So, I have all the interest in using this command over the other. The other is only to be used in exceptions.
I can see why it is better for you to do it this way now and will keep that in mind.


I should be able to look into doing a python script over the next day or two and will post something when I have it.

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#19 Post by PsySc0rpi0n »

Dai_trying wrote:
PsySc0rpi0n wrote:I need to figure out out to distinguish parameters from wrapping brackets, braces, colons and etc.
I can see how it could be difficult to fully comprehend these things and I cannot say I do myself really, but this is one of the reasons I said I would use Python for this kind of script as you can easily create json "objects" which take care of formatting for you, I will take a look and see if I can put something together when I have more time but am a little limited at the moment.
PsySc0rpi0n wrote:What about "eval" command? Would it be useful here?
I really try to avoid using eval, I have used it in the past but only because I was pushed at the time, and eventually i found another way but even then I'm not sure it would work.
PsySc0rpi0n wrote:I don't have problems with "sendtoaddress". That one is working nice. The thing with using "sendmany" is that it allows to pay way less in fees for each transaction. So, I have all the interest in using this command over the other. The other is only to be used in exceptions.
I can see why it is better for you to do it this way now and will keep that in mind.


I should be able to look into doing a python script over the next day or two and will post something when I have it.

Ok, thank you a lot.

I'm looking to 'jq' command which allows one to filter data from a JSON object. Despite the fact that what I want is the other way around, I don't know if I can pull something out from here. I'll stick to bash script until you get something out of python. Because if I'm "forced" to switch to python, I'll have to re-learn the basics again. I did some basic learning in the past but then I had to quit because time was scarce!

What I'm thinking now is to write a file in JSON format using my already done printf with the complete command (which is seen as a single long string) and then try to format it with 'jq' and maybe be able to build the command from here.

I think the JSON things here in this command are just the 2 parameters that are wrapped in {} and in []. The other parameters I think I can just append them in a later stage right before issuing the complete command from the script!

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#20 Post by Dai_trying »

I had some time this morning to look at this and although this might not work I thought I would let you see what I have so far as it's not too far off.

First I created a quick bash script to act as bitcoin-cli simply to interpret the parameters

Code: Select all

#!/bin/bash

echo "Param 1 = $1"
echo "Param 2 = $2"
echo "Param 3 = $3"
echo "Param 4 = $4"
echo "Param 5 = $5"
echo "Param 6 = $6"
echo "Param 7 = $7"
I included 1 more parameter than you would be using in case there was some extra info being sent.

Next the file with codes

Code: Select all

1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX
1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz
I named it BitcoinList.txt and you would need to edit the script to reflect the actual file you use.

Then the Python script:

Code: Select all

import json
import subprocess

a = 0
d = {}
e = []

with open('BitcoinList.txt') as input_file:
    for i in input_file:
        a += 1
        b = a/100
        d[i.strip()] = '%.2f' % b
        e.append(i.strip())

r = json.dumps(d)
s = json.dumps(e)

subprocess.check_call(['/path/to/bin/bitcoin-cli', 'sendmany', '""', r, '1', '"Description comment"', s])
which gives me the result :

Code: Select all

Param 1 = sendmany
Param 2 = ""
Param 3 = {"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX": "0.01", "1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz": "0.02"}
Param 4 = 1
Param 5 = "Description comment"
Param 6 = ["1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX", "1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz"]
Param 7 = 
What is left to do is adding the escapes (\) and maybe some extra quotes but I'm out of time for today so thought I would get this out in case you can tweak it, it is quite a simple script so should not be too difficult to do.

HTH

Post Reply