Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

command issued from within bash script not working

Programming languages, Coding, Executables, Package Creation, and Scripting.
Message
Author
Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#16 Post by Dai_trying »

You are confusing parameters with quantity of transactions, a parameter is information passed to a command, if you look at the api reference for bitcoin you will see that you have to submit parameters with the command (up to 8 for the sendmany option) so the number of parameters is static, it is the quantity of entries inside of each parameter that changes.

Looking through the api reference (https://bitcoin.org/en/developer-refere ... -core-apis) it would seem like you could use the SendToAddress to send an individual transaction, and it looks like it might be simpler to build the command. To do this you would read each line from your file and build the command from that one line, send it (using SendToAddress and then move to the next line etc, etc, etc...

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#17 Post by PsySc0rpi0n »

Dai_trying wrote:You are confusing parameters with quantity of transactions, a parameter is information passed to a command, if you look at the api reference for bitcoin you will see that you have to submit parameters with the command (up to 8 for the sendmany option) so the number of parameters is static, it is the quantity of entries inside of each parameter that changes.

Looking through the api reference (https://bitcoin.org/en/developer-refere ... -core-apis) it would seem like you could use the SendToAddress to send an individual transaction, and it looks like it might be simpler to build the command. To do this you would read each line from your file and build the command from that one line, send it (using SendToAddress and then move to the next line etc, etc, etc...
The quantity of parameters may change. Some of them are optional. But to make things simpler, I just use them all to avoid having to deal with checking if mandatory and optional are preset or not.

I understand I might have misused the correct words. Anyway, what I mean is that I need to find a way of dynamically insert the items inside those 2 parameters which are lists of items.
And also, I need to figure out out to distinguish parameters from wrapping brackets, braces, colons and etc.

What about "eval" command? Would it be useful here?

I don't have problems with "sendtoaddress". That one is working nice. The thing with using "sendmany" is that it allows to pay way less in fees for each transaction. So, I have all the interest in using this command over the other. The other is only to be used in exceptions.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#18 Post by Dai_trying »

PsySc0rpi0n wrote:I need to figure out out to distinguish parameters from wrapping brackets, braces, colons and etc.
I can see how it could be difficult to fully comprehend these things and I cannot say I do myself really, but this is one of the reasons I said I would use Python for this kind of script as you can easily create json "objects" which take care of formatting for you, I will take a look and see if I can put something together when I have more time but am a little limited at the moment.
PsySc0rpi0n wrote:What about "eval" command? Would it be useful here?
I really try to avoid using eval, I have used it in the past but only because I was pushed at the time, and eventually i found another way but even then I'm not sure it would work.
PsySc0rpi0n wrote:I don't have problems with "sendtoaddress". That one is working nice. The thing with using "sendmany" is that it allows to pay way less in fees for each transaction. So, I have all the interest in using this command over the other. The other is only to be used in exceptions.
I can see why it is better for you to do it this way now and will keep that in mind.


I should be able to look into doing a python script over the next day or two and will post something when I have it.

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#19 Post by PsySc0rpi0n »

Dai_trying wrote:
PsySc0rpi0n wrote:I need to figure out out to distinguish parameters from wrapping brackets, braces, colons and etc.
I can see how it could be difficult to fully comprehend these things and I cannot say I do myself really, but this is one of the reasons I said I would use Python for this kind of script as you can easily create json "objects" which take care of formatting for you, I will take a look and see if I can put something together when I have more time but am a little limited at the moment.
PsySc0rpi0n wrote:What about "eval" command? Would it be useful here?
I really try to avoid using eval, I have used it in the past but only because I was pushed at the time, and eventually i found another way but even then I'm not sure it would work.
PsySc0rpi0n wrote:I don't have problems with "sendtoaddress". That one is working nice. The thing with using "sendmany" is that it allows to pay way less in fees for each transaction. So, I have all the interest in using this command over the other. The other is only to be used in exceptions.
I can see why it is better for you to do it this way now and will keep that in mind.


I should be able to look into doing a python script over the next day or two and will post something when I have it.

Ok, thank you a lot.

I'm looking to 'jq' command which allows one to filter data from a JSON object. Despite the fact that what I want is the other way around, I don't know if I can pull something out from here. I'll stick to bash script until you get something out of python. Because if I'm "forced" to switch to python, I'll have to re-learn the basics again. I did some basic learning in the past but then I had to quit because time was scarce!

What I'm thinking now is to write a file in JSON format using my already done printf with the complete command (which is seen as a single long string) and then try to format it with 'jq' and maybe be able to build the command from here.

I think the JSON things here in this command are just the 2 parameters that are wrapped in {} and in []. The other parameters I think I can just append them in a later stage right before issuing the complete command from the script!

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#20 Post by Dai_trying »

I had some time this morning to look at this and although this might not work I thought I would let you see what I have so far as it's not too far off.

First I created a quick bash script to act as bitcoin-cli simply to interpret the parameters

Code: Select all

#!/bin/bash

echo "Param 1 = $1"
echo "Param 2 = $2"
echo "Param 3 = $3"
echo "Param 4 = $4"
echo "Param 5 = $5"
echo "Param 6 = $6"
echo "Param 7 = $7"
I included 1 more parameter than you would be using in case there was some extra info being sent.

Next the file with codes

Code: Select all

1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX
1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz
I named it BitcoinList.txt and you would need to edit the script to reflect the actual file you use.

Then the Python script:

Code: Select all

import json
import subprocess

a = 0
d = {}
e = []

with open('BitcoinList.txt') as input_file:
    for i in input_file:
        a += 1
        b = a/100
        d[i.strip()] = '%.2f' % b
        e.append(i.strip())

r = json.dumps(d)
s = json.dumps(e)

subprocess.check_call(['/path/to/bin/bitcoin-cli', 'sendmany', '""', r, '1', '"Description comment"', s])
which gives me the result :

Code: Select all

Param 1 = sendmany
Param 2 = ""
Param 3 = {"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX": "0.01", "1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz": "0.02"}
Param 4 = 1
Param 5 = "Description comment"
Param 6 = ["1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX", "1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz"]
Param 7 = 
What is left to do is adding the escapes (\) and maybe some extra quotes but I'm out of time for today so thought I would get this out in case you can tweak it, it is quite a simple script so should not be too difficult to do.

HTH

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#21 Post by PsySc0rpi0n »

Dai_trying wrote:I had some time this morning to look at this and although this might not work I thought I would let you see what I have so far as it's not too far off.

First I created a quick bash script to act as bitcoin-cli simply to interpret the parameters

Code: Select all

#!/bin/bash

echo "Param 1 = $1"
echo "Param 2 = $2"
echo "Param 3 = $3"
echo "Param 4 = $4"
echo "Param 5 = $5"
echo "Param 6 = $6"
echo "Param 7 = $7"
I included 1 more parameter than you would be using in case there was some extra info being sent.

Next the file with codes

Code: Select all

1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX
1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz
I named it BitcoinList.txt and you would need to edit the script to reflect the actual file you use.

Then the Python script:

Code: Select all

import json
import subprocess

a = 0
d = {}
e = []

with open('BitcoinList.txt') as input_file:
    for i in input_file:
        a += 1
        b = a/100
        d[i.strip()] = '%.2f' % b
        e.append(i.strip())

r = json.dumps(d)
s = json.dumps(e)

subprocess.check_call(['/path/to/bin/bitcoin-cli', 'sendmany', '""', r, '1', '"Description comment"', s])
which gives me the result :

Code: Select all

Param 1 = sendmany
Param 2 = ""
Param 3 = {"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX": "0.01", "1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz": "0.02"}
Param 4 = 1
Param 5 = "Description comment"
Param 6 = ["1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX", "1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz"]
Param 7 = 
What is left to do is adding the escapes (\) and maybe some extra quotes but I'm out of time for today so thought I would get this out in case you can tweak it, it is quite a simple script so should not be too difficult to do.

HTH
Hello...

Many thanks for the effort. It looks like very much to what I need. I'm going to try another approach in my bash script to see what comes out.

I'm thinking about to do what I'm already doing about building the command, but instead of trying to issue it, I'll right the output to a file already formatted and then will try to load only the JSON objects I need to send to the command already formatted!

If even like this it doesn't work, I'll consider to migrate the rest of the code and this task to python.

Thanks
Psy

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#22 Post by Dai_trying »

Had a bit of time tonight and came up with this, it seems to work, that is it gives the right characters as parameters to bitcoin-cli, I know you don't want a python script but look how short and relatively simple it is.

Code: Select all

import json
import subprocess
import re

a = 0
d = {}
e = []

with open('BitcoinList.txt') as input_file:
    for i in input_file:
        a += 1
        b = a/100
        d[i.strip()] = b
        e.append(i.strip())

r = json.dumps(d)
s = json.dumps(e)

escapedr = str(r).translate(str.maketrans({'"':  r'\"'}))
escapeds = str(s).translate(str.maketrans({'"':  r'\"'}))

subprocess.check_call(['/path/to/bin/bitcoin-cli', 'sendmany', '""', '"' + escapedr + '"', '1', '""', '"' + escapeds + '"'])
The output it gives me is this

Code: Select all

Param 1 = sendmany
Param 2 = ""
Param 3 = "{\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX\": 0.01, \"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz\": 0.02}"
Param 4 = 1
Param 5 = ""
Param 6 = "[\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX\", \"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz\"]"
and although there are a few spaces in there where you had none, I see no reason for this to fail, let me know if it works for you.

NOTE: Don't forget to change the path to bitcoin-cli (Line 22) and the filename to be used (Line 9)

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#23 Post by PsySc0rpi0n »

It looks perfect too.
I got mine working but I'll keep your solution in mind because python is also a possibility in the future!
I'm thinking about investing some money in some blockchain online courses I see around (MIT and others) where python is a common presence.

Thanks a lot for the help.

Psy

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#24 Post by Dai_trying »

You're welcome, I find python a great tool for many scripting jobs where bash can be a bit limited and so tend to lean towards it for anything other than basic scripts.

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#25 Post by PsySc0rpi0n »

Hello.

I'm back to my script.

It's already working but I need to add a new fucntion that will replace some of the code already present in the script.

So what I need to do, is to extract a values from a field of a JSON object returned by a specific command.

To make it more clear, the proccess is:

I run the following command:

Code: Select all

bitcoin-cli listunspent
This command is supposed to return an array of JSON objects, like:

Code: Select all

[
    {
     "field 1": val1,
     "field 2": val2,
     "field 3": val3,
     "field 4": val4,
     "field 5": val5
    },
    {
     "field 1":  val6,
     "field 2":  val7,
     "field 3":  val8,
     "field 4":  val9,
     "field 5": val10
    }
]
And I need to add (sum), let's say, all values of 'field4' fields.

I'm trying to use the 'jq' tool to achieve this but I'm not being able to make the liasom between the output of 'jq' with the bash code to save 'j' output.

I can get the correct output in stdout with the following command:

Code: Select all

bitcoin-cli listunspent | jq '.[].field4'
But what I need, is to save this output, which are 2 lines in this case, into 2 variables, or add them on the fly and save the result in a variable.

How can I possibly save 'jq' result individually in bash variables?

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: command issued from within bash script not working

#26 Post by Dai_trying »

Tbh I definitely wouldn't be using a bash script at this point, but you could look at parsing that json into an array (or arrays, 1 for each set of variables) or maybe into another json object organised how you want it for easy calculation.

but have only used arrays a little in bash scripts and don't recall handling jsons at all, and I find python lists and dicts are so much easier to create/use that I would definitely use them instead.

Maybe a better bash scripter could help more... Anyone?

User avatar
ruwolf
Posts: 618
Joined: 2008-02-18 05:04
Location: Banovce nad Bebravou
Has thanked: 34 times
Been thanked: 26 times

Re: command issued from within bash script not working

#27 Post by ruwolf »

You can read output of program by e.g. enclosing it in grave accent character "`" (ASCII code: 0140, 96, 0x60):

Code: Select all

myvar=`bitcoin-cli listunspent | jq '.[].field4'`
Next, you can separate lines in $myvar e.g. to array by Tutorial Kart: Bash Split String.

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#28 Post by PsySc0rpi0n »

Dai_trying wrote:Tbh I definitely wouldn't be using a bash script at this point, but you could look at parsing that json into an array (or arrays, 1 for each set of variables) or maybe into another json object organised how you want it for easy calculation.

but have only used arrays a little in bash scripts and don't recall handling jsons at all, and I find python lists and dicts are so much easier to create/use that I would definitely use them instead.

Maybe a better bash scripter could help more... Anyone?
I know you are in favour of python. I totally understand, but I want to make this in bash for now.
I'm using python for another purpose though. I'm trying to replicate this script in python but using a 3rd party API (actually I haven't started it yet, but will start soon). :)

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#29 Post by PsySc0rpi0n »

ruwolf wrote:You can read output of program by e.g. enclosing it in grave accent character "`" (ASCII code: 0140, 96, 0x60):

Code: Select all

myvar=`bitcoin-cli listunspent | jq '.[].field4'`
Next, you can separate lines in $myvar e.g. to array by Tutorial Kart: Bash Split String.
Ok, seems to be a great plan. It seems too simple, to be honest... heheh. Going to try.
I need those 2 output later as floating point numbers so that I can add them.

arzgi
Posts: 1183
Joined: 2008-02-21 17:03
Location: Finland
Been thanked: 31 times

Re: command issued from within bash script not working

#30 Post by arzgi »

I have nothing to add to what ruwolf and Dai_trying have said, but you know bash only supports integers?

If you know how long would the floating number would be, you could multiply it to get to an integer number.

Above, and the lack of return value in functions are the reasons why I use bash only for starter scripts and like.

Python in my mind too is better for general programming.

User avatar
PsySc0rpi0n
Posts: 321
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: command issued from within bash script not working

#31 Post by PsySc0rpi0n »

arzgi wrote:I have nothing to add to what ruwolf and Dai_trying have said, but you know bash only supports integers?

If you know how long would the floating number would be, you could multiply it to get to an integer number.

Above, and the lack of return value in functions are the reasons why I use bash only for starter scripts and like.

Python in my mind too is better for general programming.
I didn't know about that of integers and floats. Anyways, I managed to get it working somehow. This script is working with a different method (bitcoin-cli sendmany) and I tried it and it did the correct math andd sent the correct amount of BTC into 2 adddresses.

You can see the script here. But I need to change the sendmany method because that will only work in a specific situation which isn't what I need. So I need to use "listunspent" and use the values returned by this method in the math needed.

I'll check when I get it working. I couldn't dod anything yesterday because I had to fix another issue. My node decided to corrupt blocks so I had to fix that. No node, no script testing. :)

I'll try to work on the script today!
Last edited by PsySc0rpi0n on 2020-06-04 17:40, edited 1 time in total.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: command issued from within bash script not working

#32 Post by Head_on_a_Stick »

FWIW ksh93 does floating point arithmetic:

Code: Select all

$ float x=10 y=4
$ echo $((x/y)) 
2.5
$
https://packages.debian.org/buster/ksh

It's also *much* faster than bash :)
deadbang

User avatar
ruwolf
Posts: 618
Joined: 2008-02-18 05:04
Location: Banovce nad Bebravou
Has thanked: 34 times
Been thanked: 26 times

Re: command issued from within bash script not working

#33 Post by ruwolf »

Standard commands bc (and dc) can do floating point arithmetic (by switch -l, with almost arbitrary precision)...

Post Reply