Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

GMP library usage questions

Programming languages, Coding, Executables, Package Creation, and Scripting.
Message
Author
User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

GMP library usage questions

#1 Post by PsySc0rpi0n »

Hello.

I'm trying to kind of translate a program from Ruby to C, using this GMP library and I'm struggling to understand a few things that are not even directly related to GMP but as I'm using GMP and I may have more questions in the future, I created this thread with this name, to be able to use it in the future.

The thing is that I'm trying to convert a number into it's binary representation using the GMP library function mpz_export() which is supposed to fill in an array of words with data from the given number. Here is the documentation for this function:
GMP library wrote:
Function: void * mpz_export (void *rop, size_t *countp, int order, size_t size, int endian, size_t nails, const mpz_t op)

Fill rop with word data from op.

The parameters specify the format of the data produced. Each word will be size bytes and order can be 1 for most significant word first or -1 for least significant first. Within each word endian can be 1 for most significant byte first, -1 for least significant first, or 0 for the native endianness of the host CPU. The most significant nails bits of each word are unused and set to zero, this can be 0 to produce full words.

The number of words produced is written to *countp, or countp can be NULL to discard the count. rop must have enough space for the data, or if rop is NULL then a result array of the necessary size is allocated using the current GMP allocation function (see Custom Allocation). In either case the return value is the destination used, either rop or the allocated block.

If op is non-zero then the most significant word produced will be non-zero. If op is zero then the count returned will be zero and nothing written to rop. If rop is NULL in this case, no block is allocated, just NULL is returned.

The sign of op is ignored, just the absolute value is exported. An application can use mpz_sgn to get the sign and handle it as desired. (see Comparison Functions)

There are no data alignment restrictions on rop, any address is allowed.

When an application is allocating space itself the required size can be determined with a calculation like the following. Since mpz_sizeinbase always returns at least 1, count here will be at least one, which avoids any portability problems with malloc(0), though if z is zero no space at all is actually needed (or written).

numb = 8*size - nail;
count = (mpz_sizeinbase (z, 2) + numb-1) / numb;
p = malloc (count * size);
I have a few doubts/questions here, about some of the parameters of this function and how things should look like inside the array of words:
1 - A word has always a fixed length or it can be changed?? And here I'm referring to a word in general computing, not specifically in the context of GMP library.
2 - In general computing, isn't a word a stream of 8 bits (1 byte)? Maybe the wording is not the most correct, but you get the point, I hope?
3 - One of the parameters of mpz_export() is size_t size which holds the "word size". If, in general computing, a word has 8 bits (or 1 byte), what effect will this parameter have in the array of words of data from the number to be converted? I ask this because if the size of a word is usually 8 bit and it's a thing of the architecture of the machine the code is running in, why we have a parameter to change it?

After what I said above, I'll show here a piece of code with different values for that size_t size parameter so that someone can explain me te effect of changing the value of this parameter in the resuting array of words, because I cannot understand what I see.


Code 1:
Here I gave the value 8 to the parameter size_t size.
The printf I have in the code prints 0.

Code: Select all

#include <stdio.h>
#include <gmp.h>
#include <inttypes.h>

int main(void){
    uint8_t bin_array[1024];
    uint8_t order = 1, endian = 1;
    size_t countp = 0, size = 8, nails = 0;
    mpz_t op;

    mpz_init(op);
    mpz_set_str(op, "1000", 0);
    countp = mpz_sizeinbase(op, 2);
    printf("countp: %lu\n", countp);
    mpz_export(bin_array, &countp, order, size, endian, nails, op);
    printf("bin_arr[0]: %d\n", bin_array[0]);
    mpz_clear(op);
    return 0;
}
Code 2:
I gave the value of sizeof(unsigned char) to the parameter size_t size.
The printf in the end, prints 3.

Code: Select all

#include <stdio.h>
#include <gmp.h>
#include <inttypes.h>

int main(void){
    uint8_t bin_array[1024];
    uint8_t order = 1, endian = 1;
    size_t countp = 0, size = sizeof(unsigned char), nails = 0;
    mpz_t op;

    mpz_init(op);
    mpz_set_str(op, "1000", 0);
    countp = mpz_sizeinbase(op, 2);
    printf("countp: %lu\n", countp);
    mpz_export(bin_array, &countp, order, size, endian, nails, op);
    printf("bin_arr[0]: %d\n", bin_array[0]);
    mpz_clear(op);
    return 0;
}
And I can't understand the effect of this parameter size_t size in this function.

Aki
Global Moderator
Global Moderator
Posts: 2979
Joined: 2014-07-20 18:12
Location: Europe
Has thanked: 75 times
Been thanked: 407 times

Re: GMP library usage questions

#2 Post by Aki »

Hello,
What is your installed Debian version ?
What is the installed package containing the C library ?
What is the hardware architecture you are compiling for ?
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#3 Post by PsySc0rpi0n »

Aki wrote: 2023-11-01 10:17 Hello,
What is your installed Debian version ?
What is the installed package containing the C library ?
What is the hardware architecture you are compiling for ?
I'm on Debian Bookworm
Using gcc (Debian 12.2.0-14) 12.2.0
Using:
libgmp-dev:amd64 2:6.2.1+dfsg1-1.1 amd64 Multiprecision arithmetic library developers tools
libgmp10:amd64 2:6.2.1+dfsg1-1.1 amd64 Multiprecision arithmetic library
I'm wokring on an Intel laptop Intel(R) Core(TM) i7-3740QM CPU @ 2.70GHz

But I already understood something.
I can already answer some of my questions.
1 & 2- In generic programming context, a word will usually be 1 byte, 8 bits.
3 - I kind of understood the meaning of this parameter in the context of mpz_export() function. In this case we can have something like this, visually speaking, considering size = 1 and size = 8
Image

But when I use this simple code to try to visualize this in practice, I get this:
size = 1

Code: Select all

#include <stdio.h>
#include <gmp.h>
#include <inttypes.h>
#include <stdlib.h>

int main(void){
    uint8_t* bin_array;
    uint8_t order = 1, endian = 0;
    size_t countp = 0, size = 1 , nails = 0;
    size_t numb = 8 * size - nails;
    size_t count = 0;
    mpz_t op;

    mpz_init(op);
    mpz_set_str(op, "1000", 0);
    countp = mpz_sizeinbase(op, 2);
    count = (countp + numb - 1) / numb;
    if((bin_array = (uint8_t*) malloc(count * size)) == NULL)
        return -1;
    mpz_export(bin_array, &countp, order, size, endian, nails, op);
    printf("word size = %zu\n"
            "return of mpz_sizeinbase() =  %zu\n"
            "number of words produced = %zu\n",
            size, count, countp);
    for(uint8_t i = 0; i <= count; i++)
        for(int8_t j = 7; j >= 0; j--)
        printf("bin_arr[%d]: %d\n", i, (bin_array[i] >> j) & 1);
    mpz_clear(op);
    return 0;
}

Code: Select all

 ./test
word size = 1
return of mpz_sizeinbase() =  2
number of words produced = 2
bin_arr[0]: 0
bin_arr[0]: 0
bin_arr[0]: 0
bin_arr[0]: 0
bin_arr[0]: 0
bin_arr[0]: 0
bin_arr[0]: 1
bin_arr[0]: 1
bin_arr[1]: 1
bin_arr[1]: 1
bin_arr[1]: 1
bin_arr[1]: 0
bin_arr[1]: 1
bin_arr[1]: 0
bin_arr[1]: 0
bin_arr[1]: 0
If I use
size = 8

Code: Select all

$ ./test
word size = 8
return of mpz_sizeinbase() =  1
number of words produced = 1
bin_arr[0]: 1
bin_arr[0]: 1
bin_arr[0]: 1
bin_arr[0]: 0
bin_arr[0]: 1
bin_arr[0]: 0
bin_arr[0]: 0
bin_arr[0]: 0
I was not expecting to get an incomplete stream of bits choosing size = 8. I'm not sure where I am going wrong!

Aki
Global Moderator
Global Moderator
Posts: 2979
Joined: 2014-07-20 18:12
Location: Europe
Has thanked: 75 times
Been thanked: 407 times

Re: GMP library usage questions

#4 Post by Aki »

Hello,
PsySc0rpi0n wrote: 2023-11-01 15:03 I was not expecting to get an incomplete stream of bits choosing size = 8. I'm not sure where I am going wrong!
In the code (from the last post, see except below) the outer loop counts the words, but the inner loop doesn't take into account (it is fixed at 7 bits) the size of the word as defined by the size variable; furthermore, the outer loop performs an extra loop because the cond-expression of for statement is <= and not < when the loop starts from zero:

Code: Select all

[..]
    for(uint8_t i = 0; i <= count; i++)
        for(int8_t j = 7; j >= 0; j--)
        printf("bin_arr[%d]: %d\n", i, (bin_array[i] >> j) & 1);
[..]
I report below a little rewrite of the code from the previous post:

Code: Select all

#include <stdio.h>
#include <gmp.h>
#include <inttypes.h>
#include <stdlib.h>

/*
   Manual:
        https://gmplib.org/manual/Integer-Import-and-Export
        https://gmplib.org/manual/Miscellaneous-Integer-Functions
        https://gmplib.org/manual/Function-Index

   packages:
        https://packages.debian.org/bookworm/libgmp10
        https://packages.debian.org/bookworm/libgmp-dev

   to build:
        gcc -o words words.c -l gmp
*/

int main(void){
    uint8_t* bin_array = NULL;
    int order = 1, endian = 1;
    size_t countp = 0, word_size = 1 , nails = 0;
    size_t numb = 8 * word_size - nails;
    size_t count = 0;
    mpz_t op;
    const char dec_value[] = "1000";

    size_t word_count, byte_count, bit_count, bits_per_word = 0;

    mpz_init(op);

    if( mpz_set_str(op, dec_value, 0) < 0 ) {
        printf("error in mpz_set_str()\n");
        exit(EXIT_FAILURE);
    }

    printf("type number of bytes per word: ");
    if( scanf("%zu", &word_size) == 0 )  {
        printf("error in scanf()\n");
        exit(EXIT_FAILURE);
    }

    bits_per_word = (word_size*8);
    printf("bits per word = %zu\n", bits_per_word);

    printf("decimal value to convert to binary = %s\n", dec_value);

    /* note: the number of digits after conversion is indipendent from the word size */
    count = (mpz_sizeinbase(op, 2) + numb - 1) / numb;

    if( (bin_array = (uint8_t*) malloc(count * word_size)) == NULL) {
        printf("error in malloc()\n");
        exit(EXIT_FAILURE);
    }

    mpz_export(bin_array, &countp, order, word_size, endian, nails, op);

    printf("word size in byte = %zu\n"
           "return of mpz_sizeinbase()/numb =  %zu\n"
           "number of words produced in output = %zu\n\n",
           word_size, count, countp);

    printf("bits  in output: ");
    for(byte_count = 0; byte_count < (countp *word_size); byte_count++)
        for(bit_count = 8 ; bit_count > 0; bit_count --)
            printf("%u", (unsigned int) (bin_array[byte_count] >> bit_count-1) & 1 );

    printf("\n");

    printf("words in output: ");
    for(word_count = 0; word_count < countp; word_count++)
        for(bit_count = bits_per_word ; bit_count > 0 ; bit_count --)
            bit_count == bits_per_word  ? printf("%d", 1) : printf(" ");

    printf("\n");

    mpz_clear(op);

    free(bin_array); 

    return 0;
}
This is the output:

Code: Select all

$ ./words
type number of bytes per word: 1
bits per word = 8
decimal value to convert to binary = 1000
word size in byte = 1
return of mpz_sizeinbase()/numb =  2
number of words produced in output = 2

bits  in output: 0000001111101000
words in output: 1       1       

$ ./words
type number of bytes per word: 2
bits per word = 16
decimal value to convert to binary = 1000
word size in byte = 2
return of mpz_sizeinbase()/numb =  2
number of words produced in output = 1

bits  in output: 0000001111101000
words in output: 1               

$ ./words
type number of bytes per word: 3
bits per word = 24
decimal value to convert to binary = 1000
word size in byte = 3
return of mpz_sizeinbase()/numb =  2
number of words produced in output = 1

bits  in output: 000000000000001111101000
words in output: 1                       
Hope this helps. Let me know.
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#5 Post by PsySc0rpi0n »

@Aki it seems to be working here now in an example code I did.
Now, I'm going to try to put it in the real code I'm using!

I'll give feedback in case things goes south! :)

Thank you

Edited;

However, I just noticed a problem I cannot explain.
The documentation of mpz_export() parameter int endian, says that if we use 0, it will use the native endianness of the host cpu. So, if I use zero, I get this only for the case I choose the word size of 8 bytes! For 1 byte it works correctly.

Code: Select all

1110100000000011000000000000000000000000000000000000000000000000
This result is the same as if I use endian = -1.

Edited;
Ok, I think I understood.
There was a code I tried, different from all the last examples and it would print the word of 1 byte but ignoring the leading zeros that are usually ignored when they have no meaning. So I would have in that case:

Code: Select all

1111101000
but with this version of the code, it is printing the complete bytes, with the zeros that have no meaning, like this:

Code: Select all

001111101000
And in the case of the word of 8 bytes, all the zeros are printed.

Code: Select all

111101000 00000011........

Aki
Global Moderator
Global Moderator
Posts: 2979
Joined: 2014-07-20 18:12
Location: Europe
Has thanked: 75 times
Been thanked: 407 times

Re: GMP library usage questions

#6 Post by Aki »

Hello,
Have you got it sorted?
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#7 Post by PsySc0rpi0n »

Aki wrote: 2023-11-03 21:36 Hello,
Have you got it sorted?
Yes, this part I got it.

However, while progressing in the code I'm doing, I found more problems.
I was able to use a different approach to do the binary conversiont and I'm happy with it. But then, I found another problem.
The code I'm trying to write is to mimic a Ruby code I found. This Ruby code has a function to compute the inverse multiplicative of a number in a finite field of values.
The function in Ruby is this:

Code: Select all

def inverse(a, m = $p)
  m_orig = m # store original modulus

  a = a % m if a < 0 # make sure a is positive
  prevy, y = 0, 1
  while a > 1
    q = m / a
    y, prevy = prevy - q * y, y
    a, m = m % a, a
  end
  return y % m_orig
end
The GMP library function mpz_invert() returns correct values for some examples, but not for all.
For instance, for any value that ends in zero, like 10, 20, 100, 1000, 1010, etc, this function is returning always 0. The Ruby function is not returning always 0, so, something is wrong with this mpz_invert() function. If it is expected or not, I'm not sure, but my goal is to have a function that mimics the Ruby one!

Looking in GMP source code, I could find this:

Code: Select all

#include "gmp-impl.h"

int
mpz_invert (mpz_ptr inverse, mpz_srcptr x, mpz_srcptr n)
{
  mpz_t gcd, tmp;
  mp_size_t xsize, nsize, size;
  TMP_DECL;

  xsize = ABSIZ (x);
  nsize = ABSIZ (n);

  size = MAX (xsize, nsize) + 1;
  TMP_MARK;

  MPZ_TMP_INIT (gcd, size);
  MPZ_TMP_INIT (tmp, size);
  mpz_gcdext (gcd, tmp, (mpz_ptr) 0, x, n);

  /* If no inverse existed, return with an indication of that.  */
  if (!MPZ_EQUAL_1_P (gcd))
    {
      TMP_FREE;
      return 0;
    }

  /* Make sure we return a positive inverse.  */
  if (SIZ (tmp) < 0)
    {
      if (SIZ (n) < 0)
	mpz_sub (inverse, tmp, n);
      else
	mpz_add (inverse, tmp, n);
    }
  else
    mpz_set (inverse, tmp);

  TMP_FREE;
  return 1;
}
I'm not sure this is the definition/source code for the mpz_invert() function or not and I could not also make sure if this code mimics the Ruby one. Probably not.

The code I tried to test the values ending in 0 was this:

Code: Select all

#include <stdio.h>
#include <gmp.h>

int main(void){
    mpz_t a, P, res;
    mpz_inits(a, P, res, NULL);
    mpz_set_ui(a, 66);
    // mpz_set_str(P, "fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc2f", 16);
    mpz_set_str(P, "800", 0);
    mpz_invert(res, a, P);
    gmp_printf("The Inverse of %Zd of %Zd is %Zd\n", a, P, res);
    mpz_clears(a, P, res, NULL);
    return 0;
}
Then, I asked some AI engine why would this happen and it told me something like that this function is returning zero because the numbers ending in zero are not relatively prime to the other number in question. However, in the docs of this function, nothing is said about this detail:

Function: int mpz_invert (mpz_t rop, const mpz_t op1, const mpz_t op2)

Compute the inverse of op1 modulo op2 and put the result in rop. If the inverse exists, the return value is non-zero and rop will satisfy 0 <= rop < abs(op2) (with rop = 0 possible only when abs(op2) = 1, i.e., in the somewhat degenerate zero ring). If an inverse doesn’t exist the return value is zero and rop is undefined. The behaviour of this function is undefined when op2 is zero.
https://gmplib.org/manual/Number-Theoretic-Functions

The Ruuby function returns 400 for the fowlloing values: a= 66 and p = 800. GMP library function returns 0.
So, what can I do to mimic the Ruby function?

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#8 Post by PsySc0rpi0n »

I'm back to this and while I was trying to re-write a function named my_inverse(), I am stuck with an weird issue. A local variable is being changed by a, let me call it, a remove variable change.

The scenario is this:
I have function ecdsa_double() that calls my_inverse(). From within ecdsa_double(), I send a parameter to my_inverse() as a value.
However, when I use this value in my_inverse(), somehow it is changing its value inside ecdsa_double(), which makes no sense because I'm sending it as a value, not as a reference!
Check the arrowed comments in the code for better understanding!

Function ecdsa_double():

Code: Select all

void ecdsa_double(gmp_coord point, gmp_coord* pointDouble, mpz_t opa, mpz_t opP){
    mpz_t opPow, opSum, opMult, opInv, opSlope, opSub;
    mpz_t tmp, tmp1, tmp2, tmp3;
    mpz_inits(opPow, opSum, opMult, opInv, opSlope, opSub, NULL);
    mpz_inits(tmp, tmp1, tmp2, tmp3, NULL);

    // Compute the operations needed in the numerator of (3X² + a) / (2Y)
    mpz_pow_ui(opPow, point.x, 2); // X²
    mpz_set_ui(tmp, 3);
    mpz_mul(opMult, opPow, tmp); // 3X²
    mpz_add(opSum, opMult, opa); // 3X² + a
    // Compute the operations needed in the denominator of (3X² + a) / (2Y)
    mpz_set_ui(tmp1, 2);
    mpz_mul(tmp2, point.y, tmp1); // 2Y
    my_inverse(&opInv, tmp2, opP); <--------------------------------- my_inverse() here. Parameter in question being sent is opP
   // <---------------------------When the code returns from my_inverse(), opP parameter has a new value.
   // <---------------------------The one set in my_inverse() at line mpz_set(m, a);
    // Compute the operation needed in the equation (3X² + a) / (2Y)
    mpz_mul(tmp3, opSum, opInv); // (3X² + a) / (2Y)
    // Compute the operation ((3X² + a) / (2Y)) % p
    mpz_mod(opSlope, tmp3, opP);

    // Compute the new X coord of the doubled point
    mpz_inits(opPow, opMult, NULL);
    mpz_pow_ui(opPow, opSlope, 2); // opSlope²   
    mpz_mul(opMult, point.x, tmp1); // 2X
    mpz_sub(opSub, opPow, opMult); // opSlope² - 2x
    mpz_mod(pointDouble->x, opSub, opP); // (opSlope² - 2x) % p
    // gmp_printf("pDouble.x = %Zd\n", pointDouble->x);
        
    // Compute the new Y coord of the doubled point
    mpz_inits(opSub, opMult, tmp, NULL);
    mpz_sub(opSub, point.x, pointDouble->x); // (x - X)
    mpz_mul(opMult, opSlope, opSub); // opSlope(x - X)
    mpz_sub(tmp, opMult, point.y); // opSlope(x - X) - y
    mpz_mod(pointDouble->y, tmp, opP); // (opSlope(x - X) - y) % p

    mpz_clears(opPow, opSum, opMult, opInv, opSlope, opSub, NULL);
    mpz_clears(tmp, tmp1, tmp2, tmp3, NULL);
}
Function my_inverse():

Code: Select all

void my_inverse(mpz_t* res, mpz_t a, mpz_t m) {
    mpz_t tmp_m, q, prevy, y, tmpOp, tmp;
    
    mpz_inits(tmp_m, q, prevy, y, tmpOp, tmp, NULL);
    mpz_set(tmp_m, m);

    if(mpz_cmp_ui(a, 0) < 0)
        mpz_mod(tmp_m, a, tmp_m);

    mpz_set_ui(prevy, 0);
    mpz_set_ui(y, 1);

    while(mpz_cmp_ui(a, 1) > 0) {
        mpz_tdiv_q(q, m, a);            // q = m / a || q = 80 / 7 = 11, 2
        mpz_mul(tmpOp, q, y);           // q * y = 11 * 1, 2 * 1
        mpz_sub(tmpOp, prevy, tmpOp);   // y = prevy - q * y = 0 - 11 * 1 = -11, 1 - 2 * (-11) = 23
        mpz_set(prevy, y);              // prevy = y <=> prevy = 1
        mpz_set(y, tmpOp);              // y = -11
        mpz_mod(tmp, m, a);             // a = m % a = 80 % 7 = 3
        mpz_set(m, a);                  // m = a <=> m = 7 <-------------- The parameter in question is changed LOCALLY, here, but somehow
                                                               ------- when we return, the value in the other function changes to the value it got here
        mpz_set(a, tmp);                // a = 3
    }
    mpz_mod(*res, y, tmp_m);
    mpz_clears(tmp_m, q, prevy, y, tmpOp, tmp, NULL);
}
The complete code, so far is here:
https://gitlab.com/PsySc0rpi0n/bitcoine ... unctions.c

If anyone can explain mw why this is happening, I appreciate it a lot! :)

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#9 Post by PsySc0rpi0n »

I just ackonwledge that somehow this line of code:

Code: Select all

mpz_set_str(P, "115792089237316195423570985008687907853269984665640564039457584007908834671663", 0);
is not the same as this:

Code: Select all

mpz_set_str(P, "fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc2f", 16);
However,

Code: Select all

mpz_set_str(P, "0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc2f", 0);
is already the same as the first one.
Can anyone help me understanding why?

Aki
Global Moderator
Global Moderator
Posts: 2979
Joined: 2014-07-20 18:12
Location: Europe
Has thanked: 75 times
Been thanked: 407 times

Re: GMP library usage questions

#10 Post by Aki »

Hello,
PsySc0rpi0n wrote: 2023-11-22 22:29 I just ackonwledge that somehow this line of code:

Code: Select all

mpz_set_str(P, "115792089237316195423570985008687907853269984665640564039457584007908834671663", 0);
However,

Code: Select all

mpz_set_str(P, "0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc2f", 0);
The decimal integer number:
  • 115792089237316195423570985008687907853269984665640564039457584007908834671663
has the following value expressed in base 16 (hexdecimal):
  • FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F
According to the manual of the function, parameters in hex format must be prefixed with 0x:
Function: int mpz_set_str (mpz_t rop, const char *str, int base)
Set the value of rop from str, a null-terminated C string in base base. White space is allowed in the string, and is simply ignored.

The base may vary from 2 to 62, or if base is 0, then the leading characters are used: 0x and 0X for hexadecimal, 0b and 0B for binary, 0 for octal, or decimal otherwise.

For bases up to 36, case is ignored; upper-case and lower-case letters have the same value. For bases 37 to 62, upper-case letters represent the usual 10..35 while lower-case letters represent 36..61.

This function returns 0 if the entire string is a valid number in base base. Otherwise it returns −1.
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#11 Post by PsySc0rpi0n »

Aki wrote: 2023-11-23 16:35 Hello,
PsySc0rpi0n wrote: 2023-11-22 22:29 I just ackonwledge that somehow this line of code:

Code: Select all

mpz_set_str(P, "115792089237316195423570985008687907853269984665640564039457584007908834671663", 0);
However,

Code: Select all

mpz_set_str(P, "0xfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc2f", 0);
The decimal integer number:
  • 115792089237316195423570985008687907853269984665640564039457584007908834671663
has the following value expressed in base 16 (hexdecimal):
  • FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F
According to the manual of the function, parameters in hex format must be prefixed with 0x:
Function: int mpz_set_str (mpz_t rop, const char *str, int base)
Set the value of rop from str, a null-terminated C string in base base. White space is allowed in the string, and is simply ignored.

The base may vary from 2 to 62, or if base is 0, then the leading characters are used: 0x and 0X for hexadecimal, 0b and 0B for binary, 0 for octal, or decimal otherwise.

For bases up to 36, case is ignored; upper-case and lower-case letters have the same value. For bases 37 to 62, upper-case letters represent the usual 10..35 while lower-case letters represent 36..61.

This function returns 0 if the entire string is a valid number in base base. Otherwise it returns −1.
Yes, but after all, that was not the problem. You can define the base as 16 and not use the "0x" prefix... The problem was that somehow I screw up the conversion.
And seems that nobody that read my last post also noticed it. Later that day, after making my last post, I asked help to a friend of mine and he spotted the issue.
I was using:

Code: Select all

fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc2f
instead of

Code: Select all

fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f
There is an "e" in the last number and somehow I suppressed it in the converted number I was using before!

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#12 Post by PsySc0rpi0n »

Hello once more.

I just wrote a very simple random number generator with some entropy from /dev/urandom using getrandom() function.

Does this small program has a reasonable amount of entropy whatsoever? Or not having the getrandom() call would result in approximatelly the same entropy?

Code: Select all

#include <stdio.h>
#include <gmp.h>
#include <inttypes.h>
#include <sys/random.h>

int main(void) {
    gmp_randstate_t state;
    uint64_t seed_part = 0;
    mpz_t rand, field;
    uint64_t i = 0;

    mpz_inits(rand, field, NULL);
    mpz_set_str(field, "fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f", 16);
    getrandom(&seed_part, sizeof(seed_part), 0);
    printf("Seed: %lu\n", seed_part);
    gmp_randinit_default(state);
    gmp_randseed_ui(state, seed_part);
    for(uint16_t j = 0; j < 1000; j++){
        do{
            mpz_urandomb(rand, state, 256);
            i++;
        }while(mpz_cmp(rand, field) > 0);
        printf("Ran %ld times!\n", i);
        gmp_printf("Random number: 0x%Zx\n", rand);
        i = 0;
    }
    gmp_randclear(state);
    mpz_clears(rand, field, NULL);
    return 0;
}
There is a small part that I used only to try to make sure the do{}while(); loop would eventually run more than once to generate a number smaller than the limit imposed by the variable field, but from all the attempts I did, the loop iterated only once! Not sure if it will ever run more than once though!

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#13 Post by PsySc0rpi0n »

Well, I was able to move on with the previous question but now, I have some other question.

I have written one other small code to generate a tandom number. However, I'm not sure why I'm getting numbers with different numer of digits!
Thisis the code:

Code: Select all

#include <stdio.h>
#include <math.h>
#include <gmp.h>
#include <inttypes.h>
#include <stdlib.h>
#include <string.h>

int main(void) {
    uint8_t bytes = ceil(log2(pow(2, 256) - 1) + 1) / 8;
    // uint8_t count = 0;
    FILE* fpointer = NULL;
    char* data = NULL;
    mpz_t imp_val;

    mpz_init(imp_val);

    if( (data = (char*) malloc(bytes)) == NULL) {
        printf("Memory error!\n");
        exit(-1);
    }

    printf("bytes: %d\n", bytes); 

    if( (fpointer = fopen("/dev/urandom", "rb")) == NULL )
        return -1;
    rewind(fpointer);

    for(uint8_t i = 0; i < bytes; i++){
        fread(&data[i], 1, 1, fpointer);
        printf("%x", data[i]);
    }
    printf("\n");
    printf("Len: %ld\n", strlen(data));
    printf("Sizeof: %lu\n", sizeof(data[0]));
    mpz_import(imp_val, 4, 1, sizeof(data), 0, 0, data);
    gmp_printf("Data: %Zb\n", imp_val);
    mpz_clear(imp_val);
    free(data);
    fclose(fpointer);
    return 0;
}
So, I read, byte by byte from "/dev/urandom" and print it just to make sure digits "are coming". I print them as hex digits and save tem in "data" variable.
Then, I use function "mpz_import()" to convert that array of bytes into a number.
I use a few help lines of code to make sure how many bytes each array index uses and it tells me it uses 8 bytes. So, I tell the funtion to read 4 times 8 bytes! But probably I going wrong either here or when I read data from "/dev/urandom" or when I print data or I'm not sure where I am going wrong.
But for instance, I'm getting output like this:

Code: Select all

$ ./randoma 
bytes: 32
9ffffff947dffffffd4352f102369557dffffff89337affffff8f783dffffffde59ffffffe6ffffff81ffffffab247dffffff99ffffff9111fffffffd53a3e6b
Len: 32
Sizeof: 1
Data: 11111111111110101010110001000111010001101010000
If I put this hex number in an online hex to decimal converter, I don't get a match. Not for decimal, not for binary.
Hex:

Code: Select all

9ffffff947dffffffd4352f102369557dffffff89337affffff8f783dffffffde59ffffffe6ffffff81ffffffab247dffffff99ffffff9111fffffffd53a3e6b
binary:

Code: Select all

10011111111111111111111111111001010001111101111111111111111111111111110101000011010100101111000100000010001101101001010101010111110111111111111111111111111110001001001100110111101011111111111111111111111110001111011110000011110111111111111111111111111111011110010110011111111111111111111111111110011011111111111111111111111110000001111111111111111111111111101010110010010001111101111111111111111111111111100110011111111111111111111111111001000100010001111111111111111111111111111111010101001110100011111001101011
Can anyone tell/help me where am I going wrong?

User avatar
ruwolf
Posts: 643
Joined: 2008-02-18 05:04
Location: Banovce nad Bebravou
Has thanked: 41 times
Been thanked: 30 times

Re: GMP library usage questions

#14 Post by ruwolf »

The first problem is, that your bytes are implicitly converted to int before printing.
You can disable it by explicit conversion to uint8_t (or unsigned char).
You also need to add digits zero in number with length less than maximum ("%02x" formatting):
Try to change your line in printing to:

Code: Select all

    for(uint8_t i = 0; i < bytes; i++){
        fread(&data[i], 1, 1, fpointer);
        printf("%02x", (uint8_t)data[i]);
    }
The second is, that “strlen” function is not able to measure length of such data.
It is tailored for character strings only, which are non-zero bytes ended by null character '\0'.
So, when your data has no zero byte, it ends somewhere after data and potentially by segmentation fault (fatal error).

The third bug is, that conversion ‘b’ is not supported in gmp_printf function:
https://GMPlib.org/manual/Formatted-Output-Strings wrote: The conversions accepted are as follows…
    a A   hex floats, C99 style
    c     character
    d     decimal integer
    e E   scientific format float
    f     fixed point float
    i     same as d
    g G   fixed or scientific float
    m     'strerror' string, GLIBC style
    n     store characters written so far
    o     octal integer
    p     pointer
    s     string
    u     unsigned integer
    x X   hex integer

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#15 Post by PsySc0rpi0n »

I was able to fix the problem in the meantime and I moved on.
I have now a working code and I'm now trying to find a way to solve the next problem.

The current code can be found here, if there is any interest in taking a look at it:
https://gitlab.com/PsySc0rpi0n/bitcoinexp

Now, I need a way of prepending either "02", "03" or "04" to the resulting point I get with the current code.
The current code results in printing the coordinates of a point, and I'm printing them in hexadecimal format!

So, for instance, if the output of my program is:
PrivKey = 0x6af48b6e46838aa874fb6a20143283915b97b28fd5c0a44da4b669ac7c54bff2
PubKey Q[x, y] = privKey * [G.x, G.y]
PubKey ==> Q.x = 0xd970da2247dbc397c915321f8b7e7ab56d7f15b51baa9931d7767657abf039a2
PubKey ==> Q.y = 0xcc2fa9a21349afaa84d57d55a581380380202249e80a6b442499ec1e924f8e98
The new problem is that I need to prepend "02", "03" or "04" to the Q.x coordinate depending on 3 different scenarios, Checking the 3 different scenarios should not be difficult. The problem is prepending any of those 3 possibilities to the Q.x value so that I obtain either (ignoring the 0x for now):
02d970da2247dbc397c915321f8b7e7ab56d7f15b51baa9931d7767657abf039a2
or
03d970da2247dbc397c915321f8b7e7ab56d7f15b51baa9931d7767657abf039a2
or
04d970da2247dbc397c915321f8b7e7ab56d7f15b51baa9931d7767657abf039a2cc2fa9a21349afaa84d57d55a581380380202249e80a6b442499ec1e924f8e98
Yes the last case, the "04", I also need to append the Q.y to Q.x.

I've been trying to use
mpz_export()
but yet without success!

User avatar
ruwolf
Posts: 643
Joined: 2008-02-18 05:04
Location: Banovce nad Bebravou
Has thanked: 41 times
Been thanked: 30 times

Re: GMP library usage questions

#16 Post by ruwolf »

I hope that I have helped you at least a bit by my last answer. :-)

Why with zero? Zero in the beginning of integer has zero significance, it is irrelevant.

Adding any (unsigned long) prefix in front of positive (mpz_t) number in its hexadecimal notation is quite easy:

Code: Select all

size_t sizein16 = mpz_sizeinbase(number, 16);
mp_bitcnt_t exp2 = (mp_bitcnt_t) sizein16 << 2;  // 16 = 2**4, 4 = 2**2
mpz_t prefix_z;
mpz_init_set_ui(prefix_z, prefix);
mpz_mul_2exp(prefix_z, prefix_z, exp2);
mpz_add(number, number, prefix_z);
mpz_clear(prefix_z);

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#17 Post by PsySc0rpi0n »

ruwolf wrote: 2024-01-05 09:34 I hope that I have helped you at least a bit by my last answer. :-)

Why with zero? Zero in the beginning of integer has zero significance, it is irrelevant.

Adding any (unsigned long) prefix in front of positive (mpz_t) number in its hexadecimal notation is quite easy:

Code: Select all

size_t sizein16 = mpz_sizeinbase(number, 16);
mp_bitcnt_t exp2 = (mp_bitcnt_t) sizein16 << 2;  // 16 = 2**4, 4 = 2**2
mpz_t prefix_z;
mpz_init_set_ui(prefix_z, prefix);
mpz_mul_2exp(prefix_z, prefix_z, exp2);
mpz_add(number, number, prefix_z);
mpz_clear(prefix_z);
Could you please elaborate your lines of code? I'm excited that it seems to be easy but I'm not sure I'm following! :)

User avatar
ruwolf
Posts: 643
Joined: 2008-02-18 05:04
Location: Banovce nad Bebravou
Has thanked: 41 times
Been thanked: 30 times

Re: GMP library usage questions

#18 Post by ruwolf »

Which line do you not understand?

1st line computes length of number in hexadecimal digits.

2nd is simple multiplication by 4, it converts hexadecimal exponent to binary one, (because there is function mul_2exp, but not mul_16exp).

3rd & 4th only convert prefix from unsigned long to mpz_t.

5ft line makes shift by multiplication.
It is analogous to shifting number in decimal.
If you want to shift a number in decimal notation by E digits, you multiply it by the number 10**E. (** is a power, exponentiation sign).
In hexadecimal notation, it is the same, but you multiply it by the number 16**E.
And 16**E = (2**4)**E = 2**(4*E)

6th line adds the shifted prefix to the original number.

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#19 Post by PsySc0rpi0n »

ruwolf wrote: 2024-01-06 16:41 Which line do you not understand?

1st line computes length of number in hexadecimal digits.

2nd is simple multiplication by 4, it converts hexadecimal exponent to binary one, (because there is function mul_2exp, but not mul_16exp).

3rd & 4th only convert prefix from unsigned long to mpz_t.

5ft line makes shift by multiplication.
It is analogous to shifting number in decimal.
If you want to shift a number in decimal notation by E digits, you multiply it by the number 10**E. (** is a power, exponentiation sign).
In hexadecimal notation, it is the same, but you multiply it by the number 16**E.
And 16**E = (2**4)**E = 2**(4*E)

6th line adds the shifted prefix to the original number.

Oh ok. This is was what I needed. The conceptual explanation because I am not aware of this math to do this stuff. I only know basic stuff like left shift is the same as multiplication, right shift is the same as division and etc!

Ok, what about to add a zero before the prepended value? I mean, instead of having
0x4....
I need to have
0x04...
. I know that in terms of math, it is irrelevant, but this is only for showing / printing purposes!

User avatar
PsySc0rpi0n
Posts: 322
Joined: 2012-10-24 13:54
Location: Bitcoin World
Has thanked: 8 times
Been thanked: 1 time

Re: GMP library usage questions

#20 Post by PsySc0rpi0n »

ok, even before that (whhat I asked in my previous post), I am facing another problem while compiling. I tried to create a source code file only for this purpose and I'm getting an error I can't figure out.

So, in ./inc/comp_pubkey.h I have the function prototype like this:

Code: Select all

#include <gmp.h>
#include "../inc/ecdsa_arithmetic.h"

int compress_pubkey(gmp_coord raw_pbk, mpz_t* compressed_pbk);
I include ecdsa_arithmetic.h because I defined here a custom data type named gmp_coord, like this:

Code: Select all

#include <gmp.h>

typedef struct{
    mpz_t x;
    mpz_t y;
}gmp_coord;

...
Then, my function that I wrote, based on your help, on post #532 is this:

Code: Select all

#include <stdio.h>
#include <gmp.h>

#include "../inc/comp_pubkey.h"

int compress_pubkey(gmp_coord raw_pbk, mpz_t* compressed_pbk) {
    size_t sizein16 = mpz_sizeinbase(raw_pbk.x, 16);
    mp_bitcnt_t exp2 = (mp_bitcnt_t) sizein16 << 2;
    mpz_t prefix_z;


    // check if Y coordinate of raw pub key is odd or even
    if(mpz_tstbit(raw_pbk.y, 0))
        // prefix 03 for odd coord
        mpz_init_set_ui(prefix_z, 3);
    else
        // prefix 02 for even coord
        mpz_init_set_ui(prefix_z, 2);

    mpz_mul_2exp(prefix_z, prefix_z, exp2);
    mpz_add(*compressed_pbk, raw_pbk.x, prefix_z);

    mpz_clears(prefix_z, exp2, NULL);
    return 0;
}
In main function I include also

Code: Select all

#include "../inc/comp_pubkey.h"
and ecdsa_arithemtic.h
But when compiling I get an error telling me that there are duplicate definitions for my custom data type, if I understand the compiler output correctly.
$ make
Compiling main.c
cc -Wall -Werror -Wextra -pedantic -std=c99 -lm -g3 -c -o obj/main.o src/main.c
In file included from src/../inc/comp_pubkey.h:2,
from src/main.c:7:
src/../inc/../inc/ecdsa_arithmetic.h:6:2: error: conflicting types for ‘gmp_coord’; have ‘struct <anonymous>’
6 | }gmp_coord;
| ^~~~~~~~~
In file included from src/main.c:5:
src/../inc/ecdsa_arithmetic.h:6:2: note: previous declaration of ‘gmp_coord’ with type ‘gmp_coord’
6 | }gmp_coord;
| ^~~~~~~~~
src/../inc/../inc/ecdsa_arithmetic.h:9:6: error: conflicting types for ‘ecdsa_double’; have ‘void(gmp_coord, gmp_coord *, __mpz_struct *, __mpz_struct *)’
9 | void ecdsa_double(gmp_coord point, gmp_coord* pointDouble, mpz_t opa, mpz_t opP);
| ^~~~~~~~~~~~
src/../inc/ecdsa_arithmetic.h:9:6: note: previous declaration of ‘ecdsa_double’ with type ‘void(gmp_coord, gmp_coord *, __mpz_struct *, __mpz_struct *)’
9 | void ecdsa_double(gmp_coord point, gmp_coord* pointDouble, mpz_t opa, mpz_t opP);
| ^~~~~~~~~~~~
src/../inc/../inc/ecdsa_arithmetic.h:10:6: error: conflicting types for ‘ecdsa_add’; have ‘void(gmp_coord, gmp_coord, gmp_coord *, __mpz_struct *, __mpz_struct *)’
10 | void ecdsa_add(gmp_coord pointA, gmp_coord pointB, gmp_coord* pointAdd, mpz_t opA, mpz_t opP);
| ^~~~~~~~~
src/../inc/ecdsa_arithmetic.h:10:6: note: previous declaration of ‘ecdsa_add’ with type ‘void(gmp_coord, gmp_coord, gmp_coord *, __mpz_struct *, __mpz_struct *)’
10 | void ecdsa_add(gmp_coord pointA, gmp_coord pointB, gmp_coord* pointAdd, mpz_t opA, mpz_t opP);
| ^~~~~~~~~
src/../inc/../inc/ecdsa_arithmetic.h:11:6: error: conflicting types for ‘ecdsa_multiply’; have ‘void(gmp_coord, gmp_coord *, __mpz_struct *, __mpz_struct *, __mpz_struct *)’
11 | void ecdsa_multiply(gmp_coord point, gmp_coord* newPoint, mpz_t pKey, mpz_t opA, mpz_t opP);
| ^~~~~~~~~~~~~~
src/../inc/ecdsa_arithmetic.h:11:6: note: previous declaration of ‘ecdsa_multiply’ with type ‘void(gmp_coord, gmp_coord *, __mpz_struct *, __mpz_struct *, __mpz_struct *)’
11 | void ecdsa_multiply(gmp_coord point, gmp_coord* newPoint, mpz_t pKey, mpz_t opA, mpz_t opP);
| ^~~~~~~~~~~~~~
make: *** [Makefile:30: obj/main.o] Error 1
Some help would be appreciated, please! Can't get around this, so far!

The code is here, to be easier to track the problem:
https://gitlab.com/PsySc0rpi0n/bitcoinexp/-/tree/master

Post Reply