Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Dynamically linking lzma and misconceptions I have

Programming languages, Coding, Executables, Package Creation, and Scripting.
Post Reply
Message
Author
MultiplexLayout
Posts: 56
Joined: 2020-09-23 19:21
Has thanked: 7 times

Dynamically linking lzma and misconceptions I have

#1 Post by MultiplexLayout »

I am trying to convert a cmake project from static linking to dynamic linking and I am having some problems with liblzma. The main source expects the target lzma, instead of LibLZMA::LibLZMA, so my cmake code looks like this:

Code: Select all

if(USE_SYSTEM_LIBS)
	find_package(LibLZMA)
	add_library(lzma ALIAS LibLZMA::LibLZMA)
	# lzma seems to expect e.g. "#include <lzma/LzmaEnc.h>" but src uses "#include <LzmaEnc.h>" so we need to set the subdirectory explicitly
	find_path(lzma_HEADER_DIR PATH_SUFFIXES lzma NAMES lzma12.h)
	target_include_directories(LibLZMA::LibLZMA INTERFACE ${lzma_HEADER_DIR})
endif()
All seems well and it compiles but during linking I get:

Code: Select all

/usr/bin/ld: ../../dep/libchdr/liblibchdr.a(libchdr_chd.c.o): in function `cdlz_codec_init':
libchdr_chd.c:(.text+0x2bf3): undefined reference to `LzmaEncProps_Init'
/usr/bin/ld: libchdr_chd.c:(.text+0x2c07): undefined reference to `LzmaEncProps_Normalize'
/usr/bin/ld: libchdr_chd.c:(.text+0x2c4a): undefined reference to `LzmaEnc_Create'
/usr/bin/ld: libchdr_chd.c:(.text+0x2c5f): undefined reference to `LzmaEnc_SetProps'
/usr/bin/ld: libchdr_chd.c:(.text+0x2c73): undefined reference to `LzmaEnc_Destroy'
/usr/bin/ld: libchdr_chd.c:(.text+0x2c97): undefined reference to `LzmaEnc_WriteProperties'
/usr/bin/ld: libchdr_chd.c:(.text+0x2ca8): undefined reference to `LzmaEnc_Destroy'
/usr/bin/ld: libchdr_chd.c:(.text+0x2cc2): undefined reference to `LzmaDec_Allocate'
/usr/bin/ld: ../../dep/libchdr/liblibchdr.a(libchdr_chd.c.o): in function `cdlz_codec_free':
libchdr_chd.c:(.text+0x2d9e): undefined reference to `LzmaDec_Free'
/usr/bin/ld: ../../dep/libchdr/liblibchdr.a(libchdr_chd.c.o): in function `cdlz_codec_decompress':
libchdr_chd.c:(.text+0x2e9d): undefined reference to `LzmaDec_Init'
/usr/bin/ld: libchdr_chd.c:(.text+0x2ed2): undefined reference to `LzmaDec_DecodeToBuf'
Using "grep -r 'LzmaEncProps_Init' /usr/include/lzma gives:

Code: Select all

/usr/include/lzma/LzmaEnc.h:void LzmaEncProps_Init(CLzmaEncProps *p);
/usr/include/lzma/LzmaEnc.c:void LzmaEncProps_Init(CLzmaEncProps *p)
/usr/include/lzma/LzmaEnc.c:    LzmaEncProps_Init(&props);
Using "objdump -tT /usr/lib/x86_64-linux-gnu/liblzma.so | grep "LzmaEncProps_Init" gives no results.

This is where my confusion is. Am I linking to the wrong library? Why are there .c files in a /usr/include directory? I thought only header files went in there? Am I supposed to link the .c files statically? Whats the .so file for in that case?

I'm sorry if these questions sound dumb. I just don't get it

LE_746F6D617A7A69
Posts: 932
Joined: 2020-05-03 14:16
Has thanked: 7 times
Been thanked: 68 times

Re: Dynamically linking lzma and misconceptions I have

#2 Post by LE_746F6D617A7A69 »

MultiplexLayout wrote: 2022-09-14 16:21 Using "objdump -tT /usr/lib/x86_64-linux-gnu/liblzma.so | grep "LzmaEncProps_Init" gives no results.
Remove the pipeline to 'grep' and see what will happen ;)
... i.e. the Debian implementation of LZMA is based on XZ, which is better and faster (but, zstd is even better ;) )

Regards.
Bill Gates: "(...) In my case, I went to the garbage cans at the Computer Science Center and I fished out listings of their operating system."
The_full_story and Nothing_have_changed

MultiplexLayout
Posts: 56
Joined: 2020-09-23 19:21
Has thanked: 7 times

Re: Dynamically linking lzma and misconceptions I have

#3 Post by MultiplexLayout »

LE_746F6D617A7A69 wrote: 2022-09-14 21:01
MultiplexLayout wrote: 2022-09-14 16:21 Using "objdump -tT /usr/lib/x86_64-linux-gnu/liblzma.so | grep "LzmaEncProps_Init" gives no results.
Remove the pipeline to 'grep' and see what will happen ;)
... i.e. the Debian implementation of LZMA is based on XZ, which is better and faster (but, zstd is even better ;) )

Regards.
Without grep there is no sign of the symbols that ld says is missing. FWIW here's the output:

Code: Select all

/usr/lib/x86_64-linux-gnu/liblzma.so:     file format elf64-x86-64

SYMBOL TABLE:
no symbols


DYNAMIC SYMBOL TABLE:
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) free
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) pthread_create
0000000000000000  w   D  *UND*	0000000000000000  Base        _ITM_deregisterTMCloneTable
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.17) clock_gettime
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.3.2) pthread_cond_wait
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.6)  __sched_cpucount
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.4)  __stack_chk_fail
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) pthread_mutex_destroy
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) memset
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) pthread_condattr_destroy
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) memcmp
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.3.2) pthread_cond_signal
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) calloc
0000000000000000  w   D  *UND*	0000000000000000  Base        __gmon_start__
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) sigfillset
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.14) memcpy
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.3.2) pthread_cond_init
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) pthread_mutex_unlock
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) malloc
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.3.3) pthread_condattr_setclock
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.32) pthread_sigmask
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) pthread_condattr_init
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) memmove
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.3.2) pthread_cond_timedwait
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.3.2) pthread_cond_destroy
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) sysconf
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) pthread_join
0000000000000000  w   D  *UND*	0000000000000000  Base        _ITM_registerTMCloneTable
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.3.4) sched_getaffinity
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) pthread_mutex_init
0000000000000000  w   DF *UND*	0000000000000000 (GLIBC_2.2.5) __cxa_finalize
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) pthread_mutex_lock
0000000000003710 g    DF .text	0000000000000008  XZ_5.0      lzma_version_string
0000000000007bf0 g    DF .text	0000000000000062  XZ_5.0      lzma_index_encoder
000000000000b520 g    DF .text	000000000000005e  XZ_5.0      lzma_easy_decoder_memusage
000000000000c620 g    DF .text	0000000000000225  XZ_5.0      lzma_stream_buffer_decode
00000000000073e0 g    DF .text	000000000000003e  XZ_5.0      lzma_filter_encoder_is_supported
000000000000d070 g    DF .text	0000000000000081  XZ_5.0      lzma_stream_footer_decode
000000000000c290 g    DF .text	000000000000038b  XZ_5.0      lzma_index_hash_decode
0000000000000000 g    DO *ABS*	0000000000000000  XZ_5.0      XZ_5.0
0000000000003ed0 g    DF .text	000000000000006c  XZ_5.0      lzma_memlimit_get
0000000000003e10 g    DF .text	0000000000000027  XZ_5.2      lzma_get_progress
0000000000000000 g    DO *ABS*	0000000000000000  XZ_5.2      XZ_5.2
000000000000d2b0 g    DF .text	0000000000000018  XZ_5.0      lzma_check_size
0000000000005470 g    DF .text	00000000000002b0  XZ_5.0      lzma_index_append
000000000000c0f0 g    DF .text	00000000000000ca  XZ_5.0      lzma_index_hash_init
0000000000005280 g    DF .text	0000000000000005  XZ_5.0      lzma_index_total_size
000000000000a8a0 g    DF .text	0000000000000062  XZ_5.0      lzma_alone_decoder
00000000000051d0 g    DF .text	0000000000000059  XZ_5.0      lzma_index_memusage
0000000000004040 g    DF .text	0000000000000055  XZ_5.0      lzma_block_compressed_size
00000000000077e0 g    DF .text	00000000000000cc  XZ_5.0      lzma_filter_flags_encode
000000000000cfd0 g    DF .text	0000000000000094  XZ_5.0      lzma_stream_header_decode
0000000000005390 g    DF .text	000000000000006e  XZ_5.0      lzma_index_stream_flags
0000000000005230 g    DF .text	000000000000000c  XZ_5.0      lzma_index_memused
0000000000004e60 g    DF .text	0000000000000341  XZ_5.0      lzma_index_end
000000000000b250 g    DF .text	0000000000000062  XZ_5.0      lzma_block_decoder
000000000000d100 g    DF .text	0000000000000181  XZ_5.0      lzma_vli_decode
0000000000007240 g    DF .text	0000000000000147  XZ_5.0      lzma_raw_buffer_encode
0000000000004d30 g    DF .text	0000000000000123  XZ_5.0      lzma_index_init
0000000000005c20 g    DF .text	000000000000002d  XZ_5.0      lzma_index_iter_rewind
000000000000b890 g    DF .text	000000000000000f  XZ_5.0      lzma_raw_decoder_memusage
000000000000be60 g    DF .text	000000000000006a  XZ_5.0      lzma_index_decoder
00000000000109e0 g    DF .text	0000000000000120  XZ_5.0      lzma_lzma_preset
0000000000005f90 g    DF .text	0000000000000096  XZ_5.0      lzma_stream_flags_compare
0000000000006050 g    DF .text	0000000000000005  XZ_5.2      lzma_cputhreads
0000000000007d80 g    DF .text	0000000000000031  XZ_5.0      lzma_stream_buffer_bound
000000000000cf60 g    DF .text	000000000000006a  XZ_5.0      lzma_stream_decoder
00000000000074c0 g    DF .text	000000000000000f  XZ_5.0      lzma_raw_encoder_memusage
000000000000b820 g    DF .text	000000000000006a  XZ_5.0      lzma_raw_decoder
0000000000007170 g    DF .text	0000000000000070  XZ_5.0      lzma_easy_encoder
000000000000b8a0 g    DF .text	000000000000007c  XZ_5.0      lzma_properties_decode
00000000000052c0 g    DF .text	000000000000007e  XZ_5.0      lzma_index_file_size
000000000000ac60 g    DF .text	000000000000019d  XZ_5.0      lzma_block_buffer_decode
0000000000012200 g    DF .text	000000000000000a  XZ_5.0      lzma_mode_is_supported
000000000000c1f0 g    DF .text	000000000000009e  XZ_5.0      lzma_index_hash_append
0000000000005340 g    DF .text	0000000000000005  XZ_5.0      lzma_index_uncompressed_size
0000000000005240 g    DF .text	0000000000000005  XZ_5.0      lzma_index_block_count
00000000000041b0 g    DF .text	0000000000000189  XZ_5.0      lzma_filters_copy
0000000000005c60 g    DF .text	0000000000000222  XZ_5.0      lzma_index_iter_next
000000000000b7d0 g    DF .text	000000000000003e  XZ_5.0      lzma_filter_decoder_is_supported
000000000000c1d0 g    DF .text	000000000000001f  XZ_5.0      lzma_index_hash_size
0000000000006960 g    DF .text	0000000000000037  XZ_5.0      lzma_block_buffer_bound
0000000000006f10 g    DF .text	00000000000001ab  XZ_5.0      lzma_block_header_encode
0000000000007c60 g    DF .text	0000000000000114  XZ_5.0      lzma_index_buffer_encode
0000000000005400 g    DF .text	0000000000000066  XZ_5.0      lzma_index_stream_padding
00000000000040a0 g    DF .text	000000000000001e  XZ_5.0      lzma_block_total_size
00000000000069c0 g    DF .text	0000000000000020  XZ_5.2      lzma_block_uncomp_encode
00000000000062f0 g    DF .text	0000000000000062  XZ_5.0      lzma_alone_encoder
0000000000005950 g    DF .text	00000000000002ca  XZ_5.0      lzma_index_dup
0000000000005720 g    DF .text	0000000000000222  XZ_5.0      lzma_index_cat
000000000000d3b0 g    DF .text	0000000000000114  XZ_5.0      lzma_crc32
0000000000007690 g    DF .text	0000000000000078  XZ_5.0      lzma_properties_size
000000000000d4d0 g    DF .text	00000000000000ee  XZ_5.0      lzma_crc64
0000000000007dc0 g    DF .text	000000000000029d  XZ_5.0      lzma_stream_buffer_encode
000000000000c1c0 g    DF .text	0000000000000005  XZ_5.0      lzma_index_hash_end
0000000000006db0 g    DF .text	0000000000000062  XZ_5.0      lzma_block_encoder
0000000000005260 g    DF .text	000000000000001f  XZ_5.0      lzma_index_size
00000000000087d0 g    DF .text	00000000000000a1  XZ_5.0      lzma_stream_footer_encode
00000000000074d0 g    DF .text	0000000000000109  XZ_5.0      lzma_filters_update
0000000000005250 g    DF .text	0000000000000004  XZ_5.0      lzma_index_stream_count
0000000000006030 g    DF .text	000000000000001a  XZ_5.0      lzma_vli_size
000000000000b920 g    DF .text	00000000000000bc  XZ_5.0      lzma_filter_flags_decode
000000000000d290 g    DF .text	0000000000000015  XZ_5.0      lzma_check_is_supported
00000000000086c0 g    DF .text	0000000000000072  XZ_5.0      lzma_stream_encoder
00000000000071e0 g    DF .text	000000000000005e  XZ_5.0      lzma_easy_encoder_memusage
0000000000007440 g    DF .text	0000000000000072  XZ_5.0      lzma_raw_encoder
000000000000b580 g    DF .text	00000000000001f1  XZ_5.0      lzma_raw_buffer_decode
0000000000003a80 g    DF .text	00000000000002b0  XZ_5.0      lzma_code
0000000000003700 g    DF .text	0000000000000006  XZ_5.0      lzma_version_number
000000000000a340 g    DF .text	00000000000000ed  XZ_5.2      lzma_stream_encoder_mt_memusage
0000000000005e90 g    DF .text	00000000000000f6  XZ_5.0      lzma_index_iter_locate
000000000000a2d0 g    DF .text	000000000000006a  XZ_5.2      lzma_stream_encoder_mt
0000000000008740 g    DF .text	0000000000000081  XZ_5.0      lzma_stream_header_encode
000000000000abf0 g    DF .text	000000000000006a  XZ_5.0      lzma_auto_decoder
00000000000045d0 g    DF .text	0000000000000005  XZ_5.0      lzma_physmem
0000000000003f40 g    DF .text	000000000000006c  XZ_5.0      lzma_memlimit_set
0000000000003d30 g    DF .text	00000000000000d2  XZ_5.0      lzma_end
0000000000008880 g    DF .text	0000000000000105  XZ_5.0      lzma_vli_encode
0000000000007780 g    DF .text	000000000000005d  XZ_5.0      lzma_filter_flags_size
0000000000006e20 g    DF .text	00000000000000f0  XZ_5.0      lzma_block_header_size
00000000000070c0 g    DF .text	00000000000000a4  XZ_5.0      lzma_easy_buffer_encode
0000000000005350 g    DF .text	000000000000001a  XZ_5.0      lzma_index_checks
0000000000003fb0 g    DF .text	0000000000000089  XZ_5.0      lzma_block_unpadded_size
000000000000b2c0 g    DF .text	000000000000025b  XZ_5.0      lzma_block_header_decode
0000000000003e40 g    DF .text	000000000000001b  XZ_5.0      lzma_get_check
0000000000003e60 g    DF .text	000000000000006c  XZ_5.0      lzma_memusage
000000000000f220 g    DF .text	000000000000002e  XZ_5.0      lzma_mf_is_supported
000000000000bed0 g    DF .text	000000000000017d  XZ_5.0      lzma_index_buffer_decode
00000000000069a0 g    DF .text	0000000000000014  XZ_5.0      lzma_block_buffer_encode
0000000000007710 g    DF .text	000000000000006b  XZ_5.0      lzma_properties_encode
0000000000005c50 g    DF .text	000000000000000c  XZ_5.0      lzma_index_iter_init
0000000000005290 g    DF .text	0000000000000028  XZ_5.0      lzma_index_stream_size

I got it to compile by doing:

Code: Select all

find_package(LibLZMA)
	add_library(lzma ALIAS LibLZMA::LibLZMA)
	find_path(lzma_HEADER_DIR PATH_SUFFIXES lzma NAMES lzma12.h)
	add_library(lzma_Static STATIC 
		${lzma_HEADER_DIR}/LzmaDec.h 
		${lzma_HEADER_DIR}/LzmaDec.c 
		${lzma_HEADER_DIR}/LzmaEnc.h 
		${lzma_HEADER_DIR}/LzmaEnc.c 
		${lzma_HEADER_DIR}/LzFind.c 
		${lzma_HEADER_DIR}/LzFind.h
	)
	target_compile_definitions(lzma_Static PUBLIC _7ZIP_ST)
	target_link_libraries(LibLZMA::LibLZMA INTERFACE lzma_Static)
	target_include_directories(LibLZMA::LibLZMA INTERFACE ${lzma_HEADER_DIR})
But tbh I don't know if this is the "right" way to do it. And I still don't know why these source files aren't included in the .so file.

Aki
Global Moderator
Global Moderator
Posts: 3060
Joined: 2014-07-20 18:12
Location: Europe
Has thanked: 75 times
Been thanked: 415 times

Re: Dynamically linking lzma and misconceptions I have

#4 Post by Aki »

Hello,
MultiplexLayout wrote: 2022-09-14 16:21 I am trying to convert a cmake project from static linking to dynamic linking and I am having some problems with liblzma.
What is this project ? Is its source code freely available ?
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀

MultiplexLayout
Posts: 56
Joined: 2020-09-23 19:21
Has thanked: 7 times

Re: Dynamically linking lzma and misconceptions I have

#5 Post by MultiplexLayout »

Aki wrote: 2022-09-15 17:16 Hello,
MultiplexLayout wrote: 2022-09-14 16:21 I am trying to convert a cmake project from static linking to dynamic linking and I am having some problems with liblzma.
What is this project ? Is its source code freely available ?
It is the duckstation git repo: https://github.com/stenzek/duckstation.git

Post Reply