Repository navigation
23.0: issue with generated protobuf.pc files (it depends on utf8_range) #12746
Description
Looks like generated protobuf.pc has in Requires utf8_range.
prefix=/usr
exec_prefix=/usr
libdir=/usr/lib64
includedir=/usr/include
Name: Protocol Buffers
Description: Google's Data Interchange Format
Version: 23.0.0
Requires: absl_absl_check absl_absl_log absl_algorithm absl_base absl_bind_front absl_bits absl_btree absl_cleanup absl_cord absl_core_headers absl_debugging absl_die_if_null absl_dynamic_annotations absl_flags absl_flat_hash_map absl_flat_hash_set absl_function_ref absl_hash absl_layout absl_log_initialize absl_log_severity absl_memory absl_node_hash_map absl_node_hash_set absl_optional absl_span absl_status absl_statusor absl_strings absl_synchronization absl_time absl_type_traits absl_utility absl_variant utf8_range
Libs: -L${libdir} -lprotobuf
Cflags: -I${includedir} -DPROTOBUF_USE_DLLS
Conflicts: protobuf-lite
Activity
kloczek commentedon May 9, 2023
In generated protobuf-config.cmake I see as well:
kloczek commentedon May 9, 2023
Just in case I've build protobuf with
-D utf8_range_ENABLE_INSTALL=OFF.kloczek commentedon May 10, 2023
I've added quick hack to remove
utf8_rangefrom Requires on rpm package %install stage.With such product I was able to start testing use generated package on building other packages.
Results:
In other words looks likke protobuf 23.0 is not ready to be used.
kloczek commentedon May 11, 2023
Gentle ping .. any update?
I was able to rebuild protobuf-c usimg 22.5 however still on rebuilding libphonenumber, mysql, opencv are failing on missing symbols.
fowles commentedon May 12, 2023
@coryan do you have a moment to take a look at this?
coryan commentedon May 12, 2023
Happy to do a drive-by. It would be useful to know what platform (Linux, Windows, etc.) this was on, and what version of the compiler. Looks like some Linux distro, with GCC.
The dependency on
utf8_rangeIs
utf8_rangea hard dependency? If it is, then the install instructions should say so. Consider adding step-by-step instructions to build all the dependencies from source in https://github.com/protocolbuffers/protobuf/blob/main/src/README.mdIf it is not a hard dependency, then this:
protobuf/cmake/install.cmake
Line 7 in 56d1b0f
Needs to have a conditional:
The broken build for
libphonenumberThis error:
was fixed in #12700. I think @fowles already backported that fix to the 22.x and 23.x branches.
The broken build for
mysqlThis error
Seems like a missing
-labsl*. Most likelymysqlis usingfind_package(protobuf), using theFindProtobufembedded with CMake and missing all the new deps. That is indeed the case:https://github.com/mysql/mysql-server/blob/ea7087d885006918ad54458e7aad215b1650312c/cmake/protobuf.cmake#L124
That needs to change to
find_package(protobuf CONFIG). See #12292. It is becoming a FAQ, the Protobuf team should consider updating the release notes.The broken build for
opencvI think in this case the error explains itself:
The opencv library was configured (or maybe defaults to) compile as C++11:
I am not familiar with the configuration options for opencv. It may be necessary to use
cmake -DCMAKE_CXX_STANDARD=14 ..., or maybe just omit the-DCMAKE_CXX_STANDARD=14altogether. Most compilers default to C++14 or higher these days. There are very few cases where one needs to explicitly request C++11.In summary:
utf8_rangeis not a hard-dependency.opencvmay need some tweaks.HTH
kloczek commentedon May 13, 2023
I dont see any fix for that in 22.x branch v22.5...22.x
None of the commits from #12700 are on the list
v22.5...22.x
mysql is not using absl API and error message shows missing symbol in libprotoc -> it points on libprotoc underlinking with some of the absl DSOs.
As I've wrote I've tested that with 23.0.
I disagree. It compiles and linking fails because missing symbols.
coryan commentedon May 13, 2023
Yes. I said "if it is not a hard dependency". But I looked at the code again and it is. That means this:
Was probably not a good idea.
I am not sure why that matters since you are testing with 23.0. FWIW, the relevant commit in the 22.x series is 18fae1c and it is there:
v22.4...22.x
The relevant commit in the 23.x series is 1ca4e9c
I am not sure I follow. That is intentional.
libprotoc.sodepends on the Abseil symbols. Some of the abseil symbols are part of the public API of libprotoc. It is not expected that-lprotocby itself will be enough. If it did (for example by linking all the absl* libraries into the.so), I think applications that used Abseil and libprotoc at the same time would quickly run into ODR violations.Thanks that is useful. Could you also share the specific Linux distro and the steps you are using to compile Protobuf? It would make it easier to repro any bugs locally.
I think you are expecting that linking with just
-lprotocshould work. It does not and it is not expected to. The Protobuf team has fixed the pkg-config modules and the CMake config files to add any new-lflags. That should make it easier to migrate build scripts to add the flags where needed.Build scripts written for Protobuf <= 21.x must be updated to take into account the new dependencies. If a build script (like mysql's) relies on CMake's
FindProtobufmodule it needs to be updated. That is becauseFindProtobufdoes not know about the new dependencies.mysql uses that module and that is why the build is missing
-lflags.I also do not think the Protobuf team should (or could) fix the
FindProtobufmodule. First, because they cannot go back in time and fix the version of that module that already shipped everywhere. Second, because CMake's team plan of record is to migrate to the config files that Protobuf already generates and installs.I appreciate this is maybe surprising and/or frustrating. But I am not sure what actions are you suggesting.
kloczek commentedon May 13, 2023
Nope I'm now on 22.5 because with that version I was able to compile everythiong and have now only linking issues.
kloczek commentedon May 13, 2023
Trying to figure out why libprotobuf DSO does not provdes `google::protobuf::internal::ThreadSafeArena::thread_cache_' I found another "intersting: thing.
Despite
in which is possible to see
protobuf_BUILD_LIBPROTOC:BOOL=OFFlibprotoc DSO is build, linked and installed.kloczek commentedon May 13, 2023
BTW looks like 18fae1c is already included in 22.5.
kloczek commentedon May 13, 2023
According to
protobuf/cmake/README.md
Lines 296 to 314 in 17a2d53
PROTOBUF_USE_DLLSdefine should be used only on Win.Going to try to roll back 18fae1c.
coryan commentedon May 13, 2023
I think the documentation is very much out of date and I filed a bug requesting fixes (see #12787).
If you compile the library with PROTOBUF_USE_DLLS defined and use it with that undefined you will have an inconsistent definition for this class:
protobuf/src/google/protobuf/thread_safe_arena.h
Lines 264 to 271 in 17a2d53
And that macro is defined when CMake creates shared libraries. Both on Windows and other platforms:
protobuf/cmake/libprotobuf.cmake
Lines 30 to 34 in 42b20b3
That is what causes the undefined
google::protobuf::internal::ThreadSafeArena::thread_cache_symbol.Are you sure you are using the correct
.pcfile when building the downstream dependency? Or if you are not using.pcfiles and you are using CMake: is the downstream build using the protobuf config (as opposed to theFindProtobufmodule)?kloczek commentedon May 13, 2023
Yes I'm sure.
Generated .pc files have that
-DPROTOBUF_USE_DLLS(I've done that without checking possible impact of that kind of change).Of course reverte 18fae1c caused that it was not possible to build protobuf baceuse other sybols in that case where missing.
Nevertheless looks liks like something is wrong around use that define in some headers.
40 remaining items
kloczek commentedon May 25, 2023
Cmake settings:
kloczek commentedon May 25, 2023
libphonenumber maintainers just mentioned that they are observing as well linking issue https://issuetracker.google.com/issues/283987730#comment6
Use the same ABI for static and shared libraries on non-Windows platf…
Romain-Geissler-1A commentedon Jun 6, 2023
Hi,
FYI, on the main branch, the linking errors about
thread_cache_(and only this error, as this issue seems to mix several unrelated issues) should now be gone (for all non Windows users).Note: I am not involved at all in Protobuf development, I am just an external contributor who hit this issue and decided to submit a fix for it ;)
Cheers,
Romain
carlocab commentedon Jul 7, 2023
Not sure what the status of this issue is, but here's some information that might be helpful. At Homebrew:
-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ONand-Dprotobuf_MODULE_COMPATIBLE=ONtocmake.2 I believe this confirms 23.0: issue with generated protobuf.pc files (it depends onutf8_range) #12746 (comment).Footnotes
https://github.com/Homebrew/homebrew-core/pull/133623 ↩
https://github.com/Homebrew/homebrew-core/blob/244feeee298712178bd3cd60da0091d5a5b79a71/Formula/opencv.rb#L109-L110 ↩
https://bugs.mysql.com/bug.php?id=111469 ↩
LazyPlanet commentedon Jul 17, 2023
Linux centos also has error:
/usr/local/include/google/protobuf/thread_safe_arena.h:198: undefined reference to
google::protobuf::internal::ThreadSafeArena::thread_cache_' /usr/local/include/google/protobuf/thread_safe_arena.h:198: undefined reference togoogle::protobuf::internal::ThreadSafeArena::thread_cache_'/usr/local/include/google/protobuf/thread_safe_arena.h:198: undefined reference to `google::protobuf::internal::ThreadSafeArena::thread_cache_'
if message has repeated int32 or int64, i will get errors above, why?
while if message has repeated other type such as message or bytes, it build success...
Romain-Geissler-1A commentedon Sep 16, 2023
Why not using up to date probotuf where this very problem is fixed already, without any hack required ?
Fix linking error
Fix linking errors
Fix linking errors
Fix linking errors
github-actions commentedon Dec 17, 2023
We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.
This issue is labeled
inactivebecause the last activity was over 90 days ago.github-actions commentedon Jan 2, 2024
We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please reopen it.
This issue was closed and archived because there has been no new activity in the 14 days since the
inactivelabel was added.