Repository navigation
Feature request: Allow the build to use the system-installed protobuf lib #16104
Description
Describe the problem
Currently, it's impossible to use the system-installed protobuf library because the tensorflow build always uses the protobuf_archive version. There should be an option to use the one installed in the system.
Background: I package tensorflow for Arch Linux and we run into symbol conflicts if a user wants to use protobuf and tensorflow together in a binary because tensorflow's protobuf symbols conflict with the one installed in the system already.
Original Arch bug report: https://bugs.archlinux.org/task/56943
Activity
[-]Allow the build to use the system-installed protobuf lib[/-][+]Feature request: Allow the build to use the system-installed protobuf lib[/+]tensorflowbutler commentedon Jan 14, 2018
Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce
svenstaro commentedon Jan 22, 2018
sigh for this kind of issue, it really doesn't matter but for the sake of automation, here I go:
drpngx commentedon Jan 30, 2018
I think there was a similar question but can't seem to find it.
@gunan what's our position here?
gunan commentedon Jan 31, 2018
Eventually, we would like to enable this.
Maybe we can control this using environment variables just like we do for MKL?
dbarnes commentedon Feb 18, 2018
@drpngx could it be this question I posted a while ago #8403 ? I'd be very interested as well if any progress is made on this!
drpngx commentedon Feb 18, 2018
@danbarnes333 yes that's right on.
Maybe a stepping stone would be to allow people to link against the system protobuf during the
configurestep, via some environment variable or a configure flag. I think that would be a good contribution.We should also consider, at some point, distributing pip packages that are dependent on a libprotobuf package.
CC @allenlavoie @skye
RichDubielzig commentedon Feb 22, 2018
This would be useful to us as well. We are currently stuck in a situation where the protobuf symbols in libtensorflow_framework.so are being clobbered by symbols in a custom libprotobuf from a different API revision that is being loaded to support the operations of our custom backend to TensorFlow.
In other words, we have this library dependency tree right now:
RichDubielzig commentedon Mar 9, 2018
We found a workaround by forcing bazel to use a protobuf version that matched the one used elsewhere on the system. Here is a patch that will force usage of the 3.4.0 library, but one can also point to any other archive tag on github.
patch.tar.gz
dbarnes commentedon Mar 9, 2018
@RichDubielzig that is what we have been doing, as well as hiding more symbols in the tensorflow shared library, and has been working ok. However @drpngx mentioned on my original issue that this isnt completely safe unless I misunderstood (about halfway down #8403) but perhaps things have changed since then. Either way simply building against the system protobuf would be great.
8 remaining items
svenstaro commentedon Mar 28, 2023
This is definitely still valid.
github-actions commentedon Sep 25, 2023
This issue is stale because it has been open for 180 days with no activity. It will be closed if no further activity occurs. Thank you.
svenstaro commentedon Sep 25, 2023
It's still relevant.
github-actions commentedon Mar 24, 2024
This issue is stale because it has been open for 180 days with no activity. It will be closed if no further activity occurs. Thank you.
svenstaro commentedon Mar 24, 2024
It's still a problem.
PR #16104: [GPU][NFC] Further cleanup FMHA graph creation code.
PR #16104: [GPU][NFC] Further cleanup FMHA graph creation code.
svenstaro commentedon Aug 17, 2024
I don't see how #73933 fixes this.
aikawayataro commentedon Aug 18, 2024
Probably because issue # collided with openxla/xla#16104, which was closed in the imported patch
Any way to reopen this?