Probable compatibility regression for removing -L if they are in LIBRARY_PATH #228
Loading…
Reference in New Issue
There is no content yet.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?
There is a bug filed against CMake that happens, because pkgconf removes
-Lfrom output if the directories are present inLIBRARY_PATH. CMake's developer Brad King claims that this behavior is not present in the originalpkg-config. If so, this is probably not a CMake's bug, but a compatibility issue of pkgconf. It would be great to have your input.There is a lot of discussion in that CMake issue leading to this discovery. The important part for
pkgconfis this:I was surprised by this behaviour once but I am actually considering arguging in the opposite direction: The mangling is OK, but it should also cover rpath flags to completely remove
-Wl,-R/foo/lib(or-Wl,-rpath,/foo/liband variants) if the path is mentioned in LD_RUN_PATH, the rpath-equivalent to LIBRARY_PATH.Couldn't CMake employ LIBRARY_PATH to look for the actual library file (assuming it insinsts on locating the .so/.a file instead of just having the linker locate it using the environment)? Is it thrown off massively by the stripped -L?
I see that CMake resolved this by forcing
PKG_CONFIG_ALLOW_SYSTEM_LIBS=0to turn off the filtering. I guess that is the right thing for CMake in its way to insist on thwarting any compiler/linker search by providing the full path to library files.An alternative viewpoint is a world where you want to set LIBRARY_PATH and LD_RUN_PATH and have builds pick them up so that you don't have to specify any
-L/foo/lib -Wl,-R/foo/liband still have the stuff properly located at link time and runtime. For that to work, the rpath-related options have to be filtered out, as any presence of those cancels LD_RUN_PATH in its entirety. I really wish there was an rpath-like PATH variable that survives this just like LIBRARY_PATH.I do admit that this behavioural change surprised me at first, too, in a situation where I set LIBRARY_PATH to some additional prefixes but did not mention /usr/lib64 in there. The result was that pkgconf filtered my add-on library paths and the command line was stuck with some -L/usr/lib64 in there, which then caused the wrong old system libs being used instead of what I put in LIBRARY_PATH (-L/usr/lib64 is stronger than that variable, even if it is not totally cancelled). I resolved that by also putting the standard system paths into LIBRARY_PATH in their proper place. But it was a surprise.
Maybe
PKG_CONFIG_ALLOW_SYSTEM_LIBS=0should be the default, after all. I'm torn, as I somehow like the cleanup, but it's not necessarily obvious.We will be keeping the current functionality as is.
CMake can use
PKG_CONFIG_ALLOW_SYSTEM_LIBSto change the behavior to work as they expect, that's the point of making it tunable.I don't get. CMake is irrelevant. pkgconf claims it is compatible with pkg-config. This bug is about this claim is being a lie.
You may read our compatibility policy here.
I strongly suggest reading it multiple times until you understand that we do not provide compatibility with suboptimal pkg-config behavior by default. A tunable was added for broken software which depends on broken pkg-config behavior. If you continue to argue on this repo, you will be banned, to be clear.
I don't like the wording of the commit message, but I can understand how you got to the impression. But to be clear: the only CMake developer you had seen in this issue so far is @bradking.
Ariadne, the only aggressive person here so far is you.
First, don't attack Brad. He posted here, because he kindly helped me to resolve a CMake issue. So if you wish, you can attack me instead.
Second, you misinterpreted my intentions. I am not demanding anything. I honestly don't care how pkgconfig works and what you do, I was just trying to help. If it is not appreciated and is perceived as aggression, I am really fine never talking with you again. There is no need to pour your frustration in the README. Though, again, I don't care if you do.
@intractabilis
This (and frankly this whole bug report, including @bradking's post) is a passive-aggressive way of demanding bug-level compatibility with pkg-config.
I have received hundreds of complaints from buildsystem authors who have depended on some pkg-config bug or another. The behaviour that CMake depends on is a pkg-config bug, specifically that pkg-config does not consistently filter out toolchain-builtin paths, despite having environmental variables for it.
I added the
PKG_CONFIG_KEEP_SYSTEM_LIBSet al tunables to enable build systems which depend on that bug (to learn the full toolchain search path list) to have something similar to the pkg-config behaviour. But that behaviour in pkg-config is not an intentional feature, it is a bug.I can think of many examples where buildsystem authors have come, and demanded, or implied that pkgconf's lack of bug-compatibility with pkg-config means its not truly compatible, only to eventually refactor their buildsystem and come back later admitting that the pkg-config behavior was actually wrong (even if it enabled convenient hacks at one time).
If your intention is to help, then please engage us in good faith and consider that perhaps there are reasons why pkgconf behaves differently. Telling us about known lack of bug-compatibility with pkg-config is not helpful, at best it encourages people to try to make pkgconf a worse software.
The lack of consistency in pkg-config behaviour, as well as completely broken behaviour which had to be redesigned from scratch means that indeed, there is not lossless fidelity with pkg-config. That's a feature, not a bug: I wrote pkgconf out of frustration with pkg-config's bugs, intentionally designing it to make the most of its knowledge of the system toolchain.
That that knowledge no longer "leaks" to consumers (like CMake) without giving pkgconf a lobotomy via its environment variables, is not a bug: pkgconf is doing exactly what you asked it to do -- you told it about your toolchain, and it adjusted its behavior to remove flags that are not needed by the toolchain as they are implicit.
Calling this intentional lack of bug-compatibility a bug in pkgconf is disingenuous, and serves to make the entire project pointless: we intentionally changed some behaviors of pkgconf from what legacy pkg-config did so we could make progress for the better. While I am sorry that some buildsystems have had hacks that depend on those pkg-config bugs, I'm not going to provide bug-level compatibility with pkg-config, that again, defeats the entire purpose of the project.
If you don't like that I am grouchy about people who come and ask for bug-level compatibility, please realize that this conversation has happened hundreds of times, and every single time the buildsystem author has built a more robust buildsystem as a result of our stubbornness on this topic.
By the way, this CMake patch is actually the correct fix, not using
PKG_CONFIG_KEEP_SYSTEM_LIBSenv variable.I am not demanding anything, and I am not sure where you see aggression. If I don't care, why I would be aggressive? Brad's post is absolutely stripped of anything emotionally loaded.
Please, keep in mind, English is a second language for me, so I probably don't see what you are reacting on. I am also not a neurotypical person and use the language pretty straightforwardly, it's a part of my mental disability. Thus, there is nothing between the lines to read.
I don't see anything related to
LIBRARY_PATHin pkg-config source code. It looks like the author never meant to take it into account, and thus it looks like a feature. Why do you think it's a bug? Just in case, I am asking without aggression, I am just curious.I didn't call it intentional. On the contrary, I said it's probably a compatibility issue, and it would be great to have your input. I am grateful for your final explanation, but calling me or Brad aggressive wasn't justified as a foreword, and was aggressive per se.
Btw, for my personal needs I switched to Meson. I consider having to deal with CMake as an unfortunate burden, whenever I have to compile other libraries, or if my clients demand CMake. Meson relies on .pc, and I think it's a great combo.
I agree, it’s just that this is a rather sore spot for me these days because I’ve spent the past 10 years trying to fix bugs in pkg-config to only hit friction like this on occasion.
While I can understand that CMake wants to resolve the .so’s itself, it should be doing so in the same way the toolchain does. That’s why pkgconf follows what the toolchain does, too: we try to provide the most efficient output that has the same semantic meaning, using our knowledge of how the toolchain will behave.
LIBRARY_PATH itself is not supported by pkg-config at runtime but the default is extracted from gcc at build time. I consider it to be a bug because of that inconsistency… in fact the lack of consistency here has caused me problems before.
Sorry, to me it came off in the same way that people say “please advise” when they are demanding some action be taken. In other words it came across more as “it would be great for you to explain yourself”. But admittedly, I may apply bias here as again this is a sore spot :)
The Meson developers appreciate the changes in behaviour from the freedesktop implementation, I think. At least they participated in driving some of them.
I should also admit that I have some bias against CMake, for other reasons 🙃
I agree. My point in the discussion with them was that if a user uses
PkgConfigmodule, she would expect CMake to mimic the behavior of pkg-config. Now, with two possible versions of the tool, I am not sure how they should resolve the dilemma. Mimicking the old and bad one is an option.Oh... at the build time of pkg-config? That's bizarre, definitely has zero sense. Thanks for clarifying.
Meson is cool. I believe it's the future.
I understand, no worries. Everything is cool. Let's be friends. :)
TBH, freedesktop pkg-config is the past. In most cases, you wind up with pkgconf on Debian/Ubuntu these days, and they were kind of the last holdout. CMake should focus on the present.
(the future is probably one that does not involve pkg-config at all, but instead something like
cargoor whatever)We should be careful predicting the future. For me as a distro packager, the cargo cult (pun fully intentional by upstream, I guess) is a heavy annoyance … you have package managers nested in package managers, battling each other, with differing design goals. And each new language ecosystem rolls out a new package manager — and maybe at some point an old language like C++ does, too (well, or you think about Conan or what's it called).
I make the bold prediciton that for the next 10 years, the pkg-config interface will stay relevant and important. Actually, I pretty much wish for lots of FindFoo.cmake functions to die in favour of it. Especially with CMake, there is a lot of hacking of redundant dependency detection code, both in CMake itself (shipped modules) and in projects using CMake.
I have a patch I want to submit to CMake upstream for making FindBLAS.cmake and FindLAPACK.cmake more usable by being able to tell them the pkg-config module name to use. Apart from deferring to pkg-config, those CMake modules contain code to detect specific implementations of BLAS/LAPACK API (multiple implementations of one API being something not really modelled by pkg-config, as it's rather rare, but important in the case of differently optimized BLAS). Such code is duplicated in a lot of projects and cause headaches for a (source) distro package that wants to get a consistent choice of the actual dependency lib being used.
Having everything consistently rely on pkg-config would be significant progress from today. Please let's announce a migration from pkg-config well after that;-)
The BLAS situation could also be addressed by some games with standard names like blas.pc in neighbouring directories and playing with PKG_CONFIG_PATH … but in some cases the explicit name of the package is needed anyway, as the libs are alternatives to each other, but not with totally identical interface.
Me too.