improved cross-compile/multiarch support using personality files #166

New Issue

ariadne · 2018-01-05T18:17:54Z

ariadne commented

2018-01-05 18:17:54 +00:00

Right now the preferred approach for cross-compiling with pkgconf is to write a wrapper script that describes the target to pkgconf. By adding a simple shim layer, we could replace this with a simple configuration file instead.

Using argv[0] we can deduce whether or not we are running in native mode or cross mode.

If we are in native mode, we just load default.personality from the configuration dir. If we are in cross mode, then we just load *triplet*.personality from the configuration dir.

A cross-compiler config may look something like:

Personality: i386-adelie-linux-musl
SystemIncludePaths: /usr/i386-adelie-linux-musl/include
SystemLibraryPaths: /usr/i386-adelie-linux-musl/lib
DefaultSearchPaths: /usr/i386-adelie-linux-musl/lib/pkgconfig /usr/i386-adelie-linux-musl/share/pkgconfig

In this example, the cross-compiler is actually just the i386 compiler running on an x86_64 host, but it is important to note that the paths are relative to the sysroot, like you would do with a real cross-compiler. In other words, multi-arch and cross-compilers should be handled the same by pkgconf.

I would like to move the compiled-in data to the default.personality file so that the site administrator can more easily configure the tool's default behaviour. Perhaps a combination of /usr/share/pkgconf/personalities/default.personality and /etc/pkgconf-personalities.d/default.personality to allow site-specific customization of any personality.

cc @ignatenkobrain

Right now the preferred approach for cross-compiling with pkgconf is to write a wrapper script that describes the target to pkgconf. By adding a simple shim layer, we could replace this with a simple configuration file instead. Using `argv[0]` we can deduce whether or not we are running in native mode or cross mode. If we are in native mode, we just load `default.personality` from the configuration dir. If we are in cross mode, then we just load `*triplet*.personality` from the configuration dir. A cross-compiler config may look something like: ``` Personality: i386-adelie-linux-musl SystemIncludePaths: /usr/i386-adelie-linux-musl/include SystemLibraryPaths: /usr/i386-adelie-linux-musl/lib DefaultSearchPaths: /usr/i386-adelie-linux-musl/lib/pkgconfig /usr/i386-adelie-linux-musl/share/pkgconfig ``` In this example, the cross-compiler is actually just the i386 compiler running on an x86_64 host, but it is important to note that the paths are relative to the sysroot, like you would do with a real cross-compiler. In other words, multi-arch and cross-compilers should be handled the same by pkgconf. I would like to move the compiled-in data to the `default.personality` file so that the site administrator can more easily configure the tool's default behaviour. Perhaps a combination of `/usr/share/pkgconf/personalities/default.personality` and `/etc/pkgconf-personalities.d/default.personality` to allow site-specific customization of any personality. cc @ignatenkobrain

👍 1

Conan-Kudo commented

2018-01-05 18:35:20 +00:00

(Migrated from github.com)

Poster

Yes, it'd be quite nice to be able to replace what I've done here: cf41f5098b

Yes, it'd be quite nice to be able to replace what I've done here: https://src.fedoraproject.org/rpms/pkgconf/c/cf41f5098b598217c3745f6ddf3a5dade608cf0a

ignatenkobrain commented

2018-01-05 18:42:48 +00:00

(Migrated from github.com)

Poster

This sounds good to me!

Earnestly commented

2018-05-20 15:21:12 +00:00

(Migrated from github.com)

Poster

Just to note, since this seems related, I am getting build failures on the current commit (ab8df57205)

Using Linux and glibc, so no strlcpy here, but PERSONALITY_PATH not being defined is problematic:

ninja: Entering directory `build'
[1/20] Compiling C object 'pkgconf@sha/libpkgconf_audit.c.o'.
[2/20] Compiling C object 'pkgconf@sha/libpkgconf_bsdstubs.c.o'.
[3/20] Compiling C object 'pkgconf@sha/libpkgconf_cache.c.o'.
[4/20] Compiling C object 'pkgconf@sha/libpkgconf_argvsplit.c.o'.
[5/20] Compiling C object 'pkgconf@sha/libpkgconf_client.c.o'.
[6/20] Compiling C object 'pkgconf@sha/libpkgconf_fileio.c.o'.
[7/20] Compiling C object 'pkgconf@sha/libpkgconf_parser.c.o'.
[8/20] Compiling C object 'pkgconf@sha/libpkgconf_dependency.c.o'.
[9/20] Compiling C object 'pkgconf@sha/libpkgconf_path.c.o'.
[10/20] Compiling C object 'pkgconf@sha/libpkgconf_personality.c.o'.
FAILED: pkgconf@sha/libpkgconf_personality.c.o 
ccache cc -Ipkgconf@sha -I. -I.. -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -O0 -g -march=x86-64 -mtune=generic -O2 -fstack-protector-strong -fno-plt -D_FORTIFY_SOURCE=2 -fPIC -DLIBPKGCONF_EXPORT  -MD -MQ 'pkgconf@sha/libpkgconf_personality.c.o' -MF 'pkgconf@sha/libpkgconf_personality.c.o.d' -o 'pkgconf@sha/libpkgconf_personality.c.o' -c ../libpkgconf/personality.c
../libpkgconf/personality.c: In function ‘load_personality_with_path’:
../libpkgconf/personality.c:195:3: warning: implicit declaration of function ‘strlcpy’; did you mean ‘strncpy’? [-Wimplicit-function-declaration]
   strlcpy(pathbuf, path, sizeof pathbuf);
   ^~~~~~~
   strncpy
../libpkgconf/personality.c: In function ‘pkgconf_cross_personality_find’:
../libpkgconf/personality.c:233:21: error: ‘PERSONALITY_PATH’ undeclared (first use in this function)
  pkgconf_path_split(PERSONALITY_PATH, &plist, true);
                     ^~~~~~~~~~~~~~~~
../libpkgconf/personality.c:233:21: note: each undeclared identifier is reported only once for each function it appears in
[11/20] Compiling C object 'pkgconf@sha/libpkgconf_queue.c.o'.
[12/20] Compiling C object 'pkgconf@sha/libpkgconf_fragment.c.o'.
[13/20] Compiling C object 'pkgconf@sha/libpkgconf_tuple.c.o'.
[14/20] Compiling C object 'pkgconf@exe/cli_renderer-msvc.c.o'.
[15/20] Compiling C object 'pkgconf@sha/libpkgconf_pkg.c.o'.
ninja: build stopped: subcommand failed.

Just to note, since this seems related, I am getting build failures on the current commit (https://github.com/pkgconf/pkgconf/commit/ab8df57205e3f10a2a5bd9e0a7bc79338a9465c0) Using Linux and glibc, so no `strlcpy` here, but `PERSONALITY_PATH` not being defined is problematic: ``` ninja: Entering directory `build' [1/20] Compiling C object 'pkgconf@sha/libpkgconf_audit.c.o'. [2/20] Compiling C object 'pkgconf@sha/libpkgconf_bsdstubs.c.o'. [3/20] Compiling C object 'pkgconf@sha/libpkgconf_cache.c.o'. [4/20] Compiling C object 'pkgconf@sha/libpkgconf_argvsplit.c.o'. [5/20] Compiling C object 'pkgconf@sha/libpkgconf_client.c.o'. [6/20] Compiling C object 'pkgconf@sha/libpkgconf_fileio.c.o'. [7/20] Compiling C object 'pkgconf@sha/libpkgconf_parser.c.o'. [8/20] Compiling C object 'pkgconf@sha/libpkgconf_dependency.c.o'. [9/20] Compiling C object 'pkgconf@sha/libpkgconf_path.c.o'. [10/20] Compiling C object 'pkgconf@sha/libpkgconf_personality.c.o'. FAILED: pkgconf@sha/libpkgconf_personality.c.o ccache cc -Ipkgconf@sha -I. -I.. -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -O0 -g -march=x86-64 -mtune=generic -O2 -fstack-protector-strong -fno-plt -D_FORTIFY_SOURCE=2 -fPIC -DLIBPKGCONF_EXPORT -MD -MQ 'pkgconf@sha/libpkgconf_personality.c.o' -MF 'pkgconf@sha/libpkgconf_personality.c.o.d' -o 'pkgconf@sha/libpkgconf_personality.c.o' -c ../libpkgconf/personality.c ../libpkgconf/personality.c: In function ‘load_personality_with_path’: ../libpkgconf/personality.c:195:3: warning: implicit declaration of function ‘strlcpy’; did you mean ‘strncpy’? [-Wimplicit-function-declaration] strlcpy(pathbuf, path, sizeof pathbuf); ^~~~~~~ strncpy ../libpkgconf/personality.c: In function ‘pkgconf_cross_personality_find’: ../libpkgconf/personality.c:233:21: error: ‘PERSONALITY_PATH’ undeclared (first use in this function) pkgconf_path_split(PERSONALITY_PATH, &plist, true); ^~~~~~~~~~~~~~~~ ../libpkgconf/personality.c:233:21: note: each undeclared identifier is reported only once for each function it appears in [11/20] Compiling C object 'pkgconf@sha/libpkgconf_queue.c.o'. [12/20] Compiling C object 'pkgconf@sha/libpkgconf_fragment.c.o'. [13/20] Compiling C object 'pkgconf@sha/libpkgconf_tuple.c.o'. [14/20] Compiling C object 'pkgconf@exe/cli_renderer-msvc.c.o'. [15/20] Compiling C object 'pkgconf@sha/libpkgconf_pkg.c.o'. ninja: build stopped: subcommand failed. ```

ariadne commented

2018-05-20 16:33:47 +00:00

Poster

yes i need to sort out meson and cmake files. i'll do it in a bit.

Sign in to join this conversation.