libpkgconf segmentation fault on MacOS #149

New Issue

2017-10-13T19:40:57Z

karen-arutyunov commented

2017-10-13 19:40:57 +00:00

(Migrated from github.com)

Hello,

Currently libpkgconf uses PKGCONF_BUFSIZE size for all stack-allocated buffers, imposing high requirements for the thread stack size. This make it unusable on MacOS (as of 10.12) for non-main threads that are created with the default 512KB stack size. In particular that make it impossible to use libpkgconf API in threads created with C++11 std::thread class, that doesn't allow stack size customization.

As an example, using pkgconf_pkg_find() for quite a simple .pc file consumes at least 460KB. The measurement was made as a difference between addresses of 2 stack-allocated variables: one was defined right before the function call, another right after the buffer definition in pkgconf_tuple_parse() function.

As a result our program, that calls API from worker threads, were crashing due to segmentation fault on MacOS.

To relax the stack size requirements we have patched libpkgconf to minimize the usage of PKGCONF_BUFSIZE, inventing the smaller PKGCONF_ITEM_SIZE, and using it wherever it is possible instead. The PKGCONF_ITEM_SIZE is selected in such a way that a buffer of this size can accommodate the file system path, the value fragment, the variable name or the package dependency specification. The latest implies it also fits for the package key or the package version. This optimization decreased the stack usage for the described use case to 140KB.

I have attached the patch in case you find it useful for the issue fix.

Thanks,
Karen
pkgconf.patch.gz

Hello, Currently libpkgconf uses PKGCONF_BUFSIZE size for all stack-allocated buffers, imposing high requirements for the thread stack size. This make it unusable on MacOS (as of 10.12) for non-main threads that are created with the default 512KB stack size. In particular that make it impossible to use libpkgconf API in threads created with C++11 std::thread class, that doesn't allow stack size customization. As an example, using pkgconf_pkg_find() for quite a simple .pc file consumes at least 460KB. The measurement was made as a difference between addresses of 2 stack-allocated variables: one was defined right before the function call, another right after the buffer definition in pkgconf_tuple_parse() function. As a result our program, that calls API from worker threads, were crashing due to segmentation fault on MacOS. To relax the stack size requirements we have patched libpkgconf to minimize the usage of PKGCONF_BUFSIZE, inventing the smaller PKGCONF_ITEM_SIZE, and using it wherever it is possible instead. The PKGCONF_ITEM_SIZE is selected in such a way that a buffer of this size can accommodate the file system path, the value fragment, the variable name or the package dependency specification. The latest implies it also fits for the package key or the package version. This optimization decreased the stack usage for the described use case to 140KB. I have attached the patch in case you find it useful for the issue fix. Thanks, Karen [pkgconf.patch.gz](https://github.com/pkgconf/pkgconf/files/1383802/pkgconf.patch.gz)

Sign in to join this conversation.