“libc++” C++ Standard Library¶
Overview¶
libc++ is a new implementation of the C++ standard library, targeting C++11 and above.
- Features and Goals
- Correctness as defined by the C++11 standard.
- Fast execution.
- Minimal memory use.
- Fast compile times.
- ABI compatibility with gcc’s libstdc++ for some low-level features such as exception objects, rtti and memory allocation.
- Extensive unit tests.
- Design and Implementation:
- Extensive unit tests
- Internal linker model can be dumped/read to textual format
- Additional linking features can be plugged in as “passes”
- OS specific and CPU specific code factored out
Getting Started with libc++¶
Using libc++¶
Getting Started¶
If you already have libc++ installed you can use it with clang.
$ clang++ -stdlib=libc++ test.cpp
$ clang++ -std=c++11 -stdlib=libc++ test.cpp
On OS X and FreeBSD libc++ is the default standard library
and the -stdlib=libc++
is not required.
If you want to select an alternate installation of libc++ you can use the following options.
$ clang++ -std=c++11 -stdlib=libc++ -nostdinc++ \
-I<libcxx-install-prefix>/include/c++/v1 \
-L<libcxx-install-prefix>/lib \
-Wl,-rpath,<libcxx-install-prefix>/lib \
test.cpp
The option -Wl,-rpath,<libcxx-install-prefix>/lib
adds a runtime library
search path. Meaning that the systems dynamic linker will look for libc++ in
<libcxx-install-prefix>/lib
whenever the program is run. Alternatively the
environment variable LD_LIBRARY_PATH
(DYLD_LIBRARY_PATH
on OS X) can
be used to change the dynamic linkers search paths after a program is compiled.
An example of using LD_LIBRARY_PATH
:
$ clang++ -stdlib=libc++ -nostdinc++ \
-I<libcxx-install-prefix>/include/c++/v1
-L<libcxx-install-prefix>/lib \
test.cpp -o
$ ./a.out # Searches for libc++ in the systems library paths.
$ export LD_LIBRARY_PATH=<libcxx-install-prefix>/lib
$ ./a.out # Searches for libc++ along LD_LIBRARY_PATH
Using <filesystem>
and libc++fs¶
Libc++ provides the implementation of the filesystem library in a separate
library. Users of <filesystem>
and <experimental/filesystem>
are
required to link -lc++fs
.
Note
Prior to libc++ 7.0, users of <experimental/filesystem>
were required
to link libc++experimental.
Warning
The Filesystem library is still experimental in nature. As such normal guarantees about ABI stability and backwards compatibility do not yet apply to it. In the future, this restriction will be removed.
Using libc++experimental and <experimental/...>
¶
Libc++ provides implementations of experimental technical specifications
in a separate library, libc++experimental.a
. Users of <experimental/...>
headers may be required to link -lc++experimental
.
$ clang++ -std=c++14 -stdlib=libc++ test.cpp -lc++experimental
Libc++experimental.a may not always be available, even when libc++ is already installed. For information on building libc++experimental from source see Building Libc++ and libc++experimental CMake Options.
Note that as of libc++ 7.0 using the <experimental/filesystem>
requires linking
libc++fs instead of libc++experimental.
Also see the Experimental Library Implementation Status page.
Warning
- Experimental libraries are Experimental.
- The contents of the
<experimental/...>
headers andlibc++experimental.a
library will not remain compatible between versions. - No guarantees of API or ABI stability are provided.
- The contents of the
Using libc++ on Linux¶
On Linux libc++ can typically be used with only ‘-stdlib=libc++’. However some libc++ installations require the user manually link libc++abi themselves. If you are running into linker errors when using libc++ try adding ‘-lc++abi’ to the link line. For example:
$ clang++ -stdlib=libc++ test.cpp -lc++ -lc++abi -lm -lc -lgcc_s -lgcc
Alternately, you could just add libc++abi to your libraries list, which in most situations will give the same result:
$ clang++ -stdlib=libc++ test.cpp -lc++abi
Using libc++ with GCC¶
GCC does not provide a way to switch from libstdc++ to libc++. You must manually configure the compile and link commands.
In particular you must tell GCC to remove the libstdc++ include directories
using -nostdinc++
and to not link libstdc++.so using -nodefaultlibs
.
Note that -nodefaultlibs
removes all of the standard system libraries and
not just libstdc++ so they must be manually linked. For example:
$ g++ -nostdinc++ -I<libcxx-install-prefix>/include/c++/v1 \
test.cpp -nodefaultlibs -lc++ -lc++abi -lm -lc -lgcc_s -lgcc
GDB Pretty printers for libc++¶
GDB does not support pretty-printing of libc++ symbols by default. Unfortunately libc++ does not provide pretty-printers itself. However there are 3rd party implementations available and although they are not officially supported by libc++ they may be useful to users.
Known 3rd Party Implementations Include:
Libc++ Configuration Macros¶
Libc++ provides a number of configuration macros which can be used to enable or disable extended libc++ behavior, including enabling “debug mode” or thread safety annotations.
- _LIBCPP_DEBUG:
- See Using Debug Mode for more information.
- _LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS:
- This macro is used to enable -Wthread-safety annotations on libc++’s
std::mutex
andstd::lock_guard
. By default these annotations are disabled and must be manually enabled by the user. - _LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS:
- This macro is used to disable all visibility annotations inside libc++. Defining this macro and then building libc++ with hidden visibility gives a build of libc++ which does not export any symbols, which can be useful when building statically for inclusion into another library.
- _LIBCPP_DISABLE_EXTERN_TEMPLATE:
- This macro is used to disable extern template declarations in the libc++ headers. The intended use case is for clients who wish to use the libc++ headers without taking a dependency on the libc++ library itself.
- _LIBCPP_ENABLE_TUPLE_IMPLICIT_REDUCED_ARITY_EXTENSION:
This macro is used to re-enable an extension in std::tuple which allowed it to be implicitly constructed from fewer initializers than contained elements. Elements without an initializer are default constructed. For example:
std::tuple<std::string, int, std::error_code> foo() { return {"hello world", 42}; // default constructs error_code }
Since libc++ 4.0 this extension has been disabled by default. This macro may be defined to re-enable it in order to support existing code that depends on the extension. New use of this extension should be discouraged. See PR 27374 for more information.
Note: The “reduced-arity-initialization” extension is still offered but only for explicit conversions. Example:
auto foo() { using Tup = std::tuple<std::string, int, std::error_code>; return Tup{"hello world", 42}; // explicit constructor called. OK. }
- _LIBCPP_DISABLE_ADDITIONAL_DIAGNOSTICS:
This macro disables the additional diagnostics generated by libc++ using the diagnose_if attribute. These additional diagnostics include checks for:
- Giving set, map, multiset, multimap a comparator which is not const callable.
- _LIBCPP_NO_VCRUNTIME:
Microsoft’s C and C++ headers are fairly entangled, and some of their C++ headers are fairly hard to avoid. In particular, vcruntime_new.h gets pulled in from a lot of other headers and provides definitions which clash with libc++ headers, such as nothrow_t (note that nothrow_t is a struct, so there’s no way for libc++ to provide a compatible definition, since you can’t have multiple definitions).
By default, libc++ solves this problem by deferring to Microsoft’s vcruntime headers where needed. However, it may be undesirable to depend on vcruntime headers, since they may not always be available in cross-compilation setups, or they may clash with other headers. The _LIBCPP_NO_VCRUNTIME macro prevents libc++ from depending on vcruntime headers. Consequently, it also prevents libc++ headers from being interoperable with vcruntime headers (from the aforementioned clashes), so users of this macro are promising to not attempt to combine libc++ headers with the problematic vcruntime headers. This macro also currently prevents certain operator new/operator delete replacement scenarios from working, e.g. replacing operator new and expecting a non-replaced operator new[] to call the replaced operator new.
C++17 Specific Configuration Macros¶
- _LIBCPP_ENABLE_CXX17_REMOVED_FEATURES:
- This macro is used to re-enable all the features removed in C++17. The effect is equivalent to manually defining each macro listed below.
- _LIBCPP_ENABLE_CXX17_REMOVED_UNEXPECTED_FUNCTIONS:
- This macro is used to re-enable the set_unexpected, get_unexpected, and unexpected functions, which were removed in C++17.
- _LIBCPP_ENABLE_CXX17_REMOVED_AUTO_PTR:
- This macro is used to re-enable std::auto_ptr in C++17.
Building libc++¶
Getting Started¶
On Mac OS 10.7 (Lion) and later, the easiest way to get this library is to install Xcode 4.2 or later. However if you want to install tip-of-trunk from here (getting the bleeding edge), read on.
The basic steps needed to build libc++ are:
Checkout LLVM:
cd where-you-want-llvm-to-live
svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
Checkout libc++:
cd where-you-want-llvm-to-live
cd llvm/projects
svn co http://llvm.org/svn/llvm-project/libcxx/trunk libcxx
Checkout libc++abi:
cd where-you-want-llvm-to-live
cd llvm/projects
svn co http://llvm.org/svn/llvm-project/libcxxabi/trunk libcxxabi
Configure and build libc++ with libc++abi:
CMake is the only supported configuration system.
Clang is the preferred compiler when building and using libc++.
cd where you want to build llvm
mkdir build
cd build
cmake -G <generator> [options] <path to llvm sources>
For more information about configuring libc++ see CMake Options.
make cxx
— will build libc++ and libc++abi.make check-cxx check-cxxabi
— will run the test suites.
Shared libraries for libc++ and libc++ abi should now be present in llvm/build/lib. See using an alternate libc++ installation
Optional: Install libc++ and libc++abi
If your system already provides a libc++ installation it is important to be careful not to replace it. Remember Use the CMake option
CMAKE_INSTALL_PREFIX
to select a safe place to install libc++.make install-cxx install-cxxabi
— Will install the libraries and the headers
Warning
- Replacing your systems libc++ installation could render the system non-functional.
- Mac OS X will not boot without a valid copy of
libc++.1.dylib
in/usr/lib
.
The instructions are for building libc++ on FreeBSD, Linux, or Mac using libc++abi as the C++ ABI library. On Linux, it is also possible to use libsupc++ or libcxxrt.
It is sometimes beneficial to build outside of the LLVM tree. An out-of-tree build would look like this:
$ cd where-you-want-libcxx-to-live
$ # Check out llvm, libc++ and libc++abi.
$ ``svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm``
$ ``svn co http://llvm.org/svn/llvm-project/libcxx/trunk libcxx``
$ ``svn co http://llvm.org/svn/llvm-project/libcxxabi/trunk libcxxabi``
$ cd where-you-want-to-build
$ mkdir build && cd build
$ export CC=clang CXX=clang++
$ cmake -DLLVM_PATH=path/to/llvm \
-DLIBCXX_CXX_ABI=libcxxabi \
-DLIBCXX_CXX_ABI_INCLUDE_PATHS=path/to/libcxxabi/include \
path/to/libcxx
$ make
$ make check-libcxx # optional
Experimental Support for Windows¶
The Windows support requires building with clang-cl as cl does not support one required extension: #include_next. Furthermore, VS 2015 or newer (19.00) is required. In the case of clang-cl, we need to specify the “MS Compatibility Version” as it defaults to 2014 (18.00).
Building with Visual Studio currently does not permit running tests. However, it is the simplest way to build.
> cmake -G "Visual Studio 14 2015" ^
-T "LLVM-vs2014" ^
-DLIBCXX_ENABLE_SHARED=YES ^
-DLIBCXX_ENABLE_STATIC=NO ^
-DLIBCXX_ENABLE_EXPERIMENTAL_LIBRARY=NO ^
\path\to\libcxx
> cmake --build .
Building with ninja is required for development to enable tests. Unfortunately, doing so requires additional configuration as we cannot just specify a toolset.
> cmake -G Ninja ^
-DCMAKE_MAKE_PROGRAM=/path/to/ninja ^
-DCMAKE_SYSTEM_NAME=Windows ^
-DCMAKE_C_COMPILER=clang-cl ^
-DCMAKE_C_FLAGS="-fms-compatibility-version=19.00 --target=i686--windows" ^
-DCMAKE_CXX_COMPILER=clang-cl ^
-DCMAKE_CXX_FLAGS="-fms-compatibility-version=19.00 --target=i686--windows" ^
-DLLVM_PATH=/path/to/llvm/tree ^
-DLIBCXX_ENABLE_SHARED=YES ^
-DLIBCXX_ENABLE_STATIC=NO ^
-DLIBCXX_ENABLE_EXPERIMENTAL_LIBRARY=NO ^
\path\to\libcxx
> /path/to/ninja cxx
> /path/to/ninja check-cxx
Note that the paths specified with backward slashes must use the \ as the directory separator as clang-cl may otherwise parse the path as an argument.
CMake Options¶
Here are some of the CMake variables that are used often, along with a
brief explanation and LLVM-specific notes. For full documentation, check the
CMake docs or execute cmake --help-variable VARIABLE_NAME
.
- CMAKE_BUILD_TYPE:STRING
- Sets the build type for
make
based generators. Possible values are Release, Debug, RelWithDebInfo and MinSizeRel. On systems like Visual Studio the user sets the build type with the IDE settings. - CMAKE_INSTALL_PREFIX:PATH
- Path where LLVM will be installed if “make install” is invoked or the “INSTALL” target is built.
- CMAKE_CXX_COMPILER:STRING
- The C++ compiler to use when building and testing libc++.
libc++ specific options¶
-
LIBCXX_INSTALL_LIBRARY:BOOL
¶
Default:
ON
Toggle the installation of the library portion of libc++.
-
LIBCXX_INSTALL_HEADERS:BOOL
¶
Default:
ON
Toggle the installation of the libc++ headers.
-
LIBCXX_ENABLE_ASSERTIONS:BOOL
¶
Default:
ON
Build libc++ with assertions enabled.
-
LIBCXX_BUILD_32_BITS:BOOL
¶
Default:
OFF
Build libc++ as a 32 bit library. Also see LLVM_BUILD_32_BITS.
Default:
ON
Build libc++ as a shared library. Either LIBCXX_ENABLE_SHARED or LIBCXX_ENABLE_STATIC has to be enabled.
-
LIBCXX_ENABLE_STATIC:BOOL
¶
Default:
ON
Build libc++ as a static library. Either LIBCXX_ENABLE_SHARED or LIBCXX_ENABLE_STATIC has to be enabled.
-
LIBCXX_LIBDIR_SUFFIX:STRING
¶
Extra suffix to append to the directory where libraries are to be installed. This option overrides LLVM_LIBDIR_SUFFIX.
-
LIBCXX_INSTALL_PREFIX:STRING
¶
Default:
""
Define libc++ destination prefix.
libc++experimental Specific Options¶
-
LIBCXX_ENABLE_EXPERIMENTAL_LIBRARY:BOOL
¶
Default:
ON
Build and test libc++experimental.a.
-
LIBCXX_INSTALL_EXPERIMENTAL_LIBRARY:BOOL
¶
Default:
LIBCXX_ENABLE_EXPERIMENTAL_LIBRARY AND LIBCXX_INSTALL_LIBRARY
Install libc++experimental.a alongside libc++.
-
LIBCXX_ENABLE_FILESYSTEM:BOOL
¶
Default:
ON
Build filesystem as a standalone library libc++fs.a.
-
LIBCXX_INSTALL_FILESYSTEM_LIBRARY:BOOL
¶
Default:
LIBCXX_ENABLE_FILESYSTEM AND LIBCXX_INSTALL_LIBRARY
Install libc++fs.a alongside libc++.
ABI Library Specific Options¶
-
LIBCXX_CXX_ABI:STRING
¶
Values:
none
,libcxxabi
,libcxxrt
,libstdc++
,libsupc++
.Select the ABI library to build libc++ against.
-
LIBCXX_CXX_ABI_INCLUDE_PATHS:PATHS
¶
Provide additional search paths for the ABI library headers.
-
LIBCXX_CXX_ABI_LIBRARY_PATH:PATH
¶
Provide the path to the ABI library that libc++ should link against.
-
LIBCXX_ENABLE_STATIC_ABI_LIBRARY:BOOL
¶
Default:
OFF
If this option is enabled, libc++ will try and link the selected ABI library statically.
-
LIBCXX_ENABLE_ABI_LINKER_SCRIPT:BOOL
¶
Default:
ON
by default on UNIX platforms other than Apple unless ‘LIBCXX_ENABLE_STATIC_ABI_LIBRARY’ is ON. Otherwise the default value isOFF
.This option generate and installs a linker script as
libc++.so
which links the correct ABI library.
-
LIBCXXABI_USE_LLVM_UNWINDER:BOOL
¶
Default:
OFF
Build and use the LLVM unwinder. Note: This option can only be used when libc++abi is the C++ ABI library used.
libc++ Feature Options¶
-
LIBCXX_ENABLE_EXCEPTIONS:BOOL
¶
Default:
ON
Build libc++ with exception support.
-
LIBCXX_ENABLE_RTTI:BOOL
¶
Default:
ON
Build libc++ with run time type information.
-
LIBCXX_INCLUDE_BENCHMARKS:BOOL
¶
Default:
ON
Build the libc++ benchmark tests and the Google Benchmark library needed to support them.
-
LIBCXX_BENCHMARK_NATIVE_STDLIB:STRING
¶
Default::
""
Values::
libc++
,libstdc++
Build the libc++ benchmark tests and Google Benchmark library against the specified standard library on the platform. On linux this can be used to compare libc++ to libstdc++ by building the benchmark tests against both standard libraries.
-
LIBCXX_BENCHMARK_NATIVE_GCC_TOOLCHAIN:STRING
¶
Use the specified GCC toolchain and standard library when building the native stdlib benchmark tests.
libc++ ABI Feature Options¶
The following options allow building libc++ for a different ABI version.
-
LIBCXX_ABI_VERSION:STRING
¶
Default:
1
Defines the target ABI version of libc++.
-
LIBCXX_ABI_UNSTABLE:BOOL
¶
Default:
OFF
Build the “unstable” ABI version of libc++. Includes all ABI changing features on top of the current stable version.
-
LIBCXX_ABI_DEFINES:STRING
¶
Default:
""
A semicolon-separated list of ABI macros to persist in the site config header. See
include/__config
for the list of ABI macros.
LLVM-specific options¶
-
LLVM_LIBDIR_SUFFIX:STRING
¶
Extra suffix to append to the directory where libraries are to be installed. On a 64-bit architecture, one could use
-DLLVM_LIBDIR_SUFFIX=64
to install libraries to/usr/lib64
.
-
LLVM_BUILD_32_BITS:BOOL
¶
Build 32-bits executables and libraries on 64-bits systems. This option is available only on some 64-bits unix systems. Defaults to OFF.
-
LLVM_LIT_ARGS:STRING
¶
Arguments given to lit.
make check
andmake clang-test
are affected. By default,'-sv --no-progress-bar'
on Visual C++ and Xcode,'-sv'
on others.
Using Alternate ABI libraries¶
Using libsupc++ on Linux¶
You will need libstdc++ in order to provide libsupc++.
Figure out where the libsupc++ headers are on your system. On Ubuntu this
is /usr/include/c++/<version>
and /usr/include/c++/<version>/<target-triple>
You can also figure this out by running
$ echo | g++ -Wp,-v -x c++ - -fsyntax-only
ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/4.7/../../../../x86_64-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/include/c++/4.7
/usr/include/c++/4.7/x86_64-linux-gnu
/usr/include/c++/4.7/backward
/usr/lib/gcc/x86_64-linux-gnu/4.7/include
/usr/local/include
/usr/lib/gcc/x86_64-linux-gnu/4.7/include-fixed
/usr/include/x86_64-linux-gnu
/usr/include
End of search list.
Note that the first two entries happen to be what we are looking for. This may not be correct on other platforms.
We can now run CMake:
$ CC=clang CXX=clang++ cmake -G "Unix Makefiles" \
-DLIBCXX_CXX_ABI=libstdc++ \
-DLIBCXX_CXX_ABI_INCLUDE_PATHS="/usr/include/c++/4.7/;/usr/include/c++/4.7/x86_64-linux-gnu/" \
-DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr \
<libc++-source-dir>
You can also substitute -DLIBCXX_CXX_ABI=libsupc++
above, which will cause the library to be linked to libsupc++ instead
of libstdc++, but this is only recommended if you know that you will
never need to link against libstdc++ in the same executable as libc++.
GCC ships libsupc++ separately but only as a static library. If a
program also needs to link against libstdc++, it will provide its
own copy of libsupc++ and this can lead to subtle problems.
$ make cxx
$ make install
You can now run clang with -stdlib=libc++.
Using libcxxrt on Linux¶
You will need to keep the source tree of libcxxrt available on your build machine and your copy of the libcxxrt shared library must be placed where your linker will find it.
We can now run CMake like:
$ CC=clang CXX=clang++ cmake -G "Unix Makefiles" \
-DLIBCXX_CXX_ABI=libcxxrt \
-DLIBCXX_CXX_ABI_INCLUDE_PATHS=path/to/libcxxrt-sources/src \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr \
<libc++-source-directory>
$ make cxx
$ make install
Unfortunately you can’t simply run clang with “-stdlib=libc++” at this point, as clang is set up to link for libc++ linked to libsupc++. To get around this you’ll have to set up your linker yourself (or patch clang). For example,
$ clang++ -stdlib=libc++ helloworld.cpp \
-nodefaultlibs -lc++ -lcxxrt -lm -lc -lgcc_s -lgcc
Alternately, you could just add libcxxrt to your libraries list, which in most situations will give the same result:
$ clang++ -stdlib=libc++ helloworld.cpp -lcxxrt
Using a local ABI library installation¶
Warning
This is not recommended in almost all cases.
These instructions should only be used when you can’t install your ABI library.
Normally you must link libc++ against a ABI shared library that the
linker can find. If you want to build and test libc++ against an ABI
library not in the linker’s path you needq to set
-DLIBCXX_CXX_ABI_LIBRARY_PATH=/path/to/abi/lib
when configuring CMake.
An example build using libc++abi would look like:
$ CC=clang CXX=clang++ cmake \
-DLIBCXX_CXX_ABI=libc++abi \
-DLIBCXX_CXX_ABI_INCLUDE_PATHS="/path/to/libcxxabi/include" \
-DLIBCXX_CXX_ABI_LIBRARY_PATH="/path/to/libcxxabi-build/lib" \
path/to/libcxx
$ make
When testing libc++ LIT will automatically link against the proper ABI library.
Testing libc++¶
Getting Started¶
libc++ uses LIT to configure and run its tests. The primary way to run the libc++ tests is by using make check-libcxx. However since libc++ can be used in any number of possible configurations it is important to customize the way LIT builds and runs the tests. This guide provides information on how to use LIT directly to test libc++.
Please see the Lit Command Guide for more information about LIT.
Setting up the Environment¶
After building libc++ you must setup your environment to test libc++ using LIT.
Create a shortcut to the actual lit executable so that you can invoke it easily from the command line.
$ alias lit='python path/to/llvm/utils/lit/lit.py'
Tell LIT where to find your build configuration.
$ export LIBCXX_SITE_CONFIG=path/to/build-libcxx/test/lit.site.cfg
Example Usage¶
Once you have your environment set up and you have built libc++ you can run parts of the libc++ test suite by simply running lit on a specified test or directory. For example:
$ cd path/to/src/libcxx
$ lit -sv test/std/re # Run all of the std::regex tests
$ lit -sv test/std/depr/depr.c.headers/stdlib_h.pass.cpp # Run a single test
$ lit -sv test/std/atomics test/std/threads # Test std::thread and std::atomic
Sometimes you’ll want to change the way LIT is running the tests. Custom options can be specified using the –param=<name>=<val> flag. The most common option you’ll want to change is the standard dialect (ie -std=c++XX). By default the test suite will select the newest C++ dialect supported by the compiler and use that. However if you want to manually specify the option like so:
$ lit -sv test/std/containers # Run the tests with the newest -std
$ lit -sv --param=std=c++03 test/std/containers # Run the tests in C++03
Occasionally you’ll want to add extra compile or link flags when testing. You can do this as follows:
$ lit -sv --param=compile_flags='-Wcustom-warning'
$ lit -sv --param=link_flags='-L/custom/library/path'
Some other common examples include:
# Specify a custom compiler.
$ lit -sv --param=cxx_under_test=/opt/bin/g++ test/std
# Enable warnings in the test suite
$ lit -sv --param=enable_warnings=true test/std
# Use UBSAN when running the tests.
$ lit -sv --param=use_sanitizer=Undefined
LIT Options¶
lit [options…] [filenames…]
Command Line Options¶
To use these options you pass them on the LIT command line as –param NAME or –param NAME=VALUE. Some options have default values specified during CMake’s configuration. Passing the option on the command line will override the default.
-
cxx_under_test
=<path/to/compiler>
¶ Specify the compiler used to build the tests.
-
cxx_stdlib_under_test
=<stdlib name>
¶ Values: libc++, libstdc++
Specify the C++ standard library being tested. Unless otherwise specified libc++ is used. This option is intended to allow running the libc++ test suite against other standard library implementations.
-
std
=<standard version>
¶ Values: c++98, c++03, c++11, c++14, c++17, c++2a
Change the standard version used when building the tests.
-
libcxx_site_config
=<path/to/lit.site.cfg>
¶ Specify the site configuration to use when running the tests. This option overrides the environment variable LIBCXX_SITE_CONFIG.
-
cxx_headers
=<path/to/headers>
¶ Specify the c++ standard library headers that are tested. By default the headers in the source tree are used.
-
cxx_library_root
=<path/to/lib/>
¶ Specify the directory of the libc++ library to be tested. By default the library folder of the build directory is used. This option cannot be used when use_system_cxx_lib is provided.
-
cxx_runtime_root
=<path/to/lib/>
¶ Specify the directory of the libc++ library to use at runtime. This directory is not added to the linkers search path. This can be used to compile tests against one version of libc++ and run them using another. The default value for this option is cxx_library_root. This option cannot be used when use_system_cxx_lib is provided.
-
use_system_cxx_lib
=<bool>
¶ Default: False
Enable or disable testing against the installed version of libc++ library. Note: This does not use the installed headers.
-
use_lit_shell
=<bool>
¶ Enable or disable the use of LIT’s internal shell in ShTests. If the environment variable LIT_USE_INTERNAL_SHELL is present then that is used as the default value. Otherwise the default value is True on Windows and False on every other platform.
-
no_default_flags
=<bool>
¶ Default: False
Disable all default compile and link flags from being added. When this option is used only flags specified using the compile_flags and link_flags will be used.
-
compile_flags
="<list-of-args>"
¶ Specify additional compile flags as a space delimited string. Note: This options should not be used to change the standard version used.
-
link_flags
="<list-of-args>"
¶ Specify additional link flags as a space delimited string.
-
debug_level
=<level>
¶ Values: 0, 1
Enable the use of debug mode. Level 0 enables assertions and level 1 enables assertions and debugging of iterator misuse.
-
use_sanitizer
=<sanitizer name>
¶ Values: Memory, MemoryWithOrigins, Address, Undefined
Run the tests using the given sanitizer. If LLVM_USE_SANITIZER was given when building libc++ then that sanitizer will be used by default.
-
color_diagnostics
¶
Enable the use of colorized compile diagnostics. If the color_diagnostics option is specified or the environment variable LIBCXX_COLOR_DIAGNOSTICS is present then color diagnostics will be enabled.
Environment Variables¶
-
LIBCXX_SITE_CONFIG=<path/to/lit.site.cfg>
¶ Specify the site configuration to use when running the tests. Also see libcxx_site_config.
-
LIBCXX_COLOR_DIAGNOSTICS
¶ If
LIBCXX_COLOR_DIAGNOSTICS
is defined then the test suite will attempt to use color diagnostic outputs from the compiler. Also see color_diagnostics.
Benchmarks¶
Libc++ contains benchmark tests separately from the test of the test suite. The benchmarks are written using the Google Benchmark library, a copy of which is stored in the libc++ repository.
For more information about using the Google Benchmark library see the official documentation.
Building Benchmarks¶
The benchmark tests are not built by default. The benchmarks can be built using
the cxx-benchmarks
target.
An example build would look like:
$ cd build
$ cmake [options] <path to libcxx sources>
$ make cxx-benchmarks
This will build all of the benchmarks under <libcxx-src>/benchmarks
to be
built against the just-built libc++. The compiled tests are output into
build/benchmarks
.
The benchmarks can also be built against the platforms native standard library
using the -DLIBCXX_BUILD_BENCHMARKS_NATIVE_STDLIB=ON
CMake option. This
is useful for comparing the performance of libc++ to other standard libraries.
The compiled benchmarks are named <test>.libcxx.out
if they test libc++ and
<test>.native.out
otherwise.
Also See:
Running Benchmarks¶
The benchmarks must be run manually by the user. Currently there is no way to run them as part of the build.
For example:
$ cd build/benchmarks
$ make cxx-benchmarks
$ ./algorithms.libcxx.out # Runs all the benchmarks
$ ./algorithms.libcxx.out --benchmark_filter=BM_Sort.* # Only runs the sort benchmarks
For more information about running benchmarks see Google Benchmark.
Current Status¶
After its initial introduction, many people have asked “why start a new library instead of contributing to an existing library?” (like Apache’s libstdcxx, GNU’s libstdc++, STLport, etc). There are many contributing reasons, but some of the major ones are:
- From years of experience (including having implemented the standard library before), we’ve learned many things about implementing the standard containers which require ABI breakage and fundamental changes to how they are implemented. For example, it is generally accepted that building std::string using the “short string optimization” instead of using Copy On Write (COW) is a superior approach for multicore machines (particularly in C++11, which has rvalue references). Breaking ABI compatibility with old versions of the library was determined to be critical to achieving the performance goals of libc++.
- Mainline libstdc++ has switched to GPL3, a license which the developers of libc++ cannot use. libstdc++ 4.2 (the last GPL2 version) could be independently extended to support C++11, but this would be a fork of the codebase (which is often seen as worse for a project than starting a new independent one). Another problem with libstdc++ is that it is tightly integrated with G++ development, tending to be tied fairly closely to the matching version of G++.
- STLport and the Apache libstdcxx library are two other popular candidates, but both lack C++11 support. Our experience (and the experience of libstdc++ developers) is that adding support for C++11 (in particular rvalue references and move-only types) requires changes to almost every class and function, essentially amounting to a rewrite. Faced with a rewrite, we decided to start from scratch and evaluate every design decision from first principles based on experience. Further, both projects are apparently abandoned: STLport 5.2.1 was released in Oct‘08, and STDCXX 4.2.1 in May‘08.
Platform and Compiler Support¶
libc++ is known to work on the following platforms, using gcc-4.2 and
clang (lack of C++11 language support disables some functionality).
Note that functionality provided by <atomic>
is only functional with clang
and GCC.
OS | Arch | Compilers | ABI Library |
---|---|---|---|
Mac OS X | i386, x86_64 | Clang, GCC | libc++abi |
FreeBSD 10+ | i386, x86_64, ARM | Clang, GCC | libcxxrt, libc++abi |
Linux | i386, x86_64 | Clang, GCC | libc++abi |
The following minimum compiler versions are strongly recommended.
- Clang 3.5 and above
- GCC 4.7 and above.
Anything older may work.
C++ Dialect Support¶
Notes and Known Issues¶
This list contains known issues with libc++
- Building libc++ with
-fno-rtti
is not supported. However linking against it with-fno-rtti
is supported. - On OS X v10.8 and older the CMake option
-DLIBCXX_LIBCPPABI_VERSION=""
must be used during configuration.
A full list of currently open libc++ bugs can be found here.
Design Documents¶
Availability Markup¶
Overview¶
Libc++ is used as a system library on macOS and iOS (amongst others). In order for users to be able to compile a binary that is intended to be deployed to an older version of the platform, clang provides the availability attribute that can be placed on declarations to describe the lifecycle of a symbol in the library.
Design¶
When a new feature is introduced that requires dylib support, a macro should be created in include/__config to mark this feature as unavailable for all the systems. For example:
// Define availability macros.
#if defined(_LIBCPP_USE_AVAILABILITY_APPLE)
#define _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS __attribute__((unavailable))
#else if defined(_LIBCPP_USE_AVAILABILITY_SOME_OTHER_VENDOR)
#define _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS __attribute__((unavailable))
#else
#define _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS
#endif
When the library is updated by the platform vendor, the markup can be updated. For example:
#define _LIBCPP_AVAILABILITY_SHARED_MUTEX \
__attribute__((availability(macosx,strict,introduced=10.12))) \
__attribute__((availability(ios,strict,introduced=10.0))) \
__attribute__((availability(tvos,strict,introduced=10.0))) \
__attribute__((availability(watchos,strict,introduced=3.0)))
In the source code, the macro can be added on a class if the full class requires type info from the library for example:
_LIBCPP_BEGIN_NAMESPACE_EXPERIMENTAL
class _LIBCPP_EXCEPTION_ABI _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS bad_optional_access
: public std::logic_error {
or on a particular symbol:
_LIBCPP_OVERRIDABLE_FUNC_VIS _LIBCPP_AVAILABILITY_SIZED_NEW_DELETE void operator delete(void* __p, std::size_t __sz) _NOEXCEPT;
Testing¶
Some parameters can be passed to lit to run the test-suite and exercising the availability.
- The platform parameter controls the deployment target. For example lit can be invoked with –param=platform=macosx10.8. Default is the current host.
- The use_system_cxx_lib parameter indicates to use another library than the just built one. Invoking lit with –param=use_system_cxx_lib=true will run the test-suite against the host system library. Alternatively a path to the directory containing a specific prebuilt libc++ can be used, for example: –param=use_system_cxx_lib=/path/to/macOS/10.8/.
- The with_availability boolean parameter enables the availability markup.
Tests can be marked as XFAIL based on multiple features made available by lit:
if either use_system_cxx_lib or with_availability is passed to lit, assuming –param=platform=macosx10.8 is passed as well the following features will be available:
- availability
- availability=x86_64
- availability=macosx
- availability=x86_64-macosx
- availability=x86_64-apple-macosx10.8
- availability=macosx10.8
This feature is used to XFAIL a test that is using a class of a method marked as unavailable and that is expected to fail if deployed on an older system.
if use_system_cxx_lib is passed to lit, the following features will also be available:
- with_system_cxx_lib
- with_system_cxx_lib=x86_64
- with_system_cxx_lib=macosx
- with_system_cxx_lib=x86_64-macosx
- with_system_cxx_lib=x86_64-apple-macosx10.8
- with_system_cxx_lib=macosx10.8
This feature is used to XFAIL a test that is not using a class of a method marked as unavailable but that is expected to fail if deployed on an older system. For example if we know that it exhibits a but in the libc on a particular system version.
if with_availability is passed to lit, the following features will also be available:
- availability_markup
- availability_markup=x86_64
- availability_markup=macosx
- availability_markup=x86_64-macosx
- availability_markup=x86_64-apple-macosx10.8
- availability_markup=macosx10.8
This feature is used to XFAIL a test that is using a class of a method marked as unavailable but that is expected to pass if deployed on an older system. For example if it is using a symbol in a statically evaluated context.
Debug Mode¶
Using Debug Mode¶
Libc++ provides a debug mode that enables assertions meant to detect incorrect
usage of the standard library. By default these assertions are disabled but
they can be enabled using the _LIBCPP_DEBUG
macro.
_LIBCPP_DEBUG Macro¶
- _LIBCPP_DEBUG:
This macro is used to enable assertions and iterator debugging checks within libc++. By default it is undefined.
Values:
0
,1
Defining
_LIBCPP_DEBUG
to0
or greater enables most of libc++’s assertions. Defining_LIBCPP_DEBUG
to1
enables “iterator debugging” which provides additional assertions about the validity of iterators used by the program.Note that this option has no effect on libc++’s ABI
- _LIBCPP_DEBUG_USE_EXCEPTIONS:
- When this macro is defined
_LIBCPP_ASSERT
failures throw__libcpp_debug_exception
instead of aborting. Additionally this macro disables exception specifications on functions containing_LIBCPP_ASSERT
checks. This allows assertion failures to correctly throw through these functions.
Handling Assertion Failures¶
When a debug assertion fails the assertion handler is called via the
std::__libcpp_debug_function
function pointer. It is possible to override
this function pointer using a different handler function. Libc++ provides two
different assertion handlers, the default handler
std::__libcpp_abort_debug_handler
which aborts the program, and
std::__libcpp_throw_debug_handler
which throws an instance of
std::__libcpp_debug_exception
. Libc++ can be changed to use the throwing
assertion handler as follows:
#define _LIBCPP_DEBUG 1
#include <string>
int main() {
std::__libcpp_debug_function = std::__libcpp_throw_debug_function;
try {
std::string::iterator bad_it;
std::string str("hello world");
str.insert(bad_it, '!'); // causes debug assertion
} catch (std::__libcpp_debug_exception const&) {
return EXIT_SUCCESS;
}
return EXIT_FAILURE;
}
Debug Mode Checks¶
Libc++’s debug mode offers two levels of checking. The first enables various precondition checks throughout libc++. The second additionally enables “iterator debugging” which checks the validity of iterators used by the program.
Basic Checks¶
These checks are enabled when _LIBCPP_DEBUG
is defined to either 0 or 1.
The following checks are enabled by _LIBCPP_DEBUG
:
- FIXME: Update this list
Iterator Debugging Checks¶
These checks are enabled when _LIBCPP_DEBUG
is defined to 1.
The following containers and STL classes support iterator debugging:
std::string
std::vector<T>
(T != bool
)std::list
std::unordered_map
std::unordered_multimap
std::unordered_set
std::unordered_multiset
The remaining containers do not currently support iterator debugging. Patches welcome.
Capturing configuration information during installation¶
The Problem¶
Currently the libc++ supports building the library with a number of different configuration options. Unfortunately all of that configuration information is lost when libc++ is installed. In order to support “persistent” configurations libc++ needs a mechanism to capture the configuration options in the INSTALLED headers.
Design Goals¶
- The solution should not INSTALL any additional headers. We don’t want an extra #include slowing everybody down.
- The solution should not unduly affect libc++ developers. The problem is limited to installed versions of libc++ and the solution should be as well.
- The solution should not modify any existing headers EXCEPT during installation. It makes developers lives harder if they have to regenerate the libc++ headers every time they are modified.
- The solution should not make any of the libc++ headers dependant on files generated by the build system. The headers should be able to compile out of the box without any modification.
- The solution should not have ANY effect on users who don’t need special configuration options. The vast majority of users will never need this so it shouldn’t cost them.
The Solution¶
When you first configure libc++ using CMake we check to see if we need to capture any options. If we haven’t been given any “persistent” options then we do NOTHING.
Otherwise we create a custom installation rule that modifies the installed __config header. The rule first generates a dummy “__config_site” header containing the required #defines. The contents of the dummy header are then prepended to the installed __config header. By manually prepending the files we avoid the cost of an extra #include and we allow the __config header to be ignorant of the extra configuration all together. An example “__config” header generated when -DLIBCXX_ENABLE_THREADS=OFF is given to CMake would look something like:
//===----------------------------------------------------------------------===//
//
// The LLVM Compiler Infrastructure
//
// This file is dual licensed under the MIT and the University of Illinois Open
// Source Licenses. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
#ifndef _LIBCPP_CONFIG_SITE
#define _LIBCPP_CONFIG_SITE
/* #undef _LIBCPP_HAS_NO_GLOBAL_FILESYSTEM_NAMESPACE */
/* #undef _LIBCPP_HAS_NO_STDIN */
/* #undef _LIBCPP_HAS_NO_STDOUT */
#define _LIBCPP_HAS_NO_THREADS
/* #undef _LIBCPP_HAS_NO_MONOTONIC_CLOCK */
/* #undef _LIBCPP_HAS_NO_THREAD_UNSAFE_C_FUNCTIONS */
#endif
// -*- C++ -*-
//===--------------------------- __config ---------------------------------===//
//
// The LLVM Compiler Infrastructure
//
// This file is dual licensed under the MIT and the University of Illinois Open
// Source Licenses. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
#ifndef _LIBCPP_CONFIG
#define _LIBCPP_CONFIG
Libc++ ABI stability¶
Libc++ aims to preserve stable ABI to avoid subtle bugs when code built to the old ABI is linked with the code build to the new ABI. At the same time, libc++ allows ABI-breaking improvements and bugfixes for the scenarios when ABI change is not a issue.
To support both cases, libc++ allows specifying the ABI version at the build time. The version is defined with a cmake option LIBCXX_ABI_VERSION. Another option LIBCXX_ABI_UNSTABLE can be used to include all present ABI breaking features. These options translate into C++ macro definitions _LIBCPP_ABI_VERSION, _LIBCPP_ABI_UNSTABLE.
Any ABI-changing feature is placed under it’s own macro, _LIBCPP_ABI_XXX, which is enabled based on the value of _LIBCPP_ABI_VERSION. _LIBCPP_ABI_UNSTABLE, if set, enables all features at once.
Symbol Visibility Macros¶
Overview¶
Libc++ uses various “visibility” macros in order to provide a stable ABI in both the library and the headers. These macros work by changing the visibility and inlining characteristics of the symbols they are applied to.
Visibility Macros¶
- _LIBCPP_HIDDEN
- Mark a symbol as hidden so it will not be exported from shared libraries.
- _LIBCPP_FUNC_VIS
- Mark a symbol as being exported by the libc++ library. This attribute must be applied to the declaration of all functions exported by the libc++ dylib.
- _LIBCPP_EXTERN_VIS
- Mark a symbol as being exported by the libc++ library. This attribute may only be applied to objects defined in the libc++ library. On Windows this macro applies dllimport/dllexport to the symbol. On all other platforms this macro has no effect.
- _LIBCPP_OVERRIDABLE_FUNC_VIS
Mark a symbol as being exported by the libc++ library, but allow it to be overridden locally. On non-Windows, this is equivalent to _LIBCPP_FUNC_VIS. This macro is applied to all operator new and operator delete overloads.
Windows Behavior: Any symbol marked dllimport cannot be overridden locally, since dllimport indicates the symbol should be bound to a separate DLL. All operator new and operator delete overloads are required to be locally overridable, and therefore must not be marked dllimport. On Windows, this macro therefore expands to __declspec(dllexport) when building the library and has an empty definition otherwise.
- _LIBCPP_HIDE_FROM_ABI
- Mark a function as not being part of the ABI of any final linked image that uses it, and also as being internal to each TU that uses that function. In other words, the address of a function marked with this attribute is not guaranteed to be the same across translation units.
- _LIBCPP_HIDE_FROM_ABI_AFTER_V1
Mark a function as being hidden from the ABI (per _LIBCPP_HIDE_FROM_ABI) when libc++ is built with an ABI version after ABI v1. This macro is used to maintain ABI compatibility for symbols that have been historically exported by libc++ in v1 of the ABI, but that we don’t want to export in the future.
This macro works as follows. When we build libc++, we either hide the symbol from the ABI (if the symbol is not part of the ABI in the version we’re building), or we leave it included. From user code (i.e. when we’re not building libc++), the macro always marks symbols as internal so that programs built using new libc++ headers stop relying on symbols that are removed from the ABI in a future version. Each time we release a new stable version of the ABI, we should create a new _LIBCPP_HIDE_FROM_ABI_AFTER_XXX macro, and we can use it to start removing symbols from the ABI after that stable version.
- _LIBCPP_TYPE_VIS
- Mark a type’s typeinfo, vtable and members as having default visibility. This attribute cannot be used on class templates.
- _LIBCPP_TEMPLATE_VIS
Mark a type’s typeinfo and vtable as having default visibility. This macro has no effect on the visibility of the type’s member functions.
GCC Behavior: GCC does not support Clang’s type_visibility(…) attribute. With GCC the visibility(…) attribute is used and member functions are affected.
Windows Behavior: DLLs do not support dllimport/export on class templates. The macro has an empty definition on this platform.
- _LIBCPP_ENUM_VIS
Mark the typeinfo of an enum as having default visibility. This attribute should be applied to all enum declarations.
Windows Behavior: DLLs do not support importing or exporting enumeration typeinfo. The macro has an empty definition on this platform.
GCC Behavior: GCC un-hides the typeinfo for enumerations by default, even if -fvisibility=hidden is specified. Additionally applying a visibility attribute to an enum class results in a warning. The macro has an empty definition with GCC.
- _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS
Mark the member functions, typeinfo, and vtable of the type named in a _LIBCPP_EXTERN_TEMPLATE declaration as being exported by the libc++ library. This attribute must be specified on all extern class template declarations.
This macro is used to override the _LIBCPP_TEMPLATE_VIS attribute specified on the primary template and to export the member functions produced by the explicit instantiation in the dylib.
GCC Behavior: GCC ignores visibility attributes applied the type in extern template declarations and applying an attribute results in a warning. However since _LIBCPP_TEMPLATE_VIS is the same as __attribute__((visibility(“default”)) the visibility is already correct. The macro has an empty definition with GCC.
Windows Behavior: extern template and dllexport are fundamentally incompatible on a class template on Windows; the former suppresses instantiation, while the latter forces it. Specifying both on the same declaration makes the class template be instantiated, which is not desirable inside headers. This macro therefore expands to dllimport outside of libc++ but nothing inside of it (rather than expanding to dllexport); instead, the explicit instantiations themselves are marked as exported. Note that this applies only to extern class templates. Extern function templates obey regular import/export semantics, and applying dllexport directly to the extern template declaration (i.e. using _LIBCPP_FUNC_VIS) is the correct thing to do for them.
- _LIBCPP_CLASS_TEMPLATE_INSTANTIATION_VIS
Mark the member functions, typeinfo, and vtable of an explicit instantiation of a class template as being exported by the libc++ library. This attribute must be specified on all class template explicit instantiations.
It is only necessary to mark the explicit instantiation itself (as opposed to the extern template declaration) as exported on Windows, as discussed above. On all other platforms, this macro has an empty definition.
- _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
Mark a symbol as hidden so it will not be exported from shared libraries. This is intended specifically for method templates of either classes marked with _LIBCPP_TYPE_VIS or classes with an extern template instantiation declaration marked with _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS.
When building libc++ with hidden visibility, we want explicit template instantiations to export members, which is consistent with existing Windows behavior. We also want classes annotated with _LIBCPP_TYPE_VIS to export their members, which is again consistent with existing Windows behavior. Both these changes are necessary for clients to be able to link against a libc++ DSO built with hidden visibility without encountering missing symbols.
An unfortunate side effect, however, is that method templates of classes either marked _LIBCPP_TYPE_VIS or with extern template instantiation declarations marked with _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS also get default visibility when instantiated. These methods are often implicitly instantiated inside other libraries which use the libc++ headers, and will therefore end up being exported from those libraries, since those implicit instantiations will receive default visibility. This is not acceptable for libraries that wish to control their visibility, and led to PR30642.
Consequently, all such problematic method templates are explicitly marked either hidden (via this macro) or inline, so that they don’t leak into client libraries. The problematic methods were found by running bad-visibility-finder against the libc++ headers after making _LIBCPP_TYPE_VIS and _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS expand to default visibility.
- _LIBCPP_EXCEPTION_ABI
- Mark the member functions, typeinfo, and vtable of the type as being exported by the libc++ library. This macro must be applied to all exception types. Exception types should be defined directly in namespace std and not the versioning namespace. This allows throwing and catching some exception types between libc++ and libstdc++.
- _LIBCPP_INTERNAL_LINKAGE
- Mark the affected entity as having internal linkage (i.e. the static keyword in C). This is only a best effort: when the internal_linkage attribute is not available, we fall back to forcing the function to be inlined, which approximates internal linkage since an externally visible symbol is never generated for that function. This is an internal macro used as an implementation detail by other visibility macros. Never mark a function or a class with this macro directly.
- _LIBCPP_ALWAYS_INLINE
- Forces inlining of the function it is applied to. For visibility purposes, this macro is used to make sure that an externally visible symbol is never generated in an object file when the internal_linkage attribute is not available. This is an internal macro used by other visibility macros, and it should not be used directly.
Threading Support API¶
Overview¶
Libc++ supports using multiple different threading models and configurations
to implement the threading parts of libc++, including <thread>
and <mutex>
.
These different models provide entirely different interfaces from each
other. To address this libc++ wraps the underlying threading API in a new and
consistent API, which it uses internally to implement threading primitives.
The <__threading_support>
header is where libc++ defines its internal
threading interface. It contains forward declarations of the internal threading
interface as well as definitions for the interface.
External Threading API and the <__external_threading>
header¶
In order to support vendors with custom threading API’s libc++ allows the entire internal threading interface to be provided by an external, vendor provided, header.
When _LIBCPP_HAS_THREAD_API_EXTERNAL
is defined the <__threading_support>
header simply forwards to the <__external_threading>
header (which must exist).
It is expected that the <__external_threading>
header provide the exact
interface normally provided by <__threading_support>
.
External Threading Library¶
libc++ can be compiled with its internal threading API delegating to an external library. Such a configuration is useful for library vendors who wish to distribute a thread-agnostic libc++ library, where the users of the library are expected to provide the implementation of the libc++ internal threading API.
On a production setting, this would be achieved through a custom
<__external_threading>
header, which declares the libc++ internal threading
API but leaves out the implementation.
The -DLIBCXX_BUILD_EXTERNAL_THREAD_LIBRARY
option allows building libc++ in
such a configuration while allowing it to be tested on a platform that supports
any of the threading systems (e.g. pthread) supported in __threading_support
header. Therefore, the main purpose of this option is to allow testing of this
particular configuration of the library without being tied to a vendor-specific
threading system. This option is only meant to be used by libc++ library
developers.
Threading Configuration Macros¶
- _LIBCPP_HAS_NO_THREADS
- This macro is defined when libc++ is built without threading support. It should not be manually defined by the user.
- _LIBCPP_HAS_THREAD_API_EXTERNAL
- This macro is defined when libc++ should use the
<__external_threading>
header to provide the internal threading API. This macro overrides_LIBCPP_HAS_THREAD_API_PTHREAD
. - _LIBCPP_HAS_THREAD_API_PTHREAD
- This macro is defined when libc++ should use POSIX threads to implement the internal threading API.
- _LIBCPP_HAS_THREAD_API_WIN32
- This macro is defined when libc++ should use Win32 threads to implement the internal threading API.
- _LIBCPP_HAS_THREAD_LIBRARY_EXTERNAL
- This macro is defined when libc++ expects the definitions of the internal
threading API to be provided by an external library. When defined
<__threading_support>
will only provide the forward declarations and typedefs for the internal threading API. - _LIBCPP_BUILDING_THREAD_LIBRARY_EXTERNAL
- This macro is used to build an external threading library using the
<__threading_support>
. Specifically it exposes the threading API definitions in<__threading_support>
as non-inline definitions meant to be compiled into a library.
File Time Type¶
Motivation¶
The filesystem library provides interfaces for getting and setting the last
write time of a file or directory. The interfaces use the file_time_type
type, which is a specialization of chrono::time_point
for the
“filesystem clock”. According to [fs.filesystem.syn]
trivial-clock is an implementation-defined type that satisfies the Cpp17TrivialClock requirements ([time.clock.req]) and that is capable of representing and measuring file time values. Implementations should ensure that the resolution and range of file_time_type reflect the operating system dependent resolution and range of file time values.
On POSIX systems, file times are represented using the timespec
struct,
which is defined as follows:
struct timespec {
time_t tv_sec;
long tv_nsec;
};
To represent the range and resolution of timespec
, we need to (A) have
nanosecond resolution, and (B) use more than 64 bits (assuming a 64 bit time_t
).
As the standard requires us to use the chrono
interface, we have to define
our own filesystem clock which specifies the period and representation of
the time points and duration it provides. It will look like this:
struct _FilesystemClock {
using period = nano;
using rep = TBD; // What is this?
using duration = chrono::duration<rep, period>;
using time_point = chrono::time_point<_FilesystemClock>;
// ... //
};
using file_time_type = _FilesystemClock::time_point;
To get nanosecond resolution, we simply define period
to be std::nano
.
But what type can we use as the arithmetic representation that is capable
of representing the range of the timespec
struct?
Problems To Consider¶
Before considering solutions, let’s consider the problems they should solve, and how important solving those problems are:
Having a Smaller Range than timespec
¶
One solution to the range problem is to simply reduce the resolution of
file_time_type
to be less than that of nanoseconds. This is what libc++’s
initial implementation of file_time_type
did; it’s also what
std::system_clock
does. As a result, it can represent time points about
292 thousand years on either side of the epoch, as opposed to only 292 years
at nanosecond resolution.
timespec
can represent time points +/- 292 billion years from the epoch
(just in case you needed a time point 200 billion years before the big bang,
and with nanosecond resolution).
To get the same range, we would need to drop our resolution to that of seconds to come close to having the same range.
This begs the question, is the range problem “really a problem”? Sane usages of file time stamps shouldn’t exceed +/- 300 years, so should we care to support it?
I believe the answer is yes. We’re not designing the filesystem time API, we’re providing glorified C++ wrappers for it. If the underlying API supports a value, then we should too. Our wrappers should not place artificial restrictions on users that are not present in the underlying filesystem.
Having a smaller range that the underlying filesystem forces the
implementation to report value_too_large
errors when it encounters a time
point that it can’t represent. This can cause the call to last_write_time
to throw in cases where the user was confident the call should succeed. (See below)
#include <filesystem>
using namespace std::filesystem;
// Set the times using the system interface.
void set_file_times(const char* path, struct timespec ts) {
timespec both_times[2];
both_times[0] = ts;
both_times[1] = ts;
int result = ::utimensat(AT_FDCWD, path, both_times, 0);
assert(result != -1);
}
// Called elsewhere to set the file time to something insane, and way
// out of the 300 year range we might expect.
void some_bad_persons_code() {
struct timespec new_times;
new_times.tv_sec = numeric_limits<time_t>::max();
new_times.tv_nsec = 0;
set_file_times("/tmp/foo", new_times); // OK, supported by most FSes
}
int main() {
path p = "/tmp/foo";
file_status st = status(p);
if (!exists(st) || !is_regular_file(st))
return 1;
if ((st.permissions() & perms::others_read) == perms::none)
return 1;
// It seems reasonable to assume this call should succeed.
file_time_type tp = last_write_time(p); // BAD! Throws value_too_large.
}
Having a Smaller Resolution than timespec
¶
As mentioned in the previous section, one way to solve the range problem
is by reducing the resolution. But matching the range of timespec
using a
64 bit representation requires limiting the resolution to seconds.
So we might ask: Do users “need” nanosecond precision? Is seconds not good enough? I limit my consideration of the point to this: Why was it not good enough for the underlying system interfaces? If it wasn’t good enough for them, then it isn’t good enough for us. Our job is to match the filesystems range and representation, not design it.
Having a Larger Range than timespec
¶
We should also consider the opposite problem of having a file_time_type
that is able to represent a larger range than timespec
. At least in
this case last_write_time
can be used to get and set all possible values
supported by the underlying filesystem; meaning last_write_time(p)
will
never throw a overflow error when retrieving a value.
However, this introduces a new problem, where users are allowed to attempt to
create a time point beyond what the filesystem can represent. Two particular
values which cause this are file_time_type::min()
and
file_time_type::max()
. As a result, the following code would throw:
void test() {
last_write_time("/tmp/foo", file_time_type::max()); // Throws
last_write_time("/tmp/foo", file_time_type::min()); // Throws.
}
Apart from cases explicitly using min
and max
, I don’t see users taking
a valid time point, adding a couple hundred billions of years in error,
and then trying to update a file’s write time to that value very often.
Compared to having a smaller range, this problem seems preferable. At least now we can represent any time point the filesystem can, so users won’t be forced to revert back to system interfaces to avoid limitations in the C++ STL.
I posit that we should only consider this concern after we have something with at least the same range and resolution of the underlying filesystem. The latter two problems are much more important to solve.
Potential Solutions And Their Complications¶
Source Code Portability Across Implementations¶
As we’ve discussed, file_time_type
needs a representation that uses more
than 64 bits. The possible solutions include using __int128_t
, emulating a
128 bit integer using a class, or potentially defining a timespec
like
arithmetic type. All three will allow us to, at minimum, match the range
and resolution, and the last one might even allow us to match them exactly.
But when considering these potential solutions we need to consider more than just the values they can represent. We need to consider the effects they will have on users and their code. For example, each of them breaks the following code in some way:
// Bug caused by an unexpected 'rep' type returned by count.
void print_time(path p) {
// __int128_t doesn't have streaming operators, and neither would our
// custom arithmetic types.
cout << last_write_time(p).time_since_epoch().count() << endl;
}
// Overflow during creation bug.
file_time_type timespec_to_file_time_type(struct timespec ts) {
// woops! chrono::seconds and chrono::nanoseconds use a 64 bit representation
// this may overflow before it's converted to a file_time_type.
auto dur = seconds(ts.tv_sec) + nanoseconds(ts.tv_nsec);
return file_time_type(dur);
}
file_time_type correct_timespec_to_file_time_type(struct timespec ts) {
// This is the correct version of the above example, where we
// avoid using the chrono typedefs as they're not sufficient.
// Can we expect users to avoid this bug?
using fs_seconds = chrono::duration<file_time_type::rep>;
using fs_nanoseconds = chrono::duration<file_time_type::rep, nano>;
auto dur = fs_seconds(ts.tv_sec) + fs_nanoseconds(tv.tv_nsec);
return file_time_type(dur);
}
// Implicit truncation during conversion bug.
intmax_t get_time_in_seconds(path p) {
using fs_seconds = duration<file_time_type::rep, ratio<1, 1> >;
auto tp = last_write_time(p);
// This works with truncation for __int128_t, but what does it do for
// our custom arithmetic types.
return duration_cast<fs_seconds>().count();
}
Each of the above examples would require a user to adjust their filesystem code to the particular eccentricities of the representation, hopefully only in such a way that the code is still portable across implementations.
At least some of the above issues are unavoidable, no matter what
representation we choose. But some representations may be quirkier than others,
and, as I’ll argue later, using an actual arithmetic type (__int128_t
)
provides the least aberrant behavior.
Chrono and timespec
Emulation.¶
One of the options we’ve considered is using something akin to timespec
to represent the file_time_type
. It only seems natural seeing as that’s
what the underlying system uses, and because it might allow us to match
the range and resolution exactly. But would it work with chrono? And could
it still act at all like a timespec
struct?
For ease of consideration, let’s consider what the implementation might look like.
struct fs_timespec_rep {
fs_timespec_rep(long long v)
: tv_sec(v / nano::den), tv_nsec(v % nano::den)
{ }
private:
time_t tv_sec;
long tv_nsec;
};
bool operator==(fs_timespec_rep, fs_timespec_rep);
fs_int128_rep operator+(fs_timespec_rep, fs_timespec_rep);
// ... arithmetic operators ... //
The first thing to notice is that we can’t construct fs_timespec_rep
like
a timespec
by passing {secs, nsecs}
. Instead we’re limited to
constructing it from a single 64 bit integer.
We also can’t allow the user to inspect the tv_sec
or tv_nsec
values
directly. A chrono::duration
represents its value as a tick period and a
number of ticks stored using rep
. The representation is unaware of the
tick period it is being used to represent, but timespec
is setup to assume
a nanosecond tick period; which is the only case where the names tv_sec
and tv_nsec
match the values they store.
When we convert a nanosecond duration to seconds, fs_timespec_rep
will
use tv_sec
to represent the number of giga seconds, and tv_nsec
the
remaining seconds. Let’s consider how this might cause a bug were users allowed
to manipulate the fields directly.
template <class Period>
timespec convert_to_timespec(duration<fs_time_rep, Period> dur) {
fs_timespec_rep rep = dur.count();
return {rep.tv_sec, rep.tv_nsec}; // Oops! Period may not be nanoseconds.
}
template <class Duration>
Duration convert_to_duration(timespec ts) {
Duration dur({ts.tv_sec, ts.tv_nsec}); // Oops! Period may not be nanoseconds.
return file_time_type(dur);
file_time_type tp = last_write_time(p);
auto dur =
}
time_t extract_seconds(file_time_type tp) {
// Converting to seconds is a silly bug, but I could see it happening.
using SecsT = chrono::duration<file_time_type::rep, ratio<1, 1>>;
auto secs = duration_cast<Secs>(tp.time_since_epoch());
// tv_sec is now representing gigaseconds.
return secs.count().tv_sec; // Oops!
}
Despite fs_timespec_rep
not being usable in any manner resembling
timespec
, it still might buy us our goal of matching its range exactly,
right?
Sort of. Chrono provides a specialization point which specifies the minimum and maximum values for a custom representation. It looks like this:
template <>
struct duration_values<fs_timespec_rep> {
static fs_timespec_rep zero();
static fs_timespec_rep min();
static fs_timespec_rep max() { // assume friendship.
fs_timespec_rep val;
val.tv_sec = numeric_limits<time_t>::max();
val.tv_nsec = nano::den - 1;
return val;
}
};
Notice that duration_values
doesn’t tell the representation what tick
period it’s actually representing. This would indeed correctly limit the range
of duration<fs_timespec_rep, nano>
to exactly that of timespec
. But
nanoseconds isn’t the only tick period it will be used to represent. For
example:
void test() {
using rep = file_time_type::rep;
using fs_nsec = duration<rep, nano>;
using fs_sec = duration<rep>;
fs_nsec nsecs(fs_seconds::max()); // Truncates
}
Though the above example may appear silly, I think it follows from the incorrect
notion that using a timespec
rep in chrono actually makes it act as if it
were an actual timespec
.
Interactions with 32 bit time_t
¶
Up until now we’ve only be considering cases where time_t
is 64 bits, but what
about 32 bit systems/builds where time_t
is 32 bits? (this is the common case
for 32 bit builds).
When time_t
is 32 bits, we can implement file_time_type
simply using 64-bit
long long
. There is no need to get either __int128_t
or timespec
emulation
involved. And nor should we, as it would suffer from the numerous complications
described by this paper.
Obviously our implementation for 32-bit builds should act as similarly to the
64-bit build as possible. Code which compiles in one, should compile in the other.
This consideration is important when choosing between __int128_t
and
emulating timespec
. The solution which provides the most uniformity with
the least eccentricity is the preferable one.
Summary¶
The file_time_type
time point is used to represent the write times for files.
Its job is to act as part of a C++ wrapper for less ideal system interfaces. The
underlying filesystem uses the timespec
struct for the same purpose.
However, the initial implementation of file_time_type
could not represent
either the range or resolution of timespec
, making it unsuitable. Fixing
this requires an implementation which uses more than 64 bits to store the
time point.
We primarily considered two solutions: Using __int128_t
and using a
arithmetic emulation of timespec
. Each has its pros and cons, and both
come with more than one complication.
The Potential Solutions¶
long long
- The Status Quo¶Pros:
- As a type
long long
plays the nicest with others:- It works with streaming operators and other library entities which support
builtin integer types, but don’t support
__int128_t
. - Its the representation used by chrono’s
nanosecond
andsecond
typedefs.
- It works with streaming operators and other library entities which support
builtin integer types, but don’t support
Cons:
- It cannot provide the same resolution as
timespec
unless we limit it to a range of +/- 300 years from the epoch. - It cannot provide the same range as
timespec
unless we limit its resolution to seconds. last_write_time
has to report an error when the time reported by the filesystem is unrepresentable.
Pros:
It is an integer type.
It makes the implementation simple and efficient.
Acts exactly like other arithmetic types.
Can be implicitly converted to a builtin integer type by the user.
This is important for doing things like:
void c_interface_using_time_t(const char* p, time_t); void foo(path p) { file_time_type tp = last_write_time(p); time_t secs = duration_cast<seconds>(tp.time_since_epoch()).count(); c_interface_using_time_t(p.c_str(), secs); }
Cons:
- It isn’t always available (but on 64 bit machines, it normally is).
- It causes
file_time_type
to have a larger range thantimespec
. - It doesn’t always act the same as other builtin integer types. For example
with
cout
orto_string
. - Allows implicit truncation to 64 bit integers.
- It can be implicitly converted to a builtin integer type by the user, truncating its value.
timespec
Emulation¶Pros:
- It has the exact same range and resolution of
timespec
when representing a nanosecond tick period. - It’s always available, unlike
__int128_t
.
Cons:
- It has a larger range when representing any period longer than a nanosecond.
- Doesn’t actually allow users to use it like a
timespec
. - The required representation of using
tv_sec
to store the giga tick count andtv_nsec
to store the remainder adds nothing over a 128 bit integer, but complicates a lot. - It isn’t a builtin integer type, and can’t be used anything like one.
- Chrono can be made to work with it, but not nicely.
- Emulating arithmetic classes come with their own host of problems regarding overload resolution (Each operator needs three SFINAE constrained versions of it in order to act like builtin integer types).
- It offers little over simply using
__int128_t
. - It acts the most differently than implementations using an actual integer type, which has a high chance of breaking source compatibility.
Selected Solution - Using __int128_t
¶
The solution I selected for libc++ is using __int128_t
when available,
and otherwise falling back to using long long
with nanosecond precision.
When __int128_t
is available, or when time_t
is 32-bits, the implementation
provides same resolution and a greater range than timespec
. Otherwise
it still provides the same resolution, but is limited to a range of +/- 300
years. This final case should be rather rare, as __int128_t
is normally available in 64-bit builds, and time_t
is normally 32-bits
during 32-bit builds.
Although falling back to long long
and nanosecond precision is less than
ideal, it also happens to be the implementation provided by both libstdc++
and MSVC. (So that makes it better, right?)
Although the timespec
emulation solution is feasible and would largely
do what we want, it comes with too many complications, potential problems
and discrepancies when compared to “normal” chrono time points and durations.
An emulation of a builtin arithmetic type using a class is never going to act exactly the same, and the difference will be felt by users. It’s not reasonable to expect them to tolerate and work around these differences. And once we commit to an ABI it will be too late to change. Committing to this seems risky.
Therefore, __int128_t
seems like the better solution.
Build Bots and Test Coverage¶
Getting Involved¶
First please review our Developer’s Policy and Getting started with LLVM.
Bug Reports
If you think you’ve found a bug in libc++, please report it using the LLVM Bugzilla. If you’re not sure, you can post a message to the cfe-dev mailing list or on IRC. Please include “libc++” in your subject.
Patches
If you want to contribute a patch to libc++, the best place for that is Phabricator. Please include [libcxx] in the subject and add cfe-commits as a subscriber. Also make sure you are subscribed to the cfe-commits mailing list.
Discussion and Questions
Send discussions and questions to the cfe-dev mailing list. Please include [libcxx] in the subject.