github.com/ajguerrer/rules_go@v0.20.3/go/crosstool.rst (about) 1 Configuring a custom C toolchain 2 ================================ 3 4 .. External links are here 5 .. _Configuring CROSSTOOL: https://docs.bazel.build/versions/master/tutorial/crosstool.html 6 .. _Understanding CROSSTOOL: https://docs.bazel.build/versions/master/crosstool-reference.html 7 .. _Yet Another CROSSTOOL Writing Tutorial: https://github.com/bazelbuild/bazel/wiki/Yet-Another-CROSSTOOL-Writing-Tutorial 8 .. _cc_library: https://docs.bazel.build/versions/master/be/c-cpp.html#cc_library 9 .. _crosstool_config.proto: https://github.com/bazelbuild/bazel/blob/master/src/main/protobuf/crosstool_config.proto 10 .. _go_binary: go/core.rst#go_binary 11 .. _go_library: go/core.rst#go_library 12 .. _toolchain: https://docs.bazel.build/versions/master/be/platform.html#toolchain 13 14 References 15 ---------- 16 17 * `Configuring CROSSTOOL`_ 18 * `Understanding CROSSTOOL`_ 19 * `Yet Another CROSSTOOL Writing Tutorial`_ 20 21 Introduction 22 ------------ 23 24 The Go toolchain sometimes needs access to a working C/C++ toolchain in order to 25 produce binaries that contain cgo code or require external linking. rules_go 26 uses whatever C/C++ toolchain Bazel is configured to use. This means 27 `go_library`_ and `cc_library`_ rules can be linked into the same binary (via 28 the ``cdeps`` attribute in Go rules). 29 30 Bazel uses a CROSSTOOL file to configure the C/C++ toolchain, plus a few build 31 rules that declare constraints, dependencies, and file groups. By default, Bazel 32 will attempt to detect the toolchain installed on the host machine. This works 33 in most cases, but it's not hermetic (developers may have completely different 34 toolchains installed), and it doesn't always work with remote execution. It also 35 doesn't work with cross-compilation. Explicit configuration is required in these 36 situations. 37 38 This documented is intended to serve as a walk-through for configuring a custom 39 C/C++ toolchain for use with rules_go. 40 41 NOTE: The Go toolchain requires gcc, clang, or something that accepts the same 42 command-line arguments and produce the same error messages. MSVC is not 43 supported. This is a limitation of the Go toolchain, not rules_go. cgo infers 44 the types of C definitions based on the text of error messages. 45 46 TODO: Change the example to use a cross-compiling toolchain. 47 48 TODO: Add instructions for building a C compiler from scratch. 49 50 TODO: Build the standard C library and binutils for use with this toolchain. 51 52 Tutorial 53 -------- 54 55 In this tutorial, we'll download a binary Clang release and install it into 56 a new workspace. This workspace can be uploaded into a new repository and 57 referenced from other Bazel workspaces. 58 59 You can find a copy of the example repository described here at 60 `https://github.com/jayconrod/bazel_cc_toolchains <https://github.com/jayconrod/bazel_cc_toolchains>`_. 61 62 Step 1: Create the repository 63 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 64 65 Create a new repository and add a WORKSPACE file to the root directory. It 66 may be empty, but it's probably a good idea to give it a name. 67 68 .. code:: bash 69 70 $ cat >WORKSPACE <<EOF 71 workspace(name = "bazel_cc_toolchains") 72 EOF 73 74 Step 2: Download a toolchain 75 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 76 77 Download or compile a C toolchain, and install it in a subdirectory of your 78 workspace. I put it in ``tools``. 79 80 Note that this toolchain has Unixy subdirectories like ``bin``, ``lib``, and 81 ``include``. 82 83 .. code:: bash 84 85 $ curl http://releases.llvm.org/7.0.0/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz | tar xJ 86 $ mv clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04 tools 87 $ ls tools 88 bin include lib libexec share 89 90 Step 3: Write a CROSSTOOL 91 ~~~~~~~~~~~~~~~~~~~~~~~~~ 92 93 We'll create a file named ``tools/CROSSTOOL``, which describes our toolchain 94 to Bazel. If you have more than one C/C++ toolchain (e.g., different tools for 95 debug and optimized builds, or different compilers for different platforms), 96 they should all be configured in the same ``CROSSTOOL`` file. 97 98 The format for this file is defined in `crosstool_config.proto`_. Specifically, 99 CROSSTOOL should contain a ``CrosstoolRelease`` message, formatted as text. 100 Each ``toolchain`` field is a ``CToolchain`` message. 101 102 Here's a short example: 103 104 .. code:: proto 105 106 major_version: "local" 107 minor_version: "" 108 109 toolchain { 110 toolchain_identifier: "clang" 111 host_system_name: "linux" 112 target_system_name: "linux" 113 target_cpu: "x86_64" 114 target_libc: "x86_64" 115 compiler: "clang" 116 abi_version: "unknown" 117 abi_libc_version: "unknown" 118 119 tool_path { name: "ar" path: "bin/llvm-ar" } 120 tool_path { name: "cpp" path: "bin/clang-cpp" } 121 tool_path { name: "dwp" path: "bin/llvm-dwp" } 122 tool_path { name: "gcc" path: "bin/clang" } 123 tool_path { name: "gcov" path: "bin/llvm-profdata" } 124 tool_path { name: "ld" path: "bin/ld.lld" } 125 tool_path { name: "nm" path: "bin/llvm-nm" } 126 tool_path { name: "objcopy" path: "bin/llvm-objcopy" } 127 tool_path { name: "objdump" path: "bin/llvm-objdump" } 128 tool_path { name: "strip" path: "bin/llvm-strip" } 129 130 compiler_flag: "-no-canonical-prefixes" 131 linker_flag: "-no-canonical-prefixes" 132 133 compiler_flag: "-v" 134 cxx_builtin_include_directory: "/usr/include" 135 } 136 137 default_toolchain { 138 cpu: "x86_64" 139 toolchain_identifier: "clang" 140 } 141 142 For a more complete example, build any ``cc_binary`` with Bazel without 143 explicitly configuring ``CROSSTOOL``, then look at the ``CROSSTOOL`` that 144 Bazel generates for the automatically detected host toolchain. This can 145 be found in ``$(bazel info 146 output_base)/external/bazel_tools/tools/cpp/CROSSTOOL``. (You have to build 147 something with the host toolchain before this will show up). 148 149 Some notes: 150 151 * ``toolchain_identifier`` is the main name for the toolchain. You'll refer to 152 it using this identifier from other messages and from build files. 153 * Most of the other fields at the top of ``toolchain`` are descriptive and 154 can have any value. 155 * ``tool_path`` fields describe the various tools Bazel may invoke. The paths 156 are relative to the directory that contains the ``CROSSTOOL`` file. 157 * ``compiler_flag`` and ``linker_flag`` are passed to the compiler and linker 158 on each invocation, respectively. 159 * ``cxx_builtin_include_directory`` is a directory with include files that 160 the compiler may read. Without this declaration, these files won't be 161 visible in the sandbox. (TODO: make this hermetic). 162 163 Step 4: Write a build file 164 ~~~~~~~~~~~~~~~~~~~~~~~~~~ 165 166 We'll create a set of targets that will link the CROSSTOOL into Bazel's 167 toolchain system. It's likely this API will change in the future. This will be 168 in ``tools/BUILD.bazel``. 169 170 First, we'll create some ``filegroups`` that we can reference from other rules. 171 172 .. code:: bzl 173 174 package(default_visibility = ["//visibility:public"]) 175 176 filegroup( 177 name = "empty", 178 srcs = [], 179 ) 180 181 filegroup( 182 name = "all", 183 srcs = glob([ 184 "bin/*", 185 "lib/**", 186 "libexec/**", 187 "share/**", 188 ]), 189 ) 190 191 Next, we'll create a ``cc_toolchain`` target that tells Bazel where to find some 192 important files. This API is undocumented and will very likely change in the 193 future. We need to create one of these for each ``toolchain`` in ``CROSSTOOL``. 194 The ``toolchain_identifier`` and ``cpu`` fields should match, and the 195 filegroups should cover the files referenced in ``CROSSTOOL``. 196 197 .. code:: bzl 198 199 cc_toolchain( 200 name = "cc-compiler-clang", 201 all_files = ":all", 202 compiler_files = ":all", 203 cpu = "x86_64", 204 dwp_files = ":empty", 205 dynamic_runtime_libs = [":empty"], 206 linker_files = ":all", 207 objcopy_files = ":empty", 208 static_runtime_libs = [":empty"], 209 strip_files = ":empty", 210 supports_param_files = 1, 211 toolchain_identifier = "clang", 212 ) 213 214 Finally, we'll create a ``cc_toolchain_suite`` target. This should reference 215 ``cc_toolchain`` targets for all the toolchains in ``CROSSTOOL``. This API is 216 also undocumented and will probably change. 217 218 .. code:: bzl 219 220 cc_toolchain_suite( 221 name = "clang-toolchain", 222 toolchains = { 223 "x86_64": ":cc-compiler-clang", 224 "x86_64|clang": ":cc-compiler-clang", 225 }, 226 ) 227 228 Step 5: Verify your toolchain works 229 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 230 231 At this point, you should be able to build a simple binary by passing a bunch 232 of extra flags to Bazel. 233 234 .. code:: bash 235 236 $ mkdir example 237 $ cat >example/hello.c <<EOF 238 #include <stdio.h> 239 240 int main() { 241 printf("Hello, world!\n"); 242 return 0; 243 } 244 EOF 245 246 $ cat >example/BUILD.bazel <<EOF 247 cc_binary( 248 name = "hello", 249 srcs = ["hello.c"], 250 ) 251 EOF 252 253 $ bazel build \ 254 --crosstool_top=//tools:clang-toolchain \ 255 --cpu=x86_64 \ 256 --compiler=clang \ 257 --host_cpu=x86_64 \ 258 -s \ 259 //example:hello 260 261 You should see an invocation of ``tools/bin/clang`` in the output. 262 263 * ``--crosstool_top`` should be the label for the ``cc_toolchain_suite`` target 264 defined earlier. 265 * ``--cpu=x86_64`` should be the ``cpu`` attribute in ``cc_toolchain`` and in 266 the ``toolchain`` message in ``CROSSTOOL``. 267 * ``--compiler=clang`` should be the ``toolchain_identifier`` attribute in 268 ``cc_toolchain`` and in the ``toolchain`` message in ``CROSSTOOL``. 269 * ``--host_cpu`` should be the same as ``--cpu``. If we were cross-compiling, 270 it would be the ``cpu`` value for the execution platform (where actions are 271 performed), not the host platform (where Bazel is invoked). 272 * ``-s`` prints commands. 273 274 Step 6: Configure a Go workspace to use the toolchain 275 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 276 277 In the ``WORSKPACE`` file for your Go project, import the 278 ``bazel_cc_toolchains`` repository. The way you do this may vary depending on 279 where you've put ``bazel_cc_toolchains``. 280 281 .. code:: bzl 282 283 load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository") 284 285 git_repository( 286 name = "bazel_cc_toolchains", 287 remote = "https://github.com/jayconrod/bazel_cc_toolchains", 288 tag = "v1.0.0", 289 ) 290 291 Create a file named ``.bazelrc`` in the root directory of your Go project 292 (or add the code below to the end if already exists). Each line comprises a 293 Bazel command (such as ``build``), an optional configuration name (``clang``) 294 and a list of flags to be passed to Bazel when that configuration is used. 295 If the configuration is omitted, the flags will be passed by default. 296 297 .. code:: bash 298 299 $ cat >>.bazelrc <<EOF 300 build:clang --crosstool_top=@bazel_cc_toolchains//tools:clang-toolchain 301 build:clang --cpu=x86_64 302 build:clang --compiler=clang 303 build:clang --host_cpu=x86_64 304 EOF 305 306 You can build with ``bazel build --config=clang ...``. 307 308 Verify the toolchain is being used by compiling a "Hello world" cgo program. 309 310 .. code:: bash 311 312 $ cat >hello.go <<EOF 313 package main 314 315 /* 316 #include <stdio.h> 317 318 void say_hello() { 319 printf("Hello, world!\n"); 320 } 321 */ 322 import "C" 323 324 func main() { 325 C.say_hello() 326 } 327 EOF 328 329 $ cat >BUILD.bazel <<EOF 330 load("@io_bazel_rules_go//go:def.bzl", "go_binary") 331 332 go_binary( 333 name = "hello", 334 srcs = ["hello.go"], 335 cgo = True, 336 ) 337 338 $ bazel build --config=clang -s //:hello 339 340 You should see clang commands in Bazel's output.