golang.org/x/sys@v0.20.1-0.20240517151509-673e0f94c16d/unix/README.md (about)

     1  # Building `sys/unix`
     2  
     3  The sys/unix package provides access to the raw system call interface of the
     4  underlying operating system. See: https://godoc.org/golang.org/x/sys/unix
     5  
     6  Porting Go to a new architecture/OS combination or adding syscalls, types, or
     7  constants to an existing architecture/OS pair requires some manual effort;
     8  however, there are tools that automate much of the process.
     9  
    10  ## Build Systems
    11  
    12  There are currently two ways we generate the necessary files. We are currently
    13  migrating the build system to use containers so the builds are reproducible.
    14  This is being done on an OS-by-OS basis. Please update this documentation as
    15  components of the build system change.
    16  
    17  ### Old Build System (currently for `GOOS != "linux"`)
    18  
    19  The old build system generates the Go files based on the C header files
    20  present on your system. This means that files
    21  for a given GOOS/GOARCH pair must be generated on a system with that OS and
    22  architecture. This also means that the generated code can differ from system
    23  to system, based on differences in the header files.
    24  
    25  To avoid this, if you are using the old build system, only generate the Go
    26  files on an installation with unmodified header files. It is also important to
    27  keep track of which version of the OS the files were generated from (ex.
    28  Darwin 14 vs Darwin 15). This makes it easier to track the progress of changes
    29  and have each OS upgrade correspond to a single change.
    30  
    31  To build the files for your current OS and architecture, make sure GOOS and
    32  GOARCH are set correctly and run `mkall.sh`. This will generate the files for
    33  your specific system. Running `mkall.sh -n` shows the commands that will be run.
    34  
    35  Requirements: bash, go
    36  
    37  ### New Build System (currently for `GOOS == "linux"`)
    38  
    39  The new build system uses a Docker container to generate the go files directly
    40  from source checkouts of the kernel and various system libraries. This means
    41  that on any platform that supports Docker, all the files using the new build
    42  system can be generated at once, and generated files will not change based on
    43  what the person running the scripts has installed on their computer.
    44  
    45  The OS specific files for the new build system are located in the `${GOOS}`
    46  directory, and the build is coordinated by the `${GOOS}/mkall.go` program. When
    47  the kernel or system library updates, modify the Dockerfile at
    48  `${GOOS}/Dockerfile` to checkout the new release of the source.
    49  
    50  To build all the files under the new build system, you must be on an amd64/Linux
    51  system and have your GOOS and GOARCH set accordingly. Running `mkall.sh` will
    52  then generate all of the files for all of the GOOS/GOARCH pairs in the new build
    53  system. Running `mkall.sh -n` shows the commands that will be run.
    54  
    55  Requirements: bash, go, docker
    56  
    57  ## Component files
    58  
    59  This section describes the various files used in the code generation process.
    60  It also contains instructions on how to modify these files to add a new
    61  architecture/OS or to add additional syscalls, types, or constants. Note that
    62  if you are using the new build system, the scripts/programs cannot be called normally.
    63  They must be called from within the docker container.
    64  
    65  ### asm files
    66  
    67  The hand-written assembly file at `asm_${GOOS}_${GOARCH}.s` implements system
    68  call dispatch. There are three entry points:
    69  ```
    70    func Syscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr)
    71    func Syscall6(trap, a1, a2, a3, a4, a5, a6 uintptr) (r1, r2, err uintptr)
    72    func RawSyscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr)
    73  ```
    74  The first and second are the standard ones; they differ only in how many
    75  arguments can be passed to the kernel. The third is for low-level use by the
    76  ForkExec wrapper. Unlike the first two, it does not call into the scheduler to
    77  let it know that a system call is running.
    78  
    79  When porting Go to a new architecture/OS, this file must be implemented for
    80  each GOOS/GOARCH pair.
    81  
    82  ### mksysnum
    83  
    84  Mksysnum is a Go program located at `${GOOS}/mksysnum.go` (or `mksysnum_${GOOS}.go`
    85  for the old system). This program takes in a list of header files containing the
    86  syscall number declarations and parses them to produce the corresponding list of
    87  Go numeric constants. See `zsysnum_${GOOS}_${GOARCH}.go` for the generated
    88  constants.
    89  
    90  Adding new syscall numbers is mostly done by running the build on a sufficiently
    91  new installation of the target OS (or updating the source checkouts for the
    92  new build system). However, depending on the OS, you may need to update the
    93  parsing in mksysnum.
    94  
    95  ### mksyscall.go
    96  
    97  The `syscall.go`, `syscall_${GOOS}.go`, `syscall_${GOOS}_${GOARCH}.go` are
    98  hand-written Go files which implement system calls (for unix, the specific OS,
    99  or the specific OS/Architecture pair respectively) that need special handling
   100  and list `//sys` comments giving prototypes for ones that can be generated.
   101  
   102  The mksyscall.go program takes the `//sys` and `//sysnb` comments and converts
   103  them into syscalls. This requires the name of the prototype in the comment to
   104  match a syscall number in the `zsysnum_${GOOS}_${GOARCH}.go` file. The function
   105  prototype can be exported (capitalized) or not.
   106  
   107  Adding a new syscall often just requires adding a new `//sys` function prototype
   108  with the desired arguments and a capitalized name so it is exported. However, if
   109  you want the interface to the syscall to be different, often one will make an
   110  unexported `//sys` prototype, and then write a custom wrapper in
   111  `syscall_${GOOS}.go`.
   112  
   113  ### types files
   114  
   115  For each OS, there is a hand-written Go file at `${GOOS}/types.go` (or
   116  `types_${GOOS}.go` on the old system). This file includes standard C headers and
   117  creates Go type aliases to the corresponding C types. The file is then fed
   118  through godef to get the Go compatible definitions. Finally, the generated code
   119  is fed though mkpost.go to format the code correctly and remove any hidden or
   120  private identifiers. This cleaned-up code is written to
   121  `ztypes_${GOOS}_${GOARCH}.go`.
   122  
   123  The hardest part about preparing this file is figuring out which headers to
   124  include and which symbols need to be `#define`d to get the actual data
   125  structures that pass through to the kernel system calls. Some C libraries
   126  preset alternate versions for binary compatibility and translate them on the
   127  way in and out of system calls, but there is almost always a `#define` that can
   128  get the real ones.
   129  See `types_darwin.go` and `linux/types.go` for examples.
   130  
   131  To add a new type, add in the necessary include statement at the top of the
   132  file (if it is not already there) and add in a type alias line. Note that if
   133  your type is significantly different on different architectures, you may need
   134  some `#if/#elif` macros in your include statements.
   135  
   136  ### mkerrors.sh
   137  
   138  This script is used to generate the system's various constants. This doesn't
   139  just include the error numbers and error strings, but also the signal numbers
   140  and a wide variety of miscellaneous constants. The constants come from the list
   141  of include files in the `includes_${uname}` variable. A regex then picks out
   142  the desired `#define` statements, and generates the corresponding Go constants.
   143  The error numbers and strings are generated from `#include <errno.h>`, and the
   144  signal numbers and strings are generated from `#include <signal.h>`. All of
   145  these constants are written to `zerrors_${GOOS}_${GOARCH}.go` via a C program,
   146  `_errors.c`, which prints out all the constants.
   147  
   148  To add a constant, add the header that includes it to the appropriate variable.
   149  Then, edit the regex (if necessary) to match the desired constant. Avoid making
   150  the regex too broad to avoid matching unintended constants.
   151  
   152  ### internal/mkmerge
   153  
   154  This program is used to extract duplicate const, func, and type declarations
   155  from the generated architecture-specific files listed below, and merge these
   156  into a common file for each OS.
   157  
   158  The merge is performed in the following steps:
   159  1. Construct the set of common code that is idential in all architecture-specific files.
   160  2. Write this common code to the merged file.
   161  3. Remove the common code from all architecture-specific files.
   162  
   163  
   164  ## Generated files
   165  
   166  ### `zerrors_${GOOS}_${GOARCH}.go`
   167  
   168  A file containing all of the system's generated error numbers, error strings,
   169  signal numbers, and constants. Generated by `mkerrors.sh` (see above).
   170  
   171  ### `zsyscall_${GOOS}_${GOARCH}.go`
   172  
   173  A file containing all the generated syscalls for a specific GOOS and GOARCH.
   174  Generated by `mksyscall.go` (see above).
   175  
   176  ### `zsysnum_${GOOS}_${GOARCH}.go`
   177  
   178  A list of numeric constants for all the syscall number of the specific GOOS
   179  and GOARCH. Generated by mksysnum (see above).
   180  
   181  ### `ztypes_${GOOS}_${GOARCH}.go`
   182  
   183  A file containing Go types for passing into (or returning from) syscalls.
   184  Generated by godefs and the types file (see above).