]>
Commit | Line | Data |
---|---|---|
1 | # Building `sys/unix` | |
2 | ||
3 | The sys/unix package provides access to the raw system call interface of the | |
4 | underlying operating system. See: https://godoc.org/golang.org/x/sys/unix | |
5 | ||
6 | Porting Go to a new architecture/OS combination or adding syscalls, types, or | |
7 | constants to an existing architecture/OS pair requires some manual effort; | |
8 | however, there are tools that automate much of the process. | |
9 | ||
10 | ## Build Systems | |
11 | ||
12 | There are currently two ways we generate the necessary files. We are currently | |
13 | migrating the build system to use containers so the builds are reproducible. | |
14 | This is being done on an OS-by-OS basis. Please update this documentation as | |
15 | components of the build system change. | |
16 | ||
17 | ### Old Build System (currently for `GOOS != "linux"`) | |
18 | ||
19 | The old build system generates the Go files based on the C header files | |
20 | present on your system. This means that files | |
21 | for a given GOOS/GOARCH pair must be generated on a system with that OS and | |
22 | architecture. This also means that the generated code can differ from system | |
23 | to system, based on differences in the header files. | |
24 | ||
25 | To avoid this, if you are using the old build system, only generate the Go | |
26 | files on an installation with unmodified header files. It is also important to | |
27 | keep track of which version of the OS the files were generated from (ex. | |
28 | Darwin 14 vs Darwin 15). This makes it easier to track the progress of changes | |
29 | and have each OS upgrade correspond to a single change. | |
30 | ||
31 | To build the files for your current OS and architecture, make sure GOOS and | |
32 | GOARCH are set correctly and run `mkall.sh`. This will generate the files for | |
33 | your specific system. Running `mkall.sh -n` shows the commands that will be run. | |
34 | ||
35 | Requirements: bash, go | |
36 | ||
37 | ### New Build System (currently for `GOOS == "linux"`) | |
38 | ||
39 | The new build system uses a Docker container to generate the go files directly | |
40 | from source checkouts of the kernel and various system libraries. This means | |
41 | that on any platform that supports Docker, all the files using the new build | |
42 | system can be generated at once, and generated files will not change based on | |
43 | what the person running the scripts has installed on their computer. | |
44 | ||
45 | The OS specific files for the new build system are located in the `${GOOS}` | |
46 | directory, and the build is coordinated by the `${GOOS}/mkall.go` program. When | |
47 | the kernel or system library updates, modify the Dockerfile at | |
48 | `${GOOS}/Dockerfile` to checkout the new release of the source. | |
49 | ||
50 | To build all the files under the new build system, you must be on an amd64/Linux | |
51 | system and have your GOOS and GOARCH set accordingly. Running `mkall.sh` will | |
52 | then generate all of the files for all of the GOOS/GOARCH pairs in the new build | |
53 | system. Running `mkall.sh -n` shows the commands that will be run. | |
54 | ||
55 | Requirements: bash, go, docker | |
56 | ||
57 | ## Component files | |
58 | ||
59 | This section describes the various files used in the code generation process. | |
60 | It also contains instructions on how to modify these files to add a new | |
61 | architecture/OS or to add additional syscalls, types, or constants. Note that | |
62 | if you are using the new build system, the scripts/programs cannot be called normally. | |
63 | They must be called from within the docker container. | |
64 | ||
65 | ### asm files | |
66 | ||
67 | The hand-written assembly file at `asm_${GOOS}_${GOARCH}.s` implements system | |
68 | call dispatch. There are three entry points: | |
69 | ``` | |
70 | func Syscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr) | |
71 | func Syscall6(trap, a1, a2, a3, a4, a5, a6 uintptr) (r1, r2, err uintptr) | |
72 | func RawSyscall(trap, a1, a2, a3 uintptr) (r1, r2, err uintptr) | |
73 | ``` | |
74 | The first and second are the standard ones; they differ only in how many | |
75 | arguments can be passed to the kernel. The third is for low-level use by the | |
76 | ForkExec wrapper. Unlike the first two, it does not call into the scheduler to | |
77 | let it know that a system call is running. | |
78 | ||
79 | When porting Go to an new architecture/OS, this file must be implemented for | |
80 | each GOOS/GOARCH pair. | |
81 | ||
82 | ### mksysnum | |
83 | ||
84 | Mksysnum is a Go program located at `${GOOS}/mksysnum.go` (or `mksysnum_${GOOS}.go` | |
85 | for the old system). This program takes in a list of header files containing the | |
86 | syscall number declarations and parses them to produce the corresponding list of | |
87 | Go numeric constants. See `zsysnum_${GOOS}_${GOARCH}.go` for the generated | |
88 | constants. | |
89 | ||
90 | Adding new syscall numbers is mostly done by running the build on a sufficiently | |
91 | new installation of the target OS (or updating the source checkouts for the | |
92 | new build system). However, depending on the OS, you make need to update the | |
93 | parsing in mksysnum. | |
94 | ||
95 | ### mksyscall.go | |
96 | ||
97 | The `syscall.go`, `syscall_${GOOS}.go`, `syscall_${GOOS}_${GOARCH}.go` are | |
98 | hand-written Go files which implement system calls (for unix, the specific OS, | |
99 | or the specific OS/Architecture pair respectively) that need special handling | |
100 | and list `//sys` comments giving prototypes for ones that can be generated. | |
101 | ||
102 | The mksyscall.go program takes the `//sys` and `//sysnb` comments and converts | |
103 | them into syscalls. This requires the name of the prototype in the comment to | |
104 | match a syscall number in the `zsysnum_${GOOS}_${GOARCH}.go` file. The function | |
105 | prototype can be exported (capitalized) or not. | |
106 | ||
107 | Adding a new syscall often just requires adding a new `//sys` function prototype | |
108 | with the desired arguments and a capitalized name so it is exported. However, if | |
109 | you want the interface to the syscall to be different, often one will make an | |
110 | unexported `//sys` prototype, an then write a custom wrapper in | |
111 | `syscall_${GOOS}.go`. | |
112 | ||
113 | ### types files | |
114 | ||
115 | For each OS, there is a hand-written Go file at `${GOOS}/types.go` (or | |
116 | `types_${GOOS}.go` on the old system). This file includes standard C headers and | |
117 | creates Go type aliases to the corresponding C types. The file is then fed | |
118 | through godef to get the Go compatible definitions. Finally, the generated code | |
119 | is fed though mkpost.go to format the code correctly and remove any hidden or | |
120 | private identifiers. This cleaned-up code is written to | |
121 | `ztypes_${GOOS}_${GOARCH}.go`. | |
122 | ||
123 | The hardest part about preparing this file is figuring out which headers to | |
124 | include and which symbols need to be `#define`d to get the actual data | |
125 | structures that pass through to the kernel system calls. Some C libraries | |
126 | preset alternate versions for binary compatibility and translate them on the | |
127 | way in and out of system calls, but there is almost always a `#define` that can | |
128 | get the real ones. | |
129 | See `types_darwin.go` and `linux/types.go` for examples. | |
130 | ||
131 | To add a new type, add in the necessary include statement at the top of the | |
132 | file (if it is not already there) and add in a type alias line. Note that if | |
133 | your type is significantly different on different architectures, you may need | |
134 | some `#if/#elif` macros in your include statements. | |
135 | ||
136 | ### mkerrors.sh | |
137 | ||
138 | This script is used to generate the system's various constants. This doesn't | |
139 | just include the error numbers and error strings, but also the signal numbers | |
140 | an a wide variety of miscellaneous constants. The constants come from the list | |
141 | of include files in the `includes_${uname}` variable. A regex then picks out | |
142 | the desired `#define` statements, and generates the corresponding Go constants. | |
143 | The error numbers and strings are generated from `#include <errno.h>`, and the | |
144 | signal numbers and strings are generated from `#include <signal.h>`. All of | |
145 | these constants are written to `zerrors_${GOOS}_${GOARCH}.go` via a C program, | |
146 | `_errors.c`, which prints out all the constants. | |
147 | ||
148 | To add a constant, add the header that includes it to the appropriate variable. | |
149 | Then, edit the regex (if necessary) to match the desired constant. Avoid making | |
150 | the regex too broad to avoid matching unintended constants. | |
151 | ||
152 | ||
153 | ## Generated files | |
154 | ||
155 | ### `zerror_${GOOS}_${GOARCH}.go` | |
156 | ||
157 | A file containing all of the system's generated error numbers, error strings, | |
158 | signal numbers, and constants. Generated by `mkerrors.sh` (see above). | |
159 | ||
160 | ### `zsyscall_${GOOS}_${GOARCH}.go` | |
161 | ||
162 | A file containing all the generated syscalls for a specific GOOS and GOARCH. | |
163 | Generated by `mksyscall.go` (see above). | |
164 | ||
165 | ### `zsysnum_${GOOS}_${GOARCH}.go` | |
166 | ||
167 | A list of numeric constants for all the syscall number of the specific GOOS | |
168 | and GOARCH. Generated by mksysnum (see above). | |
169 | ||
170 | ### `ztypes_${GOOS}_${GOARCH}.go` | |
171 | ||
172 | A file containing Go types for passing into (or returning from) syscalls. | |
173 | Generated by godefs and the types file (see above). |