| Age | Commit message (Collapse) | Author |
|---|
|
mv cmd/new5l cmd/5l and so on.
Minimal changes to cmd/dist and cmd/go to keep things building.
More can be deleted in followup CLs.
Change-Id: I1449eca7654ce2580d1f413a56dc4a75f3d4618b
Reviewed-on: https://go-review.googlesource.com/6361
Reviewed-by: Rob Pike <r@golang.org>
|
|
Change-Id: Iaab2be9a1919f2fa9dbc61a5b7fbf99bcd0712a9
Reviewed-on: https://go-review.googlesource.com/6332
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
|
|
Make cmd/ld a real library invoked by the individual linkers.
There are no reverse symbol references anymore
(symbols referred to in cmd/ld but defined in cmd/5l etc).
This means that in principle we could do an automatic
conversion of these to Go, as a stopgap until cmd/link is done
or as a replacement for cmd/link.
Change-Id: I4a94570257a3a7acc31601bfe0fad9dea0aea054
Reviewed-on: https://go-review.googlesource.com/4649
Reviewed-by: Rob Pike <r@golang.org>
|
|
linklookup uses hash(name, v) as the hash table index but then
only compares name to find a symbol to return.
If hash(name, v1) == hash(name, v2) for v1 != v2, the lookup
for v2 will return the symbol with v1.
The input routines assume that each symbol is found only once,
and then each symbol is added to a linked list, with the list header
in the symbol. Adding a symbol to such a list multiple times
short-circuits the list the second time it is added, causing symbols
to be dropped.
The liblink rewrite introduced an elegant, if inefficient, handling
of duplicated symbols by creating a dummy symbol to read the
duplicate into. The dummy symbols are named .dup with
sequential version numbers. With many .dup symbols, eventually
there will be a conflict, causing a duplicate list add, causing elided
symbols, causing a crash when calling one of the elided symbols.
The bug is old (2011) but could not have manifested until the
liblink rewrite introduced this heavily duplicated symbol .dup.
(See History section below.)
1. Correct the lookup function.
2. Since we want all the .dup symbols to be different, there's no
point in inserting them into the table. Call linknewsym directly,
avoiding the lookup function entirely.
3. Since nothing can refer to the .dup symbols, do not bother
adding them to the list of functions (textp) at all.
4. In lieu of a unit test, introduce additional consistency checks to
detect adding a symbol to a list multiple times. This would have
caught the short-circuit more directly, and it will detect a variety
of double-use bugs, including the one arising from the bad lookup.
Fixes #7749.
History
On April 9, 2011, I submitted CL 4383047, making ld 25% faster.
Much of the focus was on the hash table lookup function, and
one of the changes was to remove the s->version == v comparison [1].
I don't know if this was a simple editing error or if I reasoned that
same name but different v would yield a different hash slot and
so the name test alone sufficed. It is tempting to claim the former,
but it was probably the latter.
Because the hash is an iterated multiply+add, the version ends up
adding v*3ⁿ to the hash, where n is the length of the name.
A collision would need x*3ⁿ ≡ y*3ⁿ (mod 2²⁴ mod 100003),
or equivalently x*3ⁿ ≡ x*3ⁿ + (y-x)*3ⁿ (mod 2²⁴ mod 100003),
so collisions will actually be periodic: versions x and y collide
when d = y-x satisfies d*3ⁿ ≡ 0 (mod 2²⁴ mod 100003).
Since we allocate version numbers sequentially, this is actually
about the best case one could imagine: the collision rate is
much lower than if the hash were more random.
http://play.golang.org/p/TScD41c_hA computes the collision
period for various name lengths.
The most common symbol in the new linker is .dup, and for n=4
the period is maximized: the 100004th symbol is the first collision.
Unfortunately, there are programs with more duplicated symbols
than that.
In Go 1.2 and before, duplicate symbols were handled without
creating a dummy symbol, so this particular case for generating
many duplicate symbols could not happen. Go does not use
versioned symbols. Only C does; each input file gives a different
version to its static declarations. There just aren't enough C files
for this to come up in that context.
So the bug is old but the realization of the bug is new.
[1] https://golang.org/cl/4383047/diff/5001/src/cmd/ld/lib.c
LGTM=minux.ma, iant, dave
R=golang-codereviews, minux.ma, bradfitz, iant, dave
CC=golang-codereviews, r
https://golang.org/cl/87910047
|
|
The -pack flag causes 5g, 6g, 8g to write a Go archive directly,
instead of requiring the use of 'go tool pack' to convert the .5/.6/.8
to .a format.
Writing directly avoids the copy and also avoids having the
export data stored twice in the archive (once in __.PKGDEF,
once in .5/.6/.8).
A separate CL will enable the use of this flag by cmd/go.
Other build systems that do not know about -pack will be unaffected.
The changes to cmd/ld handle a minor simplification to the format:
an unused section is removed.
R=iant, r
CC=golang-dev
https://golang.org/cl/42880043
|
|
- new object file reader/writer (liblink/objfile.c)
- remove old object file writing routines
- add pcdata iterator
- remove all trace of "line number stack" and "path fragments" from
object files, linker (!!!)
- dwarf now writes a single "compilation unit" instead of one per package
This CL disables the check for chains of no-split functions that
could overflow the stack red zone. A future CL will attack the problem
of reenabling that check (issue 6931).
This CL is just the liblink and cmd/ld changes.
There are minor associated adjustments in CL 37030045.
Each depends on the other.
R=golang-dev, dave, iant
CC=golang-dev
https://golang.org/cl/39680043
|
|
There is an enormous amount of code moving around in this CL,
but the code is the same, and it is invoked in the same ways.
This CL is preparation for the new linker structure, not the new
structure itself.
The new library's definition is in include/link.h.
The main change is the use of a Link structure to hold all the
linker-relevant state, replacing the smattering of global variables.
The Link structure should both make it clearer which state must
be carried around and make it possible to parallelize more easily
later.
The main body of the linker has moved into the architecture-independent
cmd/ld directory. That includes the list of known header types, so the
distinction between Hplan9x32 and Hplan9x64 is removed (no other
header type distinguished 32- and 64-bit formats), and code for unused
formats such as ipaq kernels has been deleted.
The code being deleted from 5l, 6l, and 8l reappears in liblink or in ld.
Because multiple files are being merged in the liblink directory,
it is not possible to show the diffs nicely in hg.
The Prog and Addr structures have been unified into an
architecture-independent form and moved to link.h, where they will
be shared by all tools: the assemblers, the compilers, and the linkers.
The unification makes it possible to write architecture-independent
traversal of Prog lists, among other benefits.
The Sym structures cannot be unified: they are too fundamentally
different between the linker and the compilers. Instead, liblink defines
an LSym - a linker Sym - to be used in the Prog and Addr structures,
and the linker now refers exclusively to LSyms. The compilers will
keep using their own syms but will fill out the corresponding LSyms in
the Prog and Addr structures.
Although code from 5l, 6l, and 8l is now in a single library, the
code has been arranged so that only one architecture needs to
be linked into a particular program: 5l will not contain the code
needed for x86 instruction layout, for example.
The object file writing code in liblink/obj.c is from cmd/gc/obj.c.
Preparation for golang.org/s/go13linker work.
This CL does not build by itself. It depends on 35740044
and will be submitted at the same time.
R=iant
CC=golang-dev
https://golang.org/cl/35790044
|
|
If you thought gcc -ansi -pedantic was pedantic, just wait
until you meet clang -fsanitize=undefined.
I think this addresses all the reported "errors", but we'll
need another run to be sure.
all.bash still passes.
Update #5764
Dave, can you please try again?
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/13334049
|
|
external linking
This CL is an aggregate of 10271047, 10499043, 9733044. Descriptions of each follow:
10499043
runtime,cmd/ld: Merge TLS symbols and teach 5l about ARM TLS
This CL prepares for external linking support to ARM.
The pseudo-symbols runtime.g and runtime.m are merged into a single
runtime.tlsgm symbol. When external linking, the offset of a thread local
variable is stored at a memory location instead of being embedded into a offset
of a ldr instruction. With a single runtime.tlsgm symbol for both g and m, only
one such offset is needed.
The larger part of this CL moves TLS code from gcc compiled to internally
compiled. The TLS code now uses the modern MRC instruction, and 5l is taught
about TLS fallbacks in case the instruction is not available or appropriate.
10271047
This CL adds support for -linkmode external to 5l.
For 5l itself, use addrel to allow for D_CALL relocations to be handled by the
host linker. Of the cases listed in rsc's comment in issue 4069, only case 5 and
63 needed an update. One of the TODO: addrel cases was since replaced, and the
rest of the cases are either covered by indirection through addpool (cases with
LTO or LFROM flags) or stubs (case 74). The addpool cases are covered because
addpool emits AWORD instructions, which in turn are handled by case 11.
In the runtime, change the argv argument in the rt0* functions slightly to be a
pointer to the argv list, instead of relying on a particular location of argv.
9733044
The -shared flag to 6l outputs a shared library, implemented in Go
and callable from non-Go programs such as C.
The main part of this CL change the thread local storage model.
Go uses the fastest and least general mode, local exec. TLS data in shared
libraries normally requires at least the local dynamic mode, however, this CL
instead opts for using the initial exec mode. Initial exec mode is faster than
local dynamic mode and can be used in linux since the linker has reserved a
limited amount of TLS space for performance sensitive TLS code.
Initial exec mode requires an extra load from the GOT table to determine the
TLS offset. This penalty will not be paid if ld is not in -shared mode, since
TLS accesses will be reduced to local exec.
The elf sections .init_array and .rela.init_array are added to register the Go
runtime entry with cgo at library load time.
The "hidden" attribute is added to Cgo functions called from Go, since Go
does not generate call through the GOT table, and adding non-GOT relocations for
a global function is not supported by gcc. Cgo symbols don't need to be global
and avoiding the GOT table is also faster.
The changes to 8l are only removes code relevant to the old -shared mode where
internal linking was used.
This CL only address the low level linker work. It can be submitted by itself,
but to be useful, the runtime changes in CL 9738047 is also needed.
Design discussion at
https://groups.google.com/forum/?fromgroups#!topic/golang-nuts/zmjXkGrEx6Q
Fixes #5590.
R=rsc
CC=golang-dev
https://golang.org/cl/12871044
|
|
Design at http://golang.org/s/go12symtab.
This enables some cleanup of the garbage collector metadata
that will be done in future CLs.
This CL does not move the old symtab and pclntab back into
an unmapped section of the file. That's a bit tricky and will be
done separately.
Fixes #4020.
R=golang-dev, dave, cshapiro, iant, r
CC=golang-dev, nigeltao
https://golang.org/cl/11085043
|
|
This is needed for SWIG when linking in internal mode. In
internal mode if a symbol was cgo_import_static we used to
forget that it was also cgo_import_dynamic.
R=rsc, r
CC=golang-dev
https://golang.org/cl/9080043
|
|
Fixes SWIG callbacks. Previously crosscall2 was only
cgo_export_static, despite the use of two #pragma declarations
in runtime/cgo/callbacks.c.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/7817048
|
|
Still disabled. Need to fix TLS.
R=golang-dev, minux.ma, bradfitz
CC=golang-dev
https://golang.org/cl/7783044
|
|
Fixes #4878.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7420052
|
|
R=ken2
CC=golang-dev
https://golang.org/cl/7626045
|
|
Dynimpname was getting too confusing.
Replace flag-like checks with tests of s->type.
R=ken2
CC=golang-dev
https://golang.org/cl/7594046
|
|
This makes binaries work with OS X nm.
R=ken2
CC=golang-dev
https://golang.org/cl/7558044
|
|
Still to do: non-linux and non-amd64.
It may work on other ELF-based amd64 systems too, but untested.
"go test -ldflags -hostobj $GOROOT/misc/cgo/test" passes.
Much may yet change, but this seems a reasonable checkpoint.
R=iant
CC=golang-dev
https://golang.org/cl/7369057
|
|
Also emit cgo_ldflag pragmas.
R=golang-dev, remyoudompheng, iant
CC=golang-dev
https://golang.org/cl/7530043
|
|
Switch to new pragma names, but leave old ones available for now.
Merge the three cgo-related sections in the .6 files into a single
cgo section.
R=golang-dev, iant, ality
CC=golang-dev
https://golang.org/cl/7424048
|
|
Added the -shared flag to 5l/6l to output a PIC executable with the required
dynamic relocations and RIP-relative addressing in machine code.
Added dummy support to 8l to avoid compilation errors
See also:
https://golang.org/cl/6822078
https://golang.org/cl/7064048
and
https://groups.google.com/d/topic/golang-nuts/P05BDjLcQ5k/discussion
R=rsc, iant
CC=golang-dev
https://golang.org/cl/6926049
|
|
To allow for stdcall decorated names on Windows, two changes were needed:
1. Change the symbol versioning delimiter '@' in cgo's dynimport output to a '#', and in cmd/ld when it parses dynimports.
2. Remove the "@N" decorator from the first argument of cgo's dynimport output (PE only).
Fixes #4607.
R=minux.ma, adg, rsc
CC=golang-dev
https://golang.org/cl/7047043
|
|
More cleanup in preparation for fixing issue 4069.
This CL replaces the three nearly identical copies of the
asmb ELF code with a single asmbelf function in elf.c.
In addition to the ELF code movement, remove the elfstr
array in favor of a simpler lookup, and identify sections by
name throughout instead of computing fragile indices.
The CL also replaces the three nearly identical copies of the
genasmsym code with a single genasmsym function in lib.c.
The ARM linker still compiles and generates binaries,
but I haven't tested the binaries. They may not work.
R=ken2
CC=golang-dev
https://golang.org/cl/7062047
|
|
In order to add these, we need to be able to find references
to such types that already exist in the binary. To do that, introduce
a new linker section holding a list of the types corresponding to
arrays, chans, maps, and slices.
To offset the storage cost of this list, and to simplify the code,
remove the interface{} header from the representation of a
runtime type. It was used in early versions of the code but was
made obsolete by the kind field: a switch on kind is more efficient
than a type switch.
In the godoc binary, removing the interface{} header cuts two
words from each of about 10,000 types. Adding back the list of pointers
to array, chan, map, and slice types reintroduces one word for
each of about 500 types. On a 64-bit machine, then, this CL *removes*
a net 156 kB of read-only data from the binary.
This CL does not include the needed support for precise garbage
collection. I have created issue 4375 to track that.
This CL also does not set the 'algorithm' - specifically the equality
and copy functions - for a new array correctly, so I have unexported
ArrayOf for now. That is also part of issue 4375.
Fixes #2339.
R=r, remyoudompheng, mirtchovski, iant
CC=golang-dev
https://golang.org/cl/6572043
|
|
This is an experiment in static analysis of Go programs
to understand which struct fields a program might use.
It is not part of the Go language specification, it must
be enabled explicitly when building the toolchain,
and it may be removed at any time.
After building the toolchain with GOEXPERIMENT=fieldtrack,
a specific field can be marked for tracking by including
`go:"track"` in the field tag:
package pkg
type T struct {
F int `go:"track"`
G int // untracked
}
To simplify usage, only named struct types can have
tracked fields, and only exported fields can be tracked.
The implementation works by making each function begin
with a sequence of no-op USEFIELD instructions declaring
which tracked fields are accessed by a specific function.
After the linker's dead code elimination removes unused
functions, the fields referred to by the remaining
USEFIELD instructions are the ones reported as used by
the binary.
The -k option to the linker specifies the fully qualified
symbol name (such as my/pkg.list) of a string variable that
should be initialized with the field tracking information
for the program. The field tracking string is a sequence
of lines, each terminated by a \n and describing a single
tracked field referred to by the program. Each line is made
up of one or more tab-separated fields. The first field is
the name of the tracked field, fully qualified, as in
"my/pkg.T.F". Subsequent fields give a shortest path of
reverse references from that field to a global variable or
function, corresponding to one way in which the program
might reach that field.
A common source of false positives in field tracking is
types with large method sets, because a reference to the
type descriptor carries with it references to all methods.
To address this problem, the CL also introduces a comment
annotation
//go:nointerface
that marks an upcoming method declaration as unavailable
for use in satisfying interfaces, both statically and
dynamically. Such a method is also invisible to package
reflect.
Again, all of this is disabled by default. It only turns on
if you have GOEXPERIMENT=fieldtrack set during make.bash.
R=iant, ken
CC=golang-dev
https://golang.org/cl/6749064
|
|
Also defend our symbol prefixes (now just "go" and "type")
from use as import paths.
Fixes #4257.
R=ken2
CC=golang-dev
https://golang.org/cl/6744072
|
|
Also corrected cmd/8l's .dynsym handling (differentiate between exported symbols and imported symbols)
Fixes #4029.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6620075
|
|
R=dave, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/6501134
|
|
Some newer Linux distributions (Ubuntu ARM at least) use a new multiarch
directory organization, where dynamic linker is no longer in the hardcoded
path in our linker.
For example, Ubuntu 12.04 ARM hardfloat places its dynamic linker at
/lib/arm-linux-gnueabihf/ld-linux.so.3
Ref: http://lackof.org/taggart/hacking/multiarch/
Also, to support Debian GNU/kFreeBSD as a FreeBSD variant, we need this capability, so it's part of issue 3533.
This CL add a new pragma (#pragma dynlinker "path") to cc.
R=iant, rsc
CC=golang-dev
https://golang.org/cl/6086043
|
|
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5569085
|
|
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5434068
|
|
This makes {5,6,8}l conform to the spec more tightly.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4826063
|
|
R=r, iant
CC=golang-dev
https://golang.org/cl/4809050
|
|
Fixes #2052.
R=r
CC=golang-dev
https://golang.org/cl/4812053
|
|
These changes are not particularly invasive and have been tested
as broadly as possible.
8l/l.h:
- #pragma varargck: added some, removed duplicates.
ld/dwarf.c:
- As Plan 9 has no NULL, changed all occurrences to nil.
- Added USED(size); where necessary.
- Added (void) argument in definition of finddebugruntimepath().
- Plan 9 compiler was complaining about multiple
assignments, repeaired by breaking up the commands.
- Correction: havedynamic = 1; restored.
ld/go.c:
- Needed USED(file); in two functions.
- Removed unused assignments flagged by the Plan 9 compiler.
ld/lib.c:
- Replaced unlink() with remove() which seems available everywhere.
- Removed USED(c4); and USED(magic) no longer required.
- Removed code flagged as unused by the Plan 9 compiler.
- Added attributes to a number of format strings.
R=rsc
CC=golang-dev
https://golang.org/cl/4435047
|
|
$ file $GOROOT/bin/{godoc,goyacc}
/home/rsc/g/go/bin/godoc: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), not strpped
/home/rsc/g/go/bin/goyacc: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
$
Fixes #1758.
R=iant
CC=golang-dev
https://golang.org/cl/4428079
|
|
Fixes #1397.
R=iant
CC=golang-dev
https://golang.org/cl/4444064
|
|
R=r
CC=golang-dev
https://golang.org/cl/4430064
|
|
The ld time was dominated by symbol table processing, so
* increase hash table size
* emit fewer symbols in gc (just 1 per string, 1 per type)
* add read-only lookup to avoid creating spurious symbols
* add linked list to speed whole-table traversals
Breaks dwarf generator (no idea why), so disable dwarf.
Reduces time for 6l to link godoc by 25%.
R=ken2
CC=golang-dev
https://golang.org/cl/4383047
|
|
R=rsc
CC=golang-dev
https://golang.org/cl/4273092
|
|
Much of the bulk of Go binaries is the symbol tables,
which give a name to every C string, Go string,
and reflection type symbol. These names are not worth
much other than seeing what's where in a binary.
This CL deletes all those names from the symbol table,
instead aggregating the symbols into contiguous blocks
and giving them the names "string.*", "go.string.*", and "type.*".
Before:
$ 6nm $(which godoc.old) | sort | grep ' string\.' | tail -10
59eda4 D string."aa87ca22be8b05378eb1c71...
59ee08 D string."b3312fa7e23ee7e4988e056...
59ee6c D string."func(*token.FileSet, st...
59eed0 D string."func(io.Writer, []uint8...
59ef34 D string."func(*tls.Config, *tls....
59ef98 D string."func(*bool, **template....
59effc D string."method(p *printer.print...
59f060 D string."method(S *scanner.Scann...
59f12c D string."func(*struct { begin in...
59f194 D string."method(ka *tls.ecdheRSA...
$
After:
$ 6nm $(which godoc) | sort | grep ' string\.' | tail -10
5e6a30 D string.*
$
Those names in the "Before" are truncated for the CL.
In the real binary they are the complete string, up to
a certain length, or else a unique identifier.
The same applies to the type and go.string symbols.
Removing the names cuts godoc by more than half:
-rwxr-xr-x 1 rsc rsc 9153405 2011-03-07 23:19 godoc.old
-rwxr-xr-x 1 rsc rsc 4290071 2011-03-07 23:19 godoc
For what it's worth, only 80% of what's left gets loaded
into memory; the other 20% is dwarf debugging information
only ever accessed by gdb:
-rwxr-xr-x 1 rsc rsc 3397787 2011-03-07 23:19 godoc.nodwarf
R=r, cw
CC=golang-dev
https://golang.org/cl/4245072
|
|
R=ken2
CC=golang-dev
https://golang.org/cl/4245055
|
|
A reference to the address of weak.foo resolves at link time
to the address of the symbol foo if foo would end up in the
binary anyway, or to zero if foo would not be in the binary.
For example:
int xxx = 1;
int yyy = 2;
int weak·xxx;
int weak·yyy;
void main·main(void) {
runtime·printf("%p %p %p\n", &xxx, &weak·xxx, &weak·yyy);
}
prints the same non-nil address twice, then 0 (because yyy is not
referenced so it was dropped from the binary).
This will be used by the reflection tables.
R=iant
CC=golang-dev
https://golang.org/cl/4223044
|
|
R=ken2
CC=golang-dev
https://golang.org/cl/4128054
|
|
* Avoid confusion between imported and exported symbols.
* Record number of imported and exported symbols correctly.
* Explictly relocate SMACHOSYM section, since it is not in datap.
R=rsc
CC=golang-dev
https://golang.org/cl/3920042
|
|
R=iant, r
CC=dho, golang-dev
https://golang.org/cl/3687041
|
|
More support for references to dynamic symbols,
including full GOT and PLT for ELF objects.
For Mach-O everything ends up in the GOT: dealing
with the real lazy PLT is too hard for now so we punt.
R=iant, iant2
CC=golang-dev
https://golang.org/cl/3491042
|
|
Using explicit relocations internally, we can
represent the data for a particular symbol as
an initialized block of memory instead of a
linked list of ADATA instructions. The real
goal here is to be able to hand off some of the
relocations to the dynamic linker when interacting
with system libraries, but a pleasant side effect is
that the memory image is much more compact
than the ADATA list, so the linkers use less memory.
R=ken2
CC=golang-dev
https://golang.org/cl/2512041
|
|
* Maintain Sym* list for text with individual
prog lists instead of using one huge list and
overloading p->pcond.
* Comment what each file is for.
* Move some output code from span.c to asm.c.
* Move profiling into prof.c, symbol table into symtab.c.
* Move mkfwd to ld/lib.c.
* Throw away dhog dynamic loading code.
* Throw away Alef become.
* Fix printing of WORD instructions in 5l -a.
Goal here is to be able to handle each piece of text or data
as a separate piece, both to make it easier to load the
occasional .o file and also to make it possible to split the
work across multiple threads.
R=ken2, r, ken3
CC=golang-dev
https://golang.org/cl/2335043
|
|
TBR=lvd
CC=golang-dev
https://golang.org/cl/2194041
|
