aboutsummaryrefslogtreecommitdiff
path: root/src/cmd/compile/internal/syntax
AgeCommit message (Collapse)Author
2017-02-10cmd/compile/internal/syntax: removed gcCompat code needed to pass orig. testsRobert Griesemer
The gcCompat mode was introduced to match the new parser's node position setup exactly with the positions used by the original parser. Some of the gcCompat adjustments were required to satisfy syntax error test cases, and the rest were required to make toolstash cmp pass. This change removes the former gcCompat adjustments and instead adjusts the respective test cases as necessary. In some cases this makes the error lines consistent with the ones reported by gccgo. Where it has changed, the position associated with a given syntactic construct is the position (line/col number) of the left-most token belonging to the construct. Change-Id: I5b60c00c5999a895c4d6d6e9b383c6405ccf725c Reviewed-on: https://go-review.googlesource.com/36695 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2017-02-09cmd/compile/internal/parser: improved syntax error for incorrect ↵Robert Griesemer
if/for/switch header Starting the error message with "expecting" rather than "missing" causes the syntax error mechanism to add additional helpful info (it recognizes "expecting" but not "missing"). Fixes #17328. Change-Id: I8482ca5e5a6a6b22e0ed0d831b7328e264156334 Reviewed-on: https://go-review.googlesource.com/36637 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-02-09cmd/compile/internal/syntax: differentiate between ';' and '\n' in syntax errorsRobert Griesemer
Towards better syntax error messages: With this change, the parser knows whether a semicolon was an actual ';' in the source, or whether it was an automatically inserted semicolon as result of a '\n' or EOF. Using this information in error messages makes them more understandable. For #17328. Change-Id: I8cd9accee8681b62569d0ecef922d38682b401eb Reviewed-on: https://go-review.googlesource.com/36636 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2017-02-07cmd/compile/internal/syntax: avoid follow-up error for incorrect if statementRobert Griesemer
This is a follow-up on https://go-review.googlesource.com/36470 and leads to a more stable fix. The above CL relied on filtering of multiple errors on the same line to avoid more than one error for an `if` statement of the form `if a := 10 {}`. This CL avoids the secondary error ("missing condition in if statement") in the first place. For #18915. Change-Id: I8517f485cc2305965276c17d8f8797d61ef9e999 Reviewed-on: https://go-review.googlesource.com/36479 TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2017-02-06cmd/compile/internal/syntax: make a parser error "1.7 compliant"Robert Griesemer
For code such as if a := 10 { ... the 1.7 compiler reported a := 10 used as value while the 1.8 compiler reported invalid condition, tag, or type switch guard Changed the error message to match the 1.7 compiler. Fixes #18915. Change-Id: I01308862e461922e717f9f8295a9db53d5a914eb Reviewed-on: https://go-review.googlesource.com/36470 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-02-01all: merge dev.inline into masterRuss Cox
Change-Id: I7715581a04e513dcda9918e853fa6b1ddc703770
2017-01-10[dev.typealias] cmd/compile: recognize type aliases but complain for now ↵Robert Griesemer
(not yet supported) Added test file. For #18130. Change-Id: Ifcfd7cd1acf9dd6a2f4f3d85979d232bb6b8c6b1 Reviewed-on: https://go-review.googlesource.com/34988 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-01-09[dev.inline] cmd/internal/src: replace src.Pos with syntax.PosRobert Griesemer
This replaces the src.Pos LineHist-based position tracking with the syntax.Pos implementation and updates all uses. The LineHist table is not used anymore - the respective code is still there but should be removed eventually. CL forthcoming. Passes toolstash -cmp when comparing to the master repo (with the exception of a couple of swapped assembly instructions, likely due to different instruction scheduling because the line-based sorting has changed; though this is won't affect correctness). The sizes of various important compiler data structures have increased significantly (see the various sizes_test.go files); this is probably the reason for an increase of compilation times (to be addressed). Here are the results of compilebench -count 5, run on a "quiet" machine (no apps running besides a terminal): name old time/op new time/op delta Template 256ms ± 1% 280ms ±15% +9.54% (p=0.008 n=5+5) Unicode 132ms ± 1% 132ms ± 1% ~ (p=0.690 n=5+5) GoTypes 891ms ± 1% 917ms ± 2% +2.88% (p=0.008 n=5+5) Compiler 3.84s ± 2% 3.99s ± 2% +3.95% (p=0.016 n=5+5) MakeBash 47.1s ± 1% 47.2s ± 2% ~ (p=0.841 n=5+5) name old user-ns/op new user-ns/op delta Template 309M ± 1% 326M ± 2% +5.18% (p=0.008 n=5+5) Unicode 165M ± 1% 168M ± 4% ~ (p=0.421 n=5+5) GoTypes 1.14G ± 2% 1.18G ± 1% +3.47% (p=0.008 n=5+5) Compiler 5.00G ± 1% 5.16G ± 1% +3.12% (p=0.008 n=5+5) Change-Id: I241c4246cdff627d7ecb95cac23060b38f9775ec Reviewed-on: https://go-review.googlesource.com/34273 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-12-10[dev.inline] cmd/compile/internal/syntax: add predicates to syntax.PosRobert Griesemer
This moves syntax.Pos closer to cmd/internal/src.Pos so that we can more easily replace src.Pos with syntax.Pos going forward. Change-Id: I9f93a65fecb4c22591edca4b9d6cda39cf0e872e Reviewed-on: https://go-review.googlesource.com/34270 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-12-09[dev.inline] cmd/compile/internal/syntax: add tests for //line directivesRobert Griesemer
Change-Id: I77dc73bfe79e43bbadf85d7eb3c5f8990ec72023 Reviewed-on: https://go-review.googlesource.com/34248 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-12-09[dev.inline] cmd/compile/internal/syntax: report byte offset rather then ↵Robert Griesemer
rune count for column value This will only become user-visible if error messages show column information. Per the discussion in #10324. For #10324. Change-Id: I5959c1655aba74bb1a22fdc261cd728ffcfa6912 Reviewed-on: https://go-review.googlesource.com/34244 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-12-09[dev.inline] cmd/compile/internal/syntax: remove gcCompat uses in scannerRobert Griesemer
- make the scanner unconditionally gc compatible - consistently use "invalid" instead "illegal" in errors Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33896/. Change-Id: I4c4253e7392f3311b0d838bbe503576c9469b203 Reviewed-on: https://go-review.googlesource.com/34237 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-12-09[dev.inline] cmd/compile/internal/syntax: use syntax.Pos for all external ↵Robert Griesemer
positions - use syntax.Pos in syntax.Error (rather than line, col) - use syntax.Pos in syntax.PragmaHandler (rather than just line) - update uses - better documentation in various places Also: - make Pos methods use Pos receiver (rather than *Pos) Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33891/. With minor adjustments to noder.go to make merge compile. Change-Id: I5507cea6c2be46a7677087c1aeb69382d31033eb Reviewed-on: https://go-review.googlesource.com/34236 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-12-09[dev.inline] cmd/compile/internal/syntax: clean up error and pragma handlingRobert Griesemer
Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33873/. - simplify error handling in source.go (move handling of first error into parser, where it belongs) - clean up error handling in scanner.go - move pragma and position base handling from scanner to parser where it belongs - have separate error methods in parser to avoid confusion with handlers from scanner.go and source.go - (source.go) and (scanner.go, source.go, tokens.go) may be stand-alone packages if so desired, which means these files are now less entangled and easier to maintain Change-Id: I81510fc7ef943b78eaa49092c0eab2075a05878c Reviewed-on: https://go-review.googlesource.com/34235 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-12-09[dev.inline] cmd/compile/internal/syntax: simplified position codeRobert Griesemer
Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33805/. Change-Id: I859d9bd5f2256ca78f7b24b330290f7ae600854d Reviewed-on: https://go-review.googlesource.com/34234 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-12-09[dev.inline] cmd/compile/internal/syntax: process //line pragmas in scannerRobert Griesemer
Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33764/. Minor adjustment in noder.go to make merge compile again. Change-Id: Ib5029b52b59944f207b0f2438c8a5aa576eb25b8 Reviewed-on: https://go-review.googlesource.com/34233 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-12-09[dev.inline] cmd/compile/internal/syntax: introduce general position info ↵Robert Griesemer
for nodes Reviewed in and cherry-picked from https://go-review.googlesource.com/#/c/33758/. Minor adjustments in noder.go to fix merge. Change-Id: Ibe429e327c7f8554f8ac205c61ce3738013aed98 Reviewed-on: https://go-review.googlesource.com/34231 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-12-02cmd/compile/internal/syntax: remove dead type declarationRobert Griesemer
Change-Id: I8a3ce0fa513ff943009c5669531132cd23ecf155 Reviewed-on: https://go-review.googlesource.com/33870 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-29cmd/compile/internal/syntax: remove unused node fieldRobert Griesemer
The doc field is not yet used - remove it for now (we may end up with a different solution for 1.9). This reduces memory consumption for parsing all of std lib by about 40MB and makes parsing slightly faster. Change-Id: Iafb00b9c7f1be9c66fdfb29096d3da5049b2fcf5 Reviewed-on: https://go-review.googlesource.com/33661 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-11-29cmd/compile: don't panic on syntax error in select statementRobert Griesemer
Fixes #18092. Change-Id: I54e2da2e0f168c068f5e4a1b22ba508d78259168 Reviewed-on: https://go-review.googlesource.com/33658 TryBot-Result: Gobot Gobot <gobot@golang.org> Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-11-11all: fix vet nitsJosh Bleecher Snyder
Fixes these complaints from vet: cmd/compile/internal/gc/noder.go:32: cmd/compile/internal/syntax.Error composite literal uses unkeyed fields cmd/compile/internal/gc/noder.go:1035: cmd/compile/internal/syntax.Error composite literal uses unkeyed fields cmd/compile/internal/gc/noder.go:1051: cmd/compile/internal/syntax.Error composite literal uses unkeyed fields cmd/compile/internal/syntax/parser_test.go:182: possible formatting directive in Error call net/http/client_test.go:1334: possible formatting directive in Fatal call Change-Id: I5f90ec30f3c106c7e66c92e2b6f8d3b4874fec66 Reviewed-on: https://go-review.googlesource.com/33133 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-09cmd/compile/internal/syntax: fix error handling for Read/Parse callsRobert Griesemer
- define syntax.Error for cleaner error reporting - abort parsing after first error if no error handler is installed - make sure to always report the first error, if any - document behavior of API calls - while at it: rename ReadXXX -> ParseXXX (clearer) - adjust cmd/compile noder.go accordingly Fixes #17774. Change-Id: I7893eedea454a64acd753e32f7a8bf811ddbb03c Reviewed-on: https://go-review.googlesource.com/32950 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-11-05cmd/go/internal/syntax: reintroduce reverted commentsRobert Griesemer
These comments were originally introduced together with the changes for alias declarations, and then reverted when we backed out alias support. Reintroduce them. Change-Id: I3ef2c4f4672d6af8a900f5d73df273edf28d1a14 Reviewed-on: https://go-review.googlesource.com/32826 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-11-05Revert "cmd/compile/internal/syntax: support for alias declarations"Robert Griesemer
This reverts commit 32db3f2756324616b7c856ac9501deccc2491239. Reason: Decision to back out current alias implementation. For #16339. Change-Id: Ib05e3d96041d8347e49cae292f66bec791a1fdc8 Reviewed-on: https://go-review.googlesource.com/32825 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-11-01cmd/compile/internal/syntax: don't depend on hardwired $GOROOT nameRobert Griesemer
Fixes #17697. Change-Id: I3c47e139b09bde81566e29a1ac0ec8c58d55a34a Reviewed-on: https://go-review.googlesource.com/32539 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-01cmd/compile/internal/syntax: avoid deadlockJosh Bleecher Snyder
When the err from ReadFile is non-nil, we call t.Fatal(err). Switch t.Fatal to t.Error + return. ensure that close(results) happens on that code path as well. Updates #17697. Change-Id: Ifaacf27a76c175446d642086ff32f4386428080d Reviewed-on: https://go-review.googlesource.com/32486 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Robert Griesemer <gri@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-01cmd/compile/internal/syntax: don't panic if no error handler is providedRobert Griesemer
If no error handler is provided, terminate parsing with first error and report that error. Fixes #17697. Change-Id: I9070faf7239bd53725de141507912b92ded3474b Reviewed-on: https://go-review.googlesource.com/32456 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-10-27cmd/compile, go/parser: disallow "type T = p.T" - must use "=>"Robert Griesemer
I had added this originally so we can play with different notations but it doesn't make sense to keep it around since gofmt will convert a type alias declaration using "=" into one using "=>" anyhow. More importantly, the spec doesn't permit it. Change-Id: Icb010b5a9976aebf877e48b3ce9d7245559ca494 Reviewed-on: https://go-review.googlesource.com/32105 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-10-25cmd/compile: simplify parsing of type aliasesRobert Griesemer
Change-Id: Ia86841cf84bc17ff6ecc6e5ac4cec86384a0da00 Reviewed-on: https://go-review.googlesource.com/31719 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-10-15cmd/compile: add go:notinheap type pragmaAustin Clements
This adds a //go:notinheap pragma for declarations of types that must not be heap allocated. We ensure these rules by disallowing new(T), make([]T), append([]T), or implicit allocation of T, by disallowing conversions to notinheap types, and by propagating notinheap to any struct or array that contains notinheap elements. The utility of this pragma is that we can eliminate write barriers for writes to pointers to go:notinheap types, since the write barrier is guaranteed to be a no-op. This will let us mark several scheduler and memory allocator structures as go:notinheap, which will let us disallow write barriers in the scheduler and memory allocator much more thoroughly and also eliminate some problematic hybrid write barriers. This also makes go:nowritebarrierrec and go:yeswritebarrierrec much more powerful. Currently we use go:nowritebarrier all over the place, but it's almost never what you actually want: when write barriers are illegal, they're typically illegal for a whole dynamic scope. Partly this is because go:nowritebarrier has been around longer, but it's also because go:nowritebarrierrec couldn't be used in situations that had no-op write barriers or where some nested scope did allow write barriers. go:notinheap eliminates many no-op write barriers and go:yeswritebarrierrec makes it possible to opt back in to write barriers, so these two changes will let us use go:nowritebarrierrec far more liberally. This updates #13386, which is about controlling pointers from non-GC'd memory to GC'd memory. That would require some additional pragma (or pragmas), but could build on this pragma. Change-Id: I6314f8f4181535dd166887c9ec239977b54940bd Reviewed-on: https://go-review.googlesource.com/30939 Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-16cmd/compile/internal/syntax: support for alias declarationsRobert Griesemer
Permits parsing of alias declarations with -newparser const/type/var/func T => p.T but the compiler will reject it with an error. For now this also accepts type T = p.T so we can experiment with a type-alias only scenario. - renamed _Arrow token to _Larrow (<-) - introduced _Rarrow token (=>) - introduced AliasDecl node - extended scanner to accept _Rarrow - extended parser and printer to handle alias declarations Change-Id: I0170d10a87df8255db9186d466b6fd405228c38e Reviewed-on: https://go-review.googlesource.com/29355 Run-TryBot: Robert Griesemer <gri@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-09-16cmd/compile/internal/gc: make Nod privateDave Cheney
Follow up to CL 29134. Generated with gofmt -r 'Nod -> nod', plus three manual adjustments to the comments in syntax/parser.go Change-Id: I02920f7ab10c70b6e850457b42d5fe35f1f3821a Reviewed-on: https://go-review.googlesource.com/29136 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-09cmd/compile/internal/syntax: remove strbyteseqlMatthew Dempsky
cmd/compile already optimizes "string(b) == s" to avoid allocating a temporary string. Change-Id: I4244fbeae8d350261494135c357f9a6e2ab7ace3 Reviewed-on: https://go-review.googlesource.com/28931 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-08-31cmd/compile: handle pragmas immediately with -newparser=1Matthew Dempsky
Instead of saving all pragmas and processing them after parsing is finished, process them immediately during scanning like the current lexer does. This is a bit unfortunate because it means we can't use syntax.ParseFile to concurrently parse files yet, but it fixes how we report syntax errors in the presence of //line pragmas. While here, add a bunch more gcCompat entries to syntax/parser.go to get "go build -toolexec='toolstash -cmp' std cmd" passing. There are still a few remaining cases only triggered building unit tests, but this seems like a nice checkpoint. Change-Id: Iaf3bbcf2849857a460496f31eea228e0c585ce13 Reviewed-on: https://go-review.googlesource.com/28226 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2016-08-25net/http, cmd/compile: minor vet fixesJosh Bleecher Snyder
Updates #11041 Change-Id: Ia0151723e3bc0d163cc687a02bfc5e0285d95ffa Reviewed-on: https://go-review.googlesource.com/27810 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-08-19cmd/compile/internal/syntax: match old parser errors and line numbersMatthew Dempsky
This makes a bunch of changes to package syntax to tweak line numbers for AST nodes. For example, short variable declaration statements are now associated with the location of the ":=" token, and function calls are associated with the location of the final ")" token. These help satisfy many unit tests that assume the old parser's behavior. Because many of these changes are questionable, they're guarded behind a new "gcCompat" const to make them easy to identify and revisit in the future. A handful of remaining tests are too difficult to make behave identically. These have been updated to execute with -newparser=0 and comments explaining why they need to be fixed. all.bash now passes with both the old and new parsers. Change-Id: Iab834b71ca8698d39269f261eb5c92a0d55a3bf4 Reviewed-on: https://go-review.googlesource.com/27199 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2016-08-19cmd/compile/internal/syntax: expose additional information for gcMatthew Dempsky
gc needs access to line offsets for Nodes. It also needs access to the end line offset for function bodies so it knows what line number to use for things like implicit returns and defer executions. Lastly, include an extra bool to distinguish between simple and full slice expressions. This is redundant in valid parse trees, but needed by gc for producing complete warnings in invalid inputs. Change-Id: I64baf334a35c72336d26fa6755c67eb9d6f4e93c Reviewed-on: https://go-review.googlesource.com/27196 Reviewed-by: Robert Griesemer <gri@golang.org> Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-08-18cmd/compile/internal/syntax: fast Go syntax trees, initial commit.Robert Griesemer
Syntax tree nodes, scanner, parser, basic printers. Builds syntax trees for entire Go std lib at a rate of ~1.8M lines/s in warmed up state (MacMini, 2.3 GHz Intel Core i7, 8GB RAM): $ go test -run StdLib -fast parsed 1074617 lines (2832 files) in 579.66364ms (1853863 lines/s) allocated 282.212Mb (486.854Mb/s) PASS Change-Id: Ie26d9a7bf4e5ff07457aedfcc9b89f0eba72ae3f Reviewed-on: https://go-review.googlesource.com/27195 Run-TryBot: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Robert Griesemer <gri@golang.org>