aboutsummaryrefslogtreecommitdiff
path: root/src/text/template/parse/lex.go
AgeCommit message (Collapse)Author
2024-09-23text/template/parse: use correct line number in error after commentyincong
Fixes #69526 Change-Id: I42467ddec02e91f24bce87185bf8d7f16f8811b0 GitHub-Last-Rev: 039a5b6884aa65f34cecbfcd127861a703a048da GitHub-Pull-Request: golang/go#69532 Reviewed-on: https://go-review.googlesource.com/c/go/+/614375 Auto-Submit: Ian Lance Taylor <iant@golang.org> Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>
2022-11-04all: fix function names in commentscui fliter
Change-Id: I871a747b4b47bccc889f2fdc93a2bcebb041b719 Reviewed-on: https://go-review.googlesource.com/c/go/+/447895 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-09-23text/template/parse: fix confusion about markers near right delimsRuss Cox
Fixes #52527. Fixes #55336. Change-Id: I8f5c521c693e74451a558788909e7e4ad1cc797a Reviewed-on: https://go-review.googlesource.com/c/go/+/433036 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-09-15text/template/parse: simplify I/O in lexingRob Pike
The concurrent model for delivering tokens was fine for pedagogy, but has caused a few problems as the package has evolved (that is, got more complicated). It's easy to eliminate it, simplifying or removing some of the hacks used to work around these prolems. The old lexer would deliver tokens over a channel to the parsing goroutine, and continue running until EOF. In this rewrite, we instead run the machine until a token is ready, and shut it down until the next token is needed. The mechanism is just to return nil as the state function, which requires a bit more threading of return values through the state functions but is not difficult. The change is modest. A couple of error messages change, but otherwise the change has no external effect. This is just an internal cleanup, long overdue. benchmark old ns/op new ns/op delta BenchmarkParseLarge-20 12222729 6769966 -44.61% BenchmarkVariableString-20 73.5 73.4 -0.16% BenchmarkListString-20 1827 1841 +0.77% benchmark old allocs new allocs delta BenchmarkVariableString-20 3 3 +0.00% BenchmarkListString-20 31 31 +0.00% benchmark old bytes new bytes delta BenchmarkVariableString-20 72 72 +0.00% BenchmarkListString-20 1473 1473 +0.00% Fixes #53261 Change-Id: I4133bed2f8df16d398b707fb9509230325765c57 Reviewed-on: https://go-review.googlesource.com/c/go/+/421883 Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Russ Cox <rsc@golang.org>
2022-06-06text/template/parse: fix data race on lexer initializationEli Bendersky
Before this change, `startParse` would write `lex.breakOK` and `lex.continueOK` when the lexer goroutine is already running, which is a potential race condition. Makes `breakOK` and `continueOK` configuration flags passed when `lexer` is created, similarly to how `emitComment` works. Fixes #53234 Change-Id: Ia65f6135509a758cd4c5a453b249a174f4fb3e21 Reviewed-on: https://go-review.googlesource.com/c/go/+/410414 Reviewed-by: Eli Bendersky <eliben@google.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org>
2022-05-16text/template/parse: make atTerminator more efficientRob Pike
The change https://go.dev/cl/398475 was too complicated and expensive. Since the whole string is always available, all that's needed is a call to strings.HasPrefix. While we're here, change the way lexer.backup works so it can be called repeatedly to back up more than one rune, in case that becomes necessary. This change also requires less state to maintain, as lexer.width was only there for backup, and prevented multiple steps. Fixes #52191 Change-Id: I43b64fc66edeb8ba73ba5aa72f3b727c377dc067 Reviewed-on: https://go-review.googlesource.com/c/go/+/406476 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Rob Pike <r@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-06text/template: support delimiters that can be confused with actionsmprahl
In fields that start with the same character as the right delimiter, the whole delimiter needs to be checked. The first character alone is not sufficient. Fixes #52165 Change-Id: I1e4086048417693757f34d0e9ff3bf86aba0d35c Reviewed-on: https://go-review.googlesource.com/c/go/+/398475 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org> Trust: Ian Lance Taylor <iant@golang.org>
2021-12-13all: gofmt -w -r 'interface{} -> any' srcRuss Cox
And then revert the bootstrap cmd directories and certain testdata. And adjust tests as needed. Not reverting the changes in std that are bootstrapped, because some of those changes would appear in API docs, and we want to use any consistently. Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories when preparing the bootstrap copy. A few files changed as a result of running gofmt -w not because of interface{} -> any but because they hadn't been updated for the new //go:build lines. Fixes #49884. Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09 Reviewed-on: https://go-review.googlesource.com/c/go/+/368254 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-09-23html/template, text/template: implement break and continue for range loopsRuss Cox
Break and continue for range loops was accepted as a proposal in June 2017. It was implemented in CL 66410 (Oct 2017) but then rolled back in CL 92155 (Feb 2018) because html/template changes had not been implemented. This CL reimplements break and continue in text/template and then adds support for them in html/template as well. Fixes #20531. Change-Id: I05330482a976f1c078b4b49c2287bd9031bb7616 Reviewed-on: https://go-review.googlesource.com/c/go/+/321491 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2020-10-12text/template: allow newlines inside action delimitersRuss Cox
This allows multiline constructs like: {{"hello" | printf}} Now that unclosed actions can span multiple lines, track and report the start of the action when reporting errors. Also clean up a few "unexpected <error message>" to be just "<error message>". Fixes #29770. Change-Id: I54c6c016029a8328b7902a4b6d85eab713ec3285 Reviewed-on: https://go-review.googlesource.com/c/go/+/254257 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2020-08-28text/template: add CommentNode to template parse treeAriel Mashraki
Fixes #34652 Change-Id: Icf6e3eda593fed826736f34f95a9d66f5450cc98 Reviewed-on: https://go-review.googlesource.com/c/go/+/229398 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-23text/template/parse: remove redundant returnAriel Mashraki
Change the `itemChar` clause to be like all other clauses that don't return a different state function than the default. Change-Id: I56c863a7d699c1264b24b42ef23138ec47eaacd8 Reviewed-on: https://go-review.googlesource.com/c/go/+/216117 Reviewed-by: Rob Pike <r@golang.org> Run-TryBot: Rob Pike <r@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-24text/template: allow using -}} with many spacesDaniel Martí
lexSpace consumed all spaces, even if the last one was part of a right delimiter like " -}}". Thus, "3 -}}" wouldn't lex as "3" and a right delimiter, but as "3", "-", and "}}". To fix that, make lexSpace stop if it encounters a right delimiter. Fixes #30948. Change-Id: I80a5546e5809e54f6823e2bf3a57a7e8808329be Reviewed-on: https://go-review.googlesource.com/c/go/+/168457 Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2019-02-26text/template: accept new number syntaxRuss Cox
This CL updates text/template's scanner to accept the new number syntaxes: - Hexadecimal floating-point values. - Digit-separating underscores. - Leading 0b and 0o prefixes. See golang.org/design/19308-number-literals for background. For #12711. For #19308. For #28493. For #29008. Change-Id: I68c16ea35c3f506701063781388de72bafee6b8d Reviewed-on: https://go-review.googlesource.com/c/160248 Reviewed-by: Rob Pike <r@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org>
2018-09-05text/template: simplify line tracking in the lexerDaniel Martí
First, move the strings.Count logic out of emit, since only itemText requires that. Use it in those call sites. itemLeftDelim and itemRightDelim cannot contain newlines, as they're the "{{" and "}}" tokens. Secondly, introduce a startLine lexer field so that we don't have to keep track of it elsewhere. That's also a requirement to move the strings.Count out of emit, as emit modifies the start position field. Change-Id: I69175f403487607a8e5b561b3f1916ee9dc3c0c6 Reviewed-on: https://go-review.googlesource.com/132275 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2018-08-29text/template: fix newline counting in raw stringsDaniel Martí
lexRawQuote already uses the next method, which keeps track of newlines on a character by character basis. Adding up newlines in emit again results in the newlines being counted twice, which can mean bad position information in error messages. Fix that, and add a test. Fixes #27319. Change-Id: Id803be065c541412dc808d388bc6d8a86a0de41e Reviewed-on: https://go-review.googlesource.com/131996 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-06-28text/template/parse: fix a comment around the assign operatorAndrew Braunstein
Fix a comment that misrepresented the Assign operator (=). Rename: colon-equals -> equals. Change-Id: I405b8acfb0bcd1b176a91a95f9bfb61a4e85815f GitHub-Last-Rev: aec0bf594c63d7b015f88f97f9953ade976817a4 GitHub-Pull-Request: golang/go#26112 Reviewed-on: https://go-review.googlesource.com/121416 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-04-04text/template: add variable assignmentsDaniel Martí
Variables can be declared and shadowing is supported, but modifying existing variables via assignments was not available. This meant that modifying a variable from a nested block was not possible: {{ $v := "init" }} {{ if true }} {{ $v := "changed" }} {{ end }} v: {{ $v }} {{/* "init" */}} Introduce the "=" assignment token, such that one can now do: {{ $v := "init" }} {{ if true }} {{ $v = "changed" }} {{ end }} v: {{ $v }} {{/* "changed" */}} To avoid confusion, rename PipeNode.Decl to PipeNode.Vars, as the variables may not always be declared after this change. Also change a few other names to better reflect the added ambiguity of variables in pipelines. Modifying the text/template/parse package in a backwards incompatible manner is acceptable, given that the package godoc clearly states that it isn't intended for general use. It's the equivalent of an internal package, back when internal packages didn't exist yet. To make the changes to the parse package sit well with the cmd/api test, update except.txt with the changes that we aren't worried about. Fixes #10608. Change-Id: I1f83a4297ee093fd45f9993cebb78fc9a9e81295 Reviewed-on: https://go-review.googlesource.com/84480 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2018-02-06text/template: revert CL 66410 "add break, continue actions in ranges"Ian Lance Taylor
The new break and continue actions do not work in html/template, and fixing them requires thinking about security issues that seem too tricky at this stage of the release. We will try again for 1.11. Original CL description: text/template: add break, continue actions in ranges Adds the two range control actions "break" and "continue". They act the same as the Go keywords break and continue, but are simplified in that only the innermost range statement can be broken out of or continued. Fixes #20531 Updates #20531 Updates #23683 Change-Id: Ia7fd3c409163e3bcb5dc42947ae90b15bdf89853 Reviewed-on: https://go-review.googlesource.com/92155 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2017-11-10text/template: remove unnecessary lexer fieldAriel Mashraki
this change removes the state field from the lexer, because it's only used by the run method and can be replaced with a local variable Change-Id: Ib7a90ab6e9a894716cba2c7d9ed71bf2ad1240c0 Reviewed-on: https://go-review.googlesource.com/47338 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-11-09text/template: remove unused lexer fieldAriel Mashraki
The lastPos field used in the past to track the line number of a token. it's irrelevant anymore, and we can remove it. Change-Id: I42c0bf55e884b79574a7da4926489f2d77618cd0 Reviewed-on: https://go-review.googlesource.com/49591 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-10-17text/template: add break, continue actions in rangesTim Cooper
Adds the two range control actions "break" and "continue". They act the same as the Go keywords break and continue, but are simplified in that only the innermost range statement can be broken out of or continued. Fixes #20531 Change-Id: I4412b3bbfd4dadb0ab74ae718e308c1ac7a0a1e9 Reviewed-on: https://go-review.googlesource.com/66410 Reviewed-by: Rob Pike <r@golang.org>
2017-09-07text/template: fix pos info when trimming newlinesDaniel Martí
The lexer keeps the byte offset and the line for the rune it's currently on. This was simple enough up until whitespace trimming was introduced. With whitespace trimming, we might skip over newlines. In that case, the lexer wasn't properly updating the line counter. Fix it. Also, TestPos now checks that the line is correct too, which it was ignoring before. This was necessary to test this scenario in the lexer. Fixes #21778. Change-Id: I3880f3adf02662eac8f818d5caa6935cca9cb33b Reviewed-on: https://go-review.googlesource.com/61870 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Rob Pike <r@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-08-29all: join some chained ifs to unindent codeDaniel Martí
Found with mvdan.cc/unindent. It skipped the cases where parentheses would need to be added, where comments would have to be moved elsewhere, or where actions and simple logic would mix. One of them was of the form "err != nil && err == io.EOF", so the first part was removed. Change-Id: Ie504c2b03a2c87d10ecbca1b9270069be1171b91 Reviewed-on: https://go-review.googlesource.com/57690 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-15text/template: efficient reporting of line numbersRob Pike
Instead of scanning the text to count newlines, which is n², keep track as we go and store the line number in the token. benchmark old ns/op new ns/op delta BenchmarkParseLarge-4 1589721293 38783310 -97.56% Fixes #17851 Change-Id: I231225c61e667535e2ce55cd2facea6d279cc59d Reviewed-on: https://go-review.googlesource.com/33234 Run-TryBot: Rob Pike <r@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-14Revert "text/template: efficient reporting of line numbers"Brad Fitzpatrick
This reverts commit 794fb71d9c1018c4beae1657baca5229e6a02ad0. Reason for revert: submitted without TryBots and it broke all three race builders. Change-Id: I80a1e566616f0ee8fa3529d4eeee04268f8a713b Reviewed-on: https://go-review.googlesource.com/33232 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-14text/template: efficient reporting of line numbersRob Pike
Instead of scanning the text to count newlines, which is n², keep track as we go and store the line number in the token. benchmark old ns/op new ns/op delta BenchmarkParseLarge-4 1589721293 38783310 -97.56% Fixes #17851 Change-Id: Ieaf89a35e371b405ad92e38baa1e3fa98d18cfb4 Reviewed-on: https://go-review.googlesource.com/32923 Reviewed-by: Robert Griesemer <gri@golang.org>
2016-09-13text/template: improve lexer performance in finding left delimiters.Paul Borman
The existing implementation calls l.next for each run up to the next instance of the left delimiter ({{). For ascii text, this is multiple function calls per byte. Change to use strings.Index to find the left delimiter. The performace improvement ranges from 1:1 (no text outside of {{}}'s) to multiple times faster (9:1 was seen on 8K of text with no {{ }}'s). Change-Id: I2f82bea63b78b6714f09a725f7b2bbb00a3448a3 Reviewed-on: https://go-review.googlesource.com/24863 Reviewed-by: Rob Pike <r@golang.org> Run-TryBot: Rob Pike <r@golang.org>
2016-02-19all: replace strings.Index with strings.Contains where possibleNathan VanBenschoten
Change-Id: Ia613f1c37bfce800ece0533a5326fca91d99a66a Reviewed-on: https://go-review.googlesource.com/18120 Reviewed-by: Robert Griesemer <gri@golang.org> Run-TryBot: Robert Griesemer <gri@golang.org>
2015-09-28text/template, html/template: add block keyword and permit template redefinitionAndrew Gerrand
This change adds a new "block" keyword that permits the definition of templates inline inside existing templates, and loosens the restriction on template redefinition. Templates may now be redefined, but in the html/template package they may only be redefined before the template is executed (and therefore escaped). The intention is that such inline templates can be redefined by subsequent template definitions, permitting a kind of template "inheritance" or "overlay". (See the example for details.) Fixes #3812 Change-Id: I733cb5332c1c201c235f759cc64333462e70dc27 Reviewed-on: https://go-review.googlesource.com/14005 Reviewed-by: Rob Pike <r@golang.org>
2015-09-09text/template: provide a way to trim leading and trailing space between actionsRob Pike
Borrowing a suggestion from the issue listed below, we modify the lexer to trim spaces at the beginning (end) of a block of text if the action immediately before (after) is marked with a minus sign. To avoid parsing/lexing ambiguity, we require an ASCII space between the minus sign and the rest of the action. Thus: {{23 -}} < {{- 45}} produces the output 23<45 All the work is done in the lexer. The modification is invisible to the parser or any outside package (except I guess for noticing some gaps in the input if one tracks error positions). Thus it slips in without worry in text/template and html/template both. Fixes long-requested issue #9969. Change-Id: I3774be650bfa6370cb993d0899aa669c211de7b2 Reviewed-on: https://go-review.googlesource.com/14391 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-07text/template: delete obsolete nil checkRob Pike
This was added during testing but is unnecessary. Thanks to gravis on GitHub for catching it. See #10574. Change-Id: I4a8f76d237e67f5a0ea189a0f3cadddbf426778a Reviewed-on: https://go-review.googlesource.com/9841 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-05-05text/template: shut down lexing goroutine on errorRob Pike
When a parse error occurred, the lexing goroutine would lay idle. It's not likely a problem but if the program is for some reason accepting badly formed data repeatedly, it's wasteful. The solution is easy: Just drain the input on error. We know this will succeed because the input is always a string and is therefore guaranteed finite. With debugging prints in the package tests I've shown this is effective, shutting down 79 goroutines that would otherwise linger, out of 123 total. Fixes #10574. Change-Id: I8aa536e327b219189a7e7f604a116fa562ae1c39 Reviewed-on: https://go-review.googlesource.com/9658 Reviewed-by: Russ Cox <rsc@golang.org>
2015-05-04text/template: ensures code consistency in lexerAymerick
At the end of lexInsideAction(), we return lexInsideAction: this is the default behaviour when we are still parsing an action. But some switch branches return lexInsideAction too. So let's ensure code consistency by always reaching the end of the lexInsideAction function when needed. Change-Id: I7e9d8d6e51f29ecd6db6bdd63b36017845d95368 Reviewed-on: https://go-review.googlesource.com/9441 Reviewed-by: Rob Pike <r@golang.org>
2015-05-01text/template: allow newlines in raw quotesRob Pike
This was disallowed for error-checking reasons but people ask for it, it's easy, and it's clear what it all means. Fixes #7323. Change-Id: I26542f5ac6519e45b335ad789713a4d9e356279b Reviewed-on: https://go-review.googlesource.com/9537 Reviewed-by: Russ Cox <rsc@golang.org>
2014-09-08build: move package sources from src/pkg to srcRuss Cox
Preparation was in CL 134570043. This CL contains only the effect of 'hg mv src/pkg/* src'. For more about the move, see golang.org/s/go14nopkg.
generated by cgit v1.3 (git 2.53.0) at 2026-04-16 07:25:15 +0000