aboutsummaryrefslogtreecommitdiff
path: root/internal/fetchdatasource/fetchdatasource_test.go
AgeCommit message (Collapse)Author
2026-03-20internal: conditionally return documentation in GetUnitEthan Lee
- Large documentations blobs should avoid being returned unless explicitly requested. Add a WithDocsSource to enable retrieval of unit information without the large documentation source information. - This does not functionally change existing GetUnit callsites. Change-Id: Ie983b7eaeb61bb92f2716ae64a3d1e1ea738dbe2 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/756480 Reviewed-by: Jonathan Amsterdam <jba@google.com> kokoro-CI: kokoro <noreply+kokoro@google.com> Auto-Submit: Ethan Lee <ethanalee@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-06internal/fetch: split FetchModule in twoMichael Matloob
First we get a LazyModule that essentially just contains the UnitMetas for each of the modules and then we process each Unit as needed so that FetchDataSource doesn't need to process all the Units when it just needs one. To construct at FetchResult, FetchModule will now get a LazyModule and then compute all the Units. A compromise to get this to work is that FetchDataSource will no longer populate Synopsis, IsRedistributable and Licenses on Subdirectories to avoid computing all those fields for all units when returning a single unit. MainVersion and MasterVersion are removed from FetchResult because they were only set on the internal/worker.FetchTask struct that embeds a FetchResult. Change-Id: Ia0db850ae570d421712ec484ee8b7815a779128e Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/557818 Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> kokoro-CI: kokoro <noreply+kokoro@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
2024-01-23internal: move UnitMeta.IsRedistributable to UnitMichael Matloob
Change-Id: Ia0a18df32d752f20b0400b47860638c7d0910eb7 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/556435 Reviewed-by: Jonathan Amsterdam <jba@google.com> kokoro-CI: kokoro <noreply+kokoro@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com>
2023-08-31all: skip tests that can't run on wasm platformsMichael Matloob
Some because the network support isn't enough for httpreplay, and some because we need the Go command. For golang/go#61209 Change-Id: I02c6b7d10553d931c746723ce9608f39834c131c Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/524797 Reviewed-by: Robert Findley <rfindley@google.com> Run-TryBot: Michael Matloob <matloob@golang.org> kokoro-CI: kokoro <noreply+kokoro@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-07-06internal/testenv: add and use MustHaveExternalNetwork functionMichael Matloob
This CL adds a copy of the MustHaveExternalNetwork function from the testenv package in the Go repo for use for filtering out tests that need the network access from running on builders that don't guarantee it. (External network access is only enabled on longtest builders). It also calls the function from two tests that are failing because they make external network calls. For #61209 Change-Id: I68ed1d3f8bb65697b1025bca0ba40aa0ab7423a3 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/508295 Reviewed-by: Robert Findley <rfindley@google.com> Run-TryBot: Michael Matloob <matloob@google.com> TryBot-Bypass: Michael Matloob <matloob@google.com> Reviewed-by: Michael Matloob <matloob@google.com>
2023-04-14internal/fetchdatasource: remove arbitrary test timeoutRob Findley
Kokoro tests are failing with this arbitrary timeout. Remove it. Updates golang/go#59608 Change-Id: Ifd3d8cf02d5c14f54e505fa5812e8de7dbc6c5d5 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/484738 TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com> Run-TryBot: Robert Findley <rfindley@google.com>
2023-03-14cmd/pkgsite: add multi-module support, invalidation, and searchRobert Findley
This change adds a new goPackagesModuleGetter, which uses x/tools/go/packages to query package information for modules requested by the pkgsite command. Additionally, add new extension interfaces that allow getters to add support for search and content invalidation. The go/packages getter uses these extensions to implement search (via a simple fuzzy-matching algorithm copied from x/tools), and invalidation via statting package files. Along the way, refactor slightly for testing ergonomics. Updates golang/go#40371 Updates golang/go#50229 Fixes golang/go#54479 Change-Id: Iea91a4d6327707733cbbc4f74a9d93052f33e295 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/474295 TryBot-Result: kokoro <noreply+kokoro@google.com> Run-TryBot: Robert Findley <rfindley@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-10-28many: cmd/pkgsite embeds static assetsJonathan Amsterdam
The `pkgsite` command now embeds all the static assets it needs, so it need not be told the location of the static/ directory with a flag. The binary is self-contained and can be placed and invoked from anywhere. This required embedding the static/ and third_party/ directories. Since //go:embed cannot reference files outside the containing package's tree, we had to add trivial Go packages in static/ and third_party/ to construct `embed.FS`s for those directories. Also, the frontend needed to accept `fs.FS` values where it previously took paths, and `template.TrustedFS` values where it previously used `template.TrustedSources`. We ended up clumsily requiring four separate config values: - A TrustedFS to load templates. - Two fs.FSs, one for static and one for third_party, to load other assets. - The path to the static directory as a string, solely to support dynamic loading in dev mode. For golang/go#48410 Change-Id: I9eeb351b1c6f23444b9e65b60f2a1d3905d59ef9 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/359395 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-09-09internal/fetchdatasource: fix test to work on macsJonathan Amsterdam
Accept a wider range of URLs to accommodate Mac temp dirs. Change-Id: I2933b71ab1956a4fbf504e842d3ec2fe5c56e127 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348795 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org> Reviewed-by: Julie Qiu <julie@golang.org>
2021-09-07cmd/pkgsite,internal: serve sourceJonathan Amsterdam
- Set up a /files endpoint on the server that can serve files from fs.FS implementations. - Add source.FilesInfo, which returns a source.Info that links to /files paths. Use it to implement the SourceInfo method of the local ModuleGetters. - Add a SourceFS method to the local MethodGetters so they can tell the server how to serve their files. - Improve the tests in cmd/pkgsite to verify the source links. Fixes golang/go#47982 Change-Id: Iff42bad15c4abaf408a364e58b7c0f0dad60b40d Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347932 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-09-07internal/fetch: get source info from module getterJonathan Amsterdam
Make it the ModuleGetter's responsibility to provide links to source files, instead of passing around a separate source.Client. This will allow getters that fetch from disk to serve those local files. For golang/go#47982 Change-Id: I6c7cd7890b03ad61d9410850a510c1c9a63f1c9a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347749 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-08-26internal/fetchdatasource: handle build contexts properlyJonathan Amsterdam
GetUnit returns a Unit with its Documentation set to a matching BuildContext. This is the same behavior as postgres.DB.GetUnit. For golang/go#47780 Change-Id: I2bc23b7bc5a006e78bec54f6f3229e59ab5a03ef Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/345269 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org> Reviewed-by: Julie Qiu <julie@golang.org>
2021-08-26internal/fetchdatasource: GetUnitMeta returns NotFound on missing packageJonathan Amsterdam
If a module exists but the package path is not in it, return NotFound. For golang/go#47780 Change-Id: If3a6602df4b99c8470020e8538e01c685880d86d Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/345251 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org> Reviewed-by: Julie Qiu <julie@golang.org>
2021-08-26internal/fetchdatasource: ignore latest-version errorsJonathan Amsterdam
A FetchDataSource may be configured with a proxy for latest-version information while still being used for local modules. So ignore latest-version errors. For golang/go#47780 Change-Id: I30448e133faec80002194df59e6f2db449f80625 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/345249 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org> Reviewed-by: Julie Qiu <julie@golang.org>
2021-08-25internal/fetchdatasource: rename from datasourceJonathan Amsterdam
Rename the package and type by including the word "fetch," to describe what it does and distinguish it from other DataSources. For golang/go#47780 Change-Id: I7c674d8ba3dc16084a857039b0cc2b3147f27a29 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/344953 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 19:36:08 +0000