aboutsummaryrefslogtreecommitdiff
path: root/internal/stdlib/stdlib.go
AgeCommit message (Collapse)Author
2026-03-26all: run go fix -slicescontainsHana Kim
Change-Id: I14479d9e612dfa7eed9188206746af5b51c10201 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/753428 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Jonathan Amsterdam <jba@google.com> kokoro-CI: kokoro <noreply+kokoro@google.com>
2024-02-21internal/stdlib: remove support for special branchesJonathan Amsterdam
Remove support for the dev.boringcrypto and dev.fuzz branches. They are no longer relevant. For golang/go#65205. Change-Id: If0e7060b148fe3ebff9139a6ce2f418b928a0bd6 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/557415 kokoro-CI: kokoro <noreply+kokoro@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2024-01-08cmd/pkgsite: fix case where we don't have runtime.GOROOT()Michael Matloob
If pkgsite is built with -trimpath, runtime.GOROOT() returns an empty string, so we don't know where the local go stdlib is. In that case, don't try to serve a local stdlib and instead fall back to the stdlibZipModuleGetter. Also fix two issues that show up when using the stdlibZipModuleGetter. First, we try to prefetch fetch.LocalVersion of the standard library when the server starts. Instead, prefetch latest which will have the correct behavior with both the stdlib zips and local stdlib. Second, when we're trying to fetch the stdlib using git make sure that we're on another branch than the one we're trying to fetch into because git won't let us fetch into the branch we're on. Fixes golang/go#64903 Change-Id: I9dfb22d50b7738080490ce2682e0cf187c16d2d1 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/554195 TryBot-Bypass: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-07-19internal/stdlib: remove dependency on github.com/go-git/go-git/v5Michael Matloob
Instead of using the in-memory git library implementation, just run all git operations using git commands. Add the MustHaveExecPath function from the internal/testenv library in the Go repo to our internal/testenv (with a modification to drop MustHaveExec because we don't really need it) so that we can check that git exists on the test platform before running tests that use this code because now we need a git binary to do git operations. For #61399 For #61341 Change-Id: I53441ee1638353a23935ba6a5ac70e2965b546f9 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/510415 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Jamal Carvalho <jamal@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> kokoro-CI: kokoro <noreply+kokoro@google.com> Run-TryBot: Michael Matloob <matloob@golang.org>
2023-07-18internal: remove repetitive wordscui fliter
Change-Id: Ie5a688c08ba46018f0fa8f5a1af495e9c29fd684 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/509695 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Robert Findley <rfindley@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> Auto-Submit: Ian Lance Taylor <iant@golang.org> kokoro-CI: kokoro <noreply+kokoro@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Robert Findley <rfindley@google.com>
2023-07-12internal/stdlib: update to account for .0 major version suffixesRob Findley
Starting with go1.21.0, the first version of new major Go versions will include the .0 suffix (compare go1.20). Update pkgsite logic to account for this switch, and add unit+integration test support. Fixes golang/go#60373 Change-Id: Ibdac8a3413a48f925427a5aae366bed2f691cfa6 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/508937 Run-TryBot: Robert Findley <rfindley@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com>
2023-03-15internal/stdlib: fix versions pages for packages only in masterJamal Carvalho
Fixes golang/go#58594. Change-Id: I13c9879a12ae070d1aa07d8e152bafa07cfb7952 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/474536 TryBot-Result: kokoro <noreply+kokoro@google.com> Run-TryBot: Jamal Carvalho <jamal@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2022-04-11all: gofmtRuss Cox
Gofmt to update doc comments to the new formatting. For golang/go#51082. Change-Id: Ia9e71e7ecac75822ff43d6c7e60f512442a5fa50 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/399617 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2022-01-13internal/stdlib: refactor go repo logicJonathan Amsterdam
Add a goRepo interface, implemented for remote, local and test repos. Functions of stdlib use a global instance that interface instead of if statements. Also, improve and rewrite some tests. For golang/go#50229 Change-Id: I73641813e6c7a6bb9667f44a672e37dce964b17a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/378094 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2022-01-12internal/stdlib: get versions from local repoJonathan Amsterdam
When a local repo path has been specified, Versions will read refs from that repo instead of the remote. I checked with strace that visiting a stdlib package did not hit the network after this change. Fixes golang/go#50575 Change-Id: I3e61ab5a3f6e1be85c437412a0e4d4149b7f1f3c Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/378054 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-09-14internal/{worker,stdlib}: /fetch-std-master: avoid cloning go repoJonathan Amsterdam
On each call of the /fetch-std-master endpoint, we cloned the Go repo for each supported branch to determine its current commit. Although not a big deal, that is a bit wasteful and it also would occasionally fail obscurely. Instead we use git's remote list feature to get all the repo's refs in one call. We then compare the hashes with the stored pseudo-versions to see if the branch has moved. Change-Id: I1d70e63668fcf6d35b90b9c231255faec614a04e Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/349753 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-09-03internal/stdlib: support opening local repoJonathan Amsterdam
By calling stdlib.SetGoRepoPath, users can point to a local clone of the Go repo for stdlib to use. This can save download time. The intent is to use it for cmd/pkgsite. Change-Id: I20d89f283b7b0b4f333599846eee1038d09cb50e Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347404 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-09-03internal/stdlib: refactor getGoRepo, add clone testsJonathan Amsterdam
Refactor getGoRepo to handle both test and cloned repos. This will enable adding a third option, getting the Go repo from a local copy. Also add a test for checking the contents of a cloned repo, to make sure the clone logic is correct. Flag-protect it because it's slow and hits an external server. Change-Id: I9de50bf12fccbd4fa35aa0566bdee3cddb25edf8 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347403 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-08-25internal/frontend: fix github.com/golang/go redirectsJulie Qiu
Previously, any URL with the prefix /github.com/golang/go redirected to their corresponding package page (for example, /std or /cmd/go). However, this also redirected paths like /github.com/golang/gofrontend. This bug is now fixed and a test is added. Fixes golang/go#47957 Change-Id: Iabcc2c57ef9f5365d93911a4971addb8bb5a903d Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/344989 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-08-23internal/{fetch,worker,stdlib}: remove stdlib.ZipJonathan Amsterdam
For golang/go#47834 Change-Id: I81acf4fdc51a19352bfa7fb0cf9a2c22bb42a65b Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/343966 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-08-23internal/stdlib: add ContentDir functionJonathan Amsterdam
Add a function that returns the content directory of the stdlib module. Remove the part of the test that checks go.mod; it was never executing, because there is no go.mod file in any of the repos being tested. For golang/go#47834 Change-Id: Idc982620f6736ec60fe9a295f0372fd272745457 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/343965 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-08-19internal: add support for std@dev.boringcryptoJulie Qiu
Change-Id: Ibe5383c8c9b9ce771b677c2e33a0c2c85621df6d Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/343218 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-07-12internal: support std@dev.fuzzJulie Qiu
It is now possible to fetch std@dev.fuzz and view /std@dev.fuzz on the frontend. For golang/go#46910 Change-Id: I2fd7259840ba32ff35ca323d07412988e533ffba Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/330413 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-07-12internal: derrors wrap VersionsJulie Qiu
The proxy and stdlib "Versions" functions are wrapped with derrors.Wrap to make it easier to debug errors. Change-Id: Ia9f8c9914b5d5495083f78521af66cf8aa63778f Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/333233 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-06-07internal/stdlib: use version constantsJulie Qiu
The strings "latest", "master" and "main" are replaced with constants. Change-Id: Icb44678e54c0a5834b9f4aebcde6db39761c3c6f Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/325391 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-06-07internal/stdlib: support fetching latest versionJulie Qiu
It's now possible to fetch std@latest from the worker. Change-Id: Iacdd376af571fa8d610f5afe3f6a9ed77fc91206 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/325309 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-05-12internal: change stdlib source repoJulie Qiu
The stdlib source repo is changed to https://cs.opensource.google/go/go. Fixes golang/go#44832 Change-Id: Ie8060ab1cabccbcf9ff20c0eec42c243c1feaf9c Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/318873 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-03-01internal: do not display README at /stdJulie Qiu
The README is no longer displayed at pkg.go.dev/std. It is also no longer stored in the database when a std module is fetched. For golang/go#44356 Change-Id: I6b1605389bc3c97694b6ec2b06e28006a38be999 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296950 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-02-22internal/stdlib: fix Zip for master versionJulie Qiu
An error is fixed when creating the zipfile for std@master, which prevented the module from being fetched. A test is also added. For golang/go#44390 Change-Id: I279146aef10311822f47f16259a2223c879140c2 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/294831 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-01-27internal/stdlib: return pseudo-version for masterJonathan Amsterdam
When fetching std@master, create a pseudo-version so the module is stored under the commit that is the current value of master. Fixes golang/go#43890 Change-Id: I99af482b1cc16a7bbb396c5b072f8e8d60a71b68 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/287213 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-01-25internal/stdlib: fix getGoRepo for requests to masterJulie Qiu
getGoRepo currently looks for a "master" tag when fetching the master repo, instead of at the master branch. This is now fixed. However, stdlib.semanticVersion still returns "master" when the requestedVersion is "master", instead of returning a resolved semantic version, so a fetch request for std@master will still result in a 500. A comment is added to fix this in the future. In a future CL, a fake pseudoversion will need to be constructed in order to resolve std@master to a semantic version. For golang/go#43890 Change-Id: I9e68b91d8c9e0bea6f6751d834853b852f0d532b Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/286153 Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com> Trust: Julie Qiu <julie@golang.org>
2021-01-25internal: resolve default branch versions for each fetchJulie Qiu
The target for @main and @master is maintained in the version_map table, while @latest and a specific semantic version is determined using the modules and units tables. As a result, these versions can be out of sync. Resolve the master/main version when a module is fetched and disableProxyFetch is off, so that the targets are up to date with the new version. Fixes golang/go#43096 Change-Id: I86dde167639c0f180ab67b0d3243316240956fc4 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/286212 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-01-19internal/stdlib: added ability to fetch master versionSai Kiran Dasika
Added the ability to fetch the master version of a module Fixes golang/go#39973 Change-Id: I66ea0dc3a2249ac4757a3d3b478c428f13c2ec89 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/279292 Reviewed-by: Jonathan Amsterdam <jba@google.com> Trust: Julie Qiu <julie@golang.org>
2020-12-15internal/source,stdlib: change stdlib source repo to go.googlesource.comJonathan Amsterdam
Link to source files in the canonical Go repo, go.googlesource.com/go. Continue to serve raw content (for images) from the GitHub mirror, since we can't from googlesource.com. Fixes golang/go#43180 Change-Id: Iac513d6481bcdb73ae73eed40d33518ffd3a0158 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/278293 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com>
2020-09-18internal/stdlib: Zip takes resolved versionJonathan Amsterdam
In the only call to this function, we already know the resolved version. For golang/go#41452 Change-Id: I96e5ebdf5acb0aeec45288b25b555e5c6cbb52ee Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/255757 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2020-09-18internal/fetch: refactor to get info separatelyJonathan Amsterdam
Reorganize FetchModule into two functions. First, one calls GetModuleInfo to get some prelimary, cheap information, including the size of the zip. Then one calls FetchModule with the result of GetModuleInfo. This allows us to make decisions based on the zip size before downloading the entire zip. For golang/go#41452 Change-Id: Ifbffa5882488e31a72e78670caf69c3d97a80546 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/255738 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2020-08-07internal/stdlib: add semanticVersion functionJulie Qiu
Logic for resolving the semantic version in Zip is moved to the function semanticVersion. For golang/go#39973 Change-Id: I5c9215a4cc326d2a9c5db79f33509656ec0b0065 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/247398 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-08-07internal: support fetching the latest version of stdlibJulie Qiu
This change adds support for resolving the latest version of the std module, using stdlib.Zip. Updates golang/go#39973 Change-Id: I0e27303bf2290f67c9dbc80fadf66b2e5c73e886 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/241604 Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-07-27internal/stdlib: fix stdlib README for new data modelJulie Qiu
The overview tab for the stdlib package is currently displaying src/README.vendor, instead of README.md, when the use-directories experiment flag is on. This is because based on the way the std module zip is constructed, both README.md and README.vendor are present at the root for version v1.4.0-beta.1 and higher. The fetch process will then choose README.vendor to include, since it comes after README.md alphabetically. The README.vendor file is no longer included in the std zip. For golang/go#39629 Change-Id: I1c9d8fcd1776f7b58be08f5f1ec1dc1b735e736d Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/244608 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-07-27internal/stdlib: sort testRefsJulie Qiu
testRefs are sorted based on the Go version and other tags. Change-Id: Ia190e661bb93ad69d025f976dc2e2597fa9650c4 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/244609 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-07-07internal/stdlib: use less-than for Compare comparisonJonathan Amsterdam
Change-Id: Iadaf802c8931eaf1c3d91afc29d39fd142f2f1a6 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/241071 Reviewed-by: Julie Qiu <julie@golang.org>
2020-06-17internal/stdlib: remove uses of FromHTTPStatusRob Findley
We should endeavor to eliminate FromHTTPStatus, and just use the internal error semantics of the derrors package. I'm looking at other uses, but for now eliminate these trivial uses: just use InvalidArgument directly. Change-Id: I55149b43031829953c9ec2a22b20f933f343b933 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/238438 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-06-16mod: target Go 1.14 and fix the go-git depsDaniel Martí
src-d as a company went bankrupt about six months ago, and their Go modules now survive under a new GitHub organization. Since the paths had to change, they jumped both to v5. The semver jump doesn't otherwise affect the API or backwards compatibility, so no other changes are necessary. Change-Id: I60f153799dd613bf15a407f9fedd2905fa93b469 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/237902 Reviewed-by: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-05-18internal/stdlib: clarify doc comment for ContainsJonathan Amsterdam
Change-Id: Ic17226b2ae53e17ed96ee288e29306845252b3e8 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/747785 Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-04-23all: rename module to golang.org/x/pkgsiteJulie Qiu
golang.org/x/discovery is renamed to golang.org/x/pkgsite. When the repository is open sourced, it will be hosted at go.googlesource.com/pkgsite. Change-Id: Ifc3b45b771a385b99179e785447f2a87afcacf87 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/724273 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-06internal/stdlib: return InvalidArg for unknown versionJonathan Amsterdam
A fetch of an unknown stdlib version results in a 400 error, since the input is bad. Previously we returned 500. Fixes b/150649087. Change-Id: Id48c9d699dc7a132d2185f068dc826d56abb7bd6 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/681866 Reviewed-by: Robert Findley <rfindley@google.com> CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com>
2020-03-27internal/fetch: check for mismatched pathJonathan Amsterdam
Read the go.mod file from the proxy, and check if its path differs from the module path. If they do differ, fail with an AlternativeModule error, and record the relationship between the two paths in a table for future use. This change means that the discovery site will not serve modules that: - are case variants of the canonical path e.g. github.com/Sirupsen/logrus vs. the correct github.com/sirupsen/logrus - bypass the vanity import path e.g. github.com/gonum/gonum vs. the correct gonum.org/v1/gonum - are "soft forks" of the module's repo: forks with an unchanged go.mod path e.g. github.com/alice02/kubernetes vs. the correct k8s.io/kubernetes Change-Id: If78c94744440112f5720750885fe10f6a7dc7ab8 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/623922 Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-03-27all: use golang.org/x/modJonathan Amsterdam
Use the semver and module packages from golang.org/x/mod. Remove them from internal/thirdparty. We also rename Encode{Version,Path} to Escape{Version,Path} because golang.org/x/module does that. Fixes b/133841935. Change-Id: I0e52c6dd7109fbd488a030c750cdc06736356868 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/619244 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-03-27internal: add a testing directoryJonathan Amsterdam
Put all test-related packages there (sample, testhelper, integration). production risk: none Change-Id: I06673aaa5c3fd6966410f1c1ad2b34e4f2d88f94 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/599080 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com>
2020-03-27internal/stdlib: handle go1Jonathan Amsterdam
Handle the idiosyncratic mapping of go1 to v1.0.0. The following frontend paths changed behavior: - std@go1?tab=versions and archive/tar@go1?tab=versions both work now - std@go1.0 and archive/tar@go1.0 now return 4xx Fixes b/144364761. Change-Id: I2345faeedb1e418028422feaf7d9ad0233bc08ff Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/597504 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-03-27internal/postgres: ignore self-imports in imported-by countsJonathan Amsterdam
Do not count package A as importing package B if they are in the same module. Change-Id: Ic0f98af8d72b8fff29e105779c6eb619214335e9 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/596525 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-03-27internal/frontend: fix major version for stdJonathan Amsterdam
Display the major version as "go1", not "v1". Also, fix the tooltip. Fixes b/143529585. Fixes b/143613230. Change-Id: I3379e8295a90f79c375f6e54e3593eee2391a335 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/589902 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-03-27internal/stdlib: export VersionForTagJulie Qiu
VersionForTag will be used in internal/frontend in the next set of CLs for the new URL design. Updates b/140191811 Change-Id: I85c11ac0d6e8c5e8c1040f2dbdc6a385a3e4ee09 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/567617 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-03-27internal/source: handle meta tagsJonathan Amsterdam
Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-03-27internal/source: a package for references to sourceJonathan Amsterdam
Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>