aboutsummaryrefslogtreecommitdiff
path: root/internal/postgres/insert_module_test.go
AgeCommit message (Collapse)Author
2026-03-26all: fix rangeintHana Kim
Change-Id: I58ed5e7d2a3bbc13bc389ac7239737e18856dbfe Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/753427 Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Ethan Lee <ethanalee@google.com> kokoro-CI: kokoro <noreply+kokoro@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2026-03-25all: add LatestVersion to ModuleInfoEthan Lee
- Add LatestVersion field to internal.ModuleInfo and update tests accordingly. - Changed some test formatting to improve readability. Change-Id: I1238e54614ef276d219b31d125cd2cad6b5a66f7 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/756901 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> kokoro-CI: kokoro <noreply+kokoro@google.com> Auto-Submit: Ethan Lee <ethanalee@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2025-11-17Revert "internal/postgres: utilize num_imports column in insert_module"Ethan Lee
This reverts commit 28eed86815232e48f5e757569fcd061f858142d4. Reason for revert: migration takes too long Change-Id: I279e62342c79d2604cda4e7c9819bdaa8aac752c Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/721101 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> kokoro-CI: kokoro <noreply+kokoro@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
2025-11-13internal/postgres: utilize num_imports column in insert_moduleEthan Lee
- Avoid recomputing number of imports for every query in getUnitWithAllFields. For golang/go#76284 Change-Id: I7f7d683efb1b2ff49e33681802896a65b1354d25 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/720261 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com> Auto-Submit: Ethan Lee <ethanalee@google.com> kokoro-CI: kokoro <noreply+kokoro@google.com>
2025-10-23internal/postgres: add skip-symbols list to saveModuleEthan Lee
- Modules in the skip-symbols list will not have symbol information inserted into postgres. Change-Id: Ia52b9eb2d7cab662b967fa4a74c40cced0e44be4 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/713060 Reviewed-by: Jonathan Amsterdam <jba@google.com> Reviewed-by: Peter Weinberger <pjw@google.com> kokoro-CI: kokoro <noreply+kokoro@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-09-04internal/postgres: insert module path even if not a packageJonathan Amsterdam
When inserting the paths of a module into the paths table, we were inserting all the package paths and the v1 module path (the module path without "/vN" suffix). That is almost always sufficient. But if a module has a version suffix and is not itself a package, then we were not inserting its path. The module path is needed by insertSymbols, so this was causing that function to fail. Add the module path, refactor the code to simplify it and allow testing, and add a test. (For a vN module the code also adds the v1 package path for every package. I think this is unnecessary, but it's too risky to remove it.) Change-Id: I8caa3dd945692a4ec0a1128199237de6b77aa719 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/609117 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com> kokoro-CI: kokoro <noreply+kokoro@google.com>
2023-08-23all: remove arbitrary hard-coded timeouts in testsBryan C. Mills
If a test times out, that implies that it got stuck on something. By default, the Go testing package dumps goroutines when its own timeout is passed, which prints a goroutine dump, helping to reveal what was stuck. Adding an arbitrary timeout on top of the testing package's own timeout is, in my experience, almost always counterproductive. If the arbitrary timeout catches a real hang, it causes the test to fail instead of dumping goroutines, making it much harder to see what was stuck. On the other hand, if the timeouts are set aggressively enough to make the test fail early, they are often too aggressive for CI testing, causing flakes that then have to be triaged on an ongoing basis. On balance, the value of saving a minute or two for developers who have introduced a hang is not worth the cost of suppressing debugging information and causing flakes that have to be triaged. Fixes #61556. For #59347. Change-Id: I0263d0d9b18283470f100e5a0155818b87b5312f Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/521837 TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> kokoro-CI: kokoro <noreply+kokoro@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2022-06-17many: re-enable staticcheck and fix problemsJonathan Amsterdam
Most of the checks were about io/ioutil. There were a couple of other minor ones. I didn't address the check for strings.Title; instead, I turned off that check globally with a staticcheck.conf file. Change-Id: I286a6894fb1fd891818ab9e451c891f52a3828fc Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/412675 Reviewed-by: Jamal Carvalho <jamal@golang.org> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com>
2022-03-07internal/database: use generics for Collect functionsJonathan Amsterdam
Reimplement some reflection-based collection functions to use generics. In some cases we still need reflection, but at least we can provide a type-safe wrapper with generics. Change-Id: Id95949a7a22ee687166ecdfc1191150d79568889 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/389657 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-11-15internal/licenses: remove Metadata.OldCoverageJonathan Amsterdam
We don't need the OldCoverage field anymore. All DBs have only the new Coverage information. Change-Id: Ib2d8e0cdacdc2f7b6514a8a5310d6905be15136c Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/363954 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-10-15internal/{postgres,frontend}: support modules with zero commit timesJonathan Amsterdam
It's valid to have a module with a zero commit time (as reported by the proxy .info endpoint). That can happen if the module is served from a vanity URL with a go-import meta tag of type `mod`. Process those modules and display them with a publish date of "unknown." Fixes golang/go#48952 Change-Id: I8efdebfa27c8b6ed68c183bac77d87ee715f8262 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/356129 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-09-14internal/postgres: change UpsertModuleVersionStates to updateJulie Qiu
UpsertModuleVersionStates is changed to UpdateModuleVersionStates. There should never be a situation where UpsertModuleVersionStates is called and a row does not already exist for that module. If that happens, an error is now returned. For golang/go#46985 Fixes golang/go#39628 Change-Id: I357396cee6eb743513ae249609f76f4cd4c19e9b Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/341860 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com>
2021-08-24internal/fetch: populate Unit.BuildContextsJonathan Amsterdam
Populate the lists of build contexts when fetching a module. Currently we only do this when reading from the DB. We want DataSources that rely only on fetch to have the right value. For golang/go#47780 Change-Id: Ie85fba8ccb318bb4ed5c1f2153b0ba4216358022 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/344670 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-08-16internal/postgres: don't insert into search if a prefixJonathan Amsterdam
In ReconcileSearch, don't insert a module into search_documents if there is already a module path that is a suffix of the current one. This avoids cases like the following: 1. github.com/araalinetworks/api, which has a golang package, is fetched and added to search_documents. 2. github.com/araalinetworks/api/golang is fetched and added to search_documents, correctly replacing github.com/araalinetworks/api because it is a longer module path. 3. Another version of github.com/araalinetworks/api is fetched. It is bad, so it isn't inserted in the usual way, but ReconcileSearch re-inserts the latest good version, which overwrites the longer module path inserted in step 2. Change-Id: If8ae59acfde7838c0ea48882d6beb0a457e125db Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/342469 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org> Reviewed-by: Julie Qiu <julie@golang.org>
2021-08-16internal/postgres: rename ReInsertLatestVersionJonathan Amsterdam
Rename to ReconcileSearch, which better describes its purpose. Change-Id: Ib7562e6bf511ff1ad88129619feec110c81168d3 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/342169 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-08-16internal/{worker,postgres}: check current module for alternativeJonathan Amsterdam
When reconciling the contents of search_documents when we insert a module, consider whether the version currently being processed is alternative. Previously, we didn't consider the current module because we only checked the DB, and the status code for the current module hasn't been inserted yet. Change-Id: Ibeb02d7f56341119e2bf4f20f148b345b2b82828 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/341868 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-08-13internal/{worker,postgres}: remove alternative modules from searchJonathan Amsterdam
When processing a module, if it is an alternative module, remove it from search_documents and imports_unique. Change-Id: I3fd12e8522331b8a4f060c9576d97c0ee710785d Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/341854 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com>
2021-08-13internl/postgres: move delete functions into delete.goJonathan Amsterdam
Pure code motion. Change-Id: I80055dbb95f2341c72d3716b74e08335bdaa6fd4 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/341851 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-08-12internal/postgres: remove a fully retracted module from searchJonathan Amsterdam
If all the versions of a module have been retracted, remove it from the search_documents table so it doesn't appear in search. Also remove it from the imports_unique table so it doesn't contribute to the imported-by counts of other modules. For golang/go#47590 Change-Id: I76a6e33a1291abe4a19f1b3521c9dc8ebed4f110 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/341591 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-07-21internal/{database,postgres}: export CollectStringsJonathan Amsterdam
Export the CollectStrings function and move to the database package so it can be used elsewhere. Change-Id: I1032e4f5042209e42e8f0de9cebfb48653d56f69 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/336209 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-07-13internal/postgres: fix bug in isAlternativeModuleJulie Qiu
At the moment, isAlternativeModulePath does not filter on the modulePath argument, which means that it reports whether the latest version of a random module has status 491, and not the specified module. This bug is fixed and a test is added. Change-Id: I7a0feb960e1f1e6b7de2fb7a82ec0ea2fdcce738 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/332378 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-06-18many: remove read-imports experimentJonathan Amsterdam
Remove the read-imports experiment and all references to the package_imports table. Change-Id: If926c47e1d01b3ab0f31bb6efb9c2234418fca1b Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/329229 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-06-14internal/postgres: don't touch package_importsJonathan Amsterdam
If the read-imports experiment is on, replace all references to the package_imports table with the imports table: don't write into or read from package_imports. Change-Id: I02631aba574b9938a44b06bce855997606984939 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/327749 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-05-05internal: enable symbol history for stdlibJulie Qiu
Symbol history for standard library packages are now always displayed. For golang/go#37102 Change-Id: I323c7735726a22d4caac39bebd2e8d912fd03aad Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/317013 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-04-29internal/postgres: read only one doc source from DBJonathan Amsterdam
If GetUnit is passed a build context, it will read documentation source only for that build context. It still reads all the build context values, just only one serialized blob of file ASTs. This sped up getUnitWithAllFields by about 2x on net/http, using detail-stats against the staging DB from a local frontend. Change-Id: Ib8e988940b7f8f8ef2c270129c4338bb3d56b9eb Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/312269 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-04-21internal/postgres: remove do-not-insert-new-documentationJulie Qiu
Change-Id: I4b5e4b4e6556f6f724dd01c08b7014c30b10a2b3 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/310313 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-04-20internal: add do-not-insert-new-documentation to testsJulie Qiu
All tests that affect the documentation table are updated so that experiment do-not-insert-new-documentation is active. Change-Id: I3672a7b18e50bc1cbe4820efe0e6e13ff9687596 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/310811 Reviewed-by: Jonathan Amsterdam <jba@google.com> Trust: Julie Qiu <julie@golang.org>
2021-04-13many: remove "retractions" experimentJonathan Amsterdam
For golang/go#43265 Change-Id: Iac39814ce532adf5846bb768802a46ad7a77fa84 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/309609 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-04-08internal/postgres,etc: finish MustInsertModule cleanupJonathan Amsterdam
Now that there are no callers of MustInsertModule, rename MustInsertModuleLatest to MustInsertModule. We can also remove MustInsertModuleLMV. Change-Id: Ieb554ee32696c168be4cc0a14ecece9f4c6b91b4 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/308271 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-04-08internal/postgres: continue MustInsertModule cleanupJonathan Amsterdam
Remove calls to MustInsertModuleLMV. It helped to reorganize TestGetLatestUnitVersion to be clearer about the go.mod contents of the various module versions. Change-Id: I7fa41bd742cf8bba2b5274172ad5b79fc22e73a9 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/308270 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-04-07internal/postgres: use MustInsertModuleLatestJonathan Amsterdam
For most internal/postgres tests, update latest-version information when we insert a module. That is more like what actually happens on the worker, and some features now rely on it, so we should do it everywhere. Some tests still use a custom go.mod file, so we can't switch to MustInsertModuleLatest for those, because it uses a minimal go.mod file. Change-Id: Ie5030aee4ed9c837931fddb8757e37bf6373a8c7 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/307871 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com>
2021-03-22internal/postgres: add imports_unique to ReInsertLatestVersionJonathan Amsterdam
The ReInsertLatestVersion method now changes imports_unique as well as search_documents. For golang/go#44710 Change-Id: Ie5dcfe8e362313be76f0ad4958882e37fc725e44 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/303313 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-03-22internal/postgres: add ReInsertLatestVersionJonathan Amsterdam
New latest-version info can retract the current latest good version, resulting in an older version becoming the new latest good version. Add a function that updates the search_documents table when that happens. A later CL will also update imports_unique, and will call this function from the worker. For golang/go#44710 Change-Id: I09569e27d384ed7defb06a4e81facdb96c22caeb Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/303312 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org> Reviewed-by: Julie Qiu <julie@golang.org>
2021-03-19internal/postgres: reimplement alternative-module checkJonathan Amsterdam
When inserting a module, we skip inserting into search_documents if the module path is an alternative one. This CL factors out that test, and also reimplements it in terms of the latest cooked version. Previously, any later version with a 491 status would classify the path as alternative, but this version uses the latest cooked (non-retracted) version's status to decide, allowing module authors to fix mistakes. Change-Id: I3588f3df6a1a4dd4db59333314c34167eda51df1 For golang/go#44710 Change-Id: I670d9085800fed198e6dd6469f1a8d5728c5f039 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/303309 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-03-16internal/worker,etc.: improvements to latest-version infoJonathan Amsterdam
- Fetch and update the latest-version info before processing a module rather than after, so the information in the DB matches what InsertModule is using. - Pass latest-version info to InsertModule so it can use it to decide whether the version being inserted is the latest version. - InsertModule writes the good latest version to the latest_module_versions table while holding the advisory lock, avoiding a race condition where two different versions both think they're the latest. For golang/go#44710 Change-Id: Id5e8fa7a384ec6d4d86257362fd33a8fcd5215f7 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/301929 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-03-02internal/postgres: group UpsertModuleVersionState argsJonathan Amsterdam
Organize the many arguments to UpsertModuleVersionState into a struct. Change-Id: Ia0266af623ca2be3961fde6ba8e15ba5540cf36b Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296810 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-03-01internal: do not display README at /stdJulie Qiu
The README is no longer displayed at pkg.go.dev/std. It is also no longer stored in the database when a std module is fetched. For golang/go#44356 Change-Id: I6b1605389bc3c97694b6ec2b06e28006a38be999 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296950 Trust: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-02-25internal: embed ModuleInfo into UnitMetaJonathan Amsterdam
UnitMeta had most of the fields of ModuleInfo, but did not embed it. Now that we have added four more fields to ModuleInfo for deprecation and retraction, it makes sense to embed it rather than duplicate those fields. For golang/go#43265 For golang/go#41321 Change-Id: I20e2b922b49c7873a5535745d644631123de37cd Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296209 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-02-23internal/discovery: add retraction fieldsJonathan Amsterdam
Add fields to hold retraction information. Also, switch deprecation to a pair of bool and string rather than a pointer, because it's clearer. For golang/go#44437 Change-Id: Ica771c5ec8e5c0abeec16f9d998339ef60b401a5 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/295195 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-02-17internal/sample: improve treatment of sample licensesJonathan Amsterdam
To avoid sharing between tests, which can lead to race conditions, create new sample license slices and structs each time we need them. This uncovered some problems in the postgres tests. - The UpsertModule test was changing the top-level module license, but didn't propagate that change through all the units. We sample.ReplaceLicense to address that. - The checkModule was filling in missing license with defaults, instead of the correct data. Change-Id: I6074914155eb33d6ed6882f81a9d57b20135e422 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/292929 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-02-12internal/postgres: parallelize testsJonathan Amsterdam
Run tests in parallel on separate databases. With four DBs, the time to run all the package's tests is cut in about half, from 50s to 25s (the time varies from run to run). Instead of a global variable holding a single DB, we set up a channel populated with DBs. Each test receives from the channel to get a DB to use, and sends it to the channel when done. Because a test may have to wait to get a DB, we can't set an individual timeout for some tests because acquiring the DB can take some time. Individual test timeouts aren't really important, because `go test` will time out the entire run after 10 minutes anyway. The only test that can't be run in parallel is TestSearch, because it reads metrics before and after each sub-test to compute a delta, thereby implicitly assuming that no other Search calls are running in parallel. Change-Id: Icb35dbd2f146e27d82d4eef81343cf9725252155 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/291449 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-02-12internal/postgres: add tests for inserting non-release versionsJulie Qiu
Test are added to TestInsertModulde to verify that a prerelease or pseudoversion module is inserted correctly. Change-Id: I604d3d94cfb7bd16c7869ae45b7c55d7b8d2bf4e Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/291649 TryBot-Result: kokoro <noreply+kokoro@google.com> Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-02-11internal/postgres: InsertModule returns isLatestJonathan Amsterdam
InsertModule reports whether it just inserted the latest version of a module. We'll use this information later to determine whether to invalidate the cache. Also, rewrite tests to use postgres.MustInsertModule. For golang/go#44217 Change-Id: I8bed8aa8e86a347a4add85c2b16c549b7ad740f0 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/291270 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-02-11internal/postgres: add MustInsertModule for testingJonathan Amsterdam
Simplifies frequent calls to InsertModule. Change-Id: Iafcf5ec8d71eb9ab541ceecb2d85eb0d9b5119b1 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/291269 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-02-09internal/postgres: read and write deprecated commentJonathan Amsterdam
Insert the deprecated comment into the modules table, and read it back out. For golang/go#41321 Change-Id: Ib4f2c8ceb70e28940b18f52d25b2a0d05c57648d Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/290098 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-02-02many: allow for multiple documentations for a UnitJonathan Amsterdam
Change internal.Unit.Documentation from a single internal.Documentation to a slice, so that a Unit can have doc for multiple build contexts. However, don't use this ability yet: fetch takes only the first build context it finds, postgres inserts only the first, frontend renders only the first, etc. For golang/go#37232 Change-Id: I17fd1ff2378e0862c029c8077057c207001f6136 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/288217 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org> Reviewed-by: Jamal Carvalho <jamal@golang.org>
2021-01-14internal/licenses: use new licensecheckJonathan Amsterdam
Use the new version of licensecheck, which is more precise and has other improvements. The new version uses regexps instead of fuzzy matching to categorize licenses. That makes it much stricter than the old version, so we need many exceptions in order to match the same licenses we did before. It is also intolerant of various corruptions like truncation and search-and-replace errors. We observed many of these, and filed bugs against the offending repos where possible. We validated that except for corrupted licenses, this package accepts the same licensesn as it did before for all latest-version modules. The new version uses SPDX identifiers where they exist, so we no longer need to map licensecheck names to SPDX IDs. We also need include many variant SPDX identifiers in our list of redistributable licenses (e.g. "GPL2.0-or-later"). The new version dispenses with "-Short" and "-Header" variants, so we don't need to canonicalize the names. Change-Id: I5c1aaa2bdf745aff6ebb8c121fa63d6550931069 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/283654 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-01-13internal/licenses: rename Coverage fieldsJonathan Amsterdam
Coverage -> OldCoverage NewCoverage -> Coverage Change-Id: I05b41e1525700b3539dad9894b6b678f6c88c67d Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/283653 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-01-13internal/postgres: save and restore new licensecheck CoverageJonathan Amsterdam
Add a NewCoverage field internal/licenses.Metadata to support the upcoming change to the new licensecheck. When saving to the DB, save the NewCoverage field if it is populated; otherwise save the original Coverage field. When reading the JSON for the coverage field from the DB, distinguish old and new Coverage structs and populate one of the two fields of the licenses.Metadata value as appropriate. Change-Id: I5be5623fbaec668265e1e30a878864cf4aef2ae6 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/283652 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2020-12-28internal/postgres: list non-redistributable packagesJulie Qiu
Currently, non-redistributable packages are not listed in the directories section, since they won't have a row in the documentation section. These are now listed with an empty synopsis. Change-Id: Id787b91cd58c698daf547e9dff8921df5c0f097f Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/280094 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jamal Carvalho <jamal@golang.org>
generated by cgit v1.3 (git 2.53.0) at 2026-04-15 07:43:23 +0000