aboutsummaryrefslogtreecommitdiff
path: root/internal/postgres/version_map.go
AgeCommit message (Collapse)Author
2021-02-12internal/postgres: add a stack to all wrapped errorsJonathan Amsterdam
Add a stack trace when we wrap an error from the DB. These traces can be sent to the error reporting service. For golang/go#44231 Change-Id: I096cdec4e97a6dcb0b7eb2ccdb4c955e1a0f4ccd Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/291492 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
2021-01-20internal: handle 290s on 404Julie Qiu
When a path 404s but the top level module exists, return the relevant response text. Also update the response text for clarity. Change-Id: I9425355daf4ad3f88c58261cc6f778f79f0fc003 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/284585 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2021-01-06internal/postgres: add GetVersionMapsNon2xxStatusJulie Qiu
GetVersionMapsWithNon2xxStatus is added, which returns the version_maps correlating to a given path with a 4xx or 5xx status. Change-Id: I99d52bbc38183939331e2d991a3107635036ea5b Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/281673 Trust: Julie Qiu <julie@golang.org> Run-TryBot: Julie Qiu <julie@golang.org> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-07-27internal/postgres: fetch version_map.updated_atJulie Qiu
GetVersionMap now returns the UpdatedAt timestamp for a given row. Since it is possible for a module to 404, but appear in the index at a later time, this information will be used to determine whether we should refetch a given module. For golang/go#37002 Change-Id: If6a9e3a5d3580d58f8aafd9b40495b1fa0551e91 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/244600 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-05-26internal/postgres: read GoModPath from version_mpJulie Qiu
Change-Id: Ib73ccb8d505143825ffb267ede9589693d0613d0 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/753602 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-05-21internal/postgres: populate version_map.go_mod_pathJulie Qiu
The version_map.go_mod_path column is now populated in UpsertVersionMap. Updates golang/go#36811 Updates golang/go#37002 Updates golang/go#37106 Change-Id: I4e474d28bee67caf7625045082df46f93394f3f6 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/751269 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-05-13internal/postgres: change GetVersionMap logicJulie Qiu
The logic for GetVersionMap is changed so that an explicit module path is now required. All logic is also now the same for all requestedVersions (as opposed to a special case for latest). This function is only used by the frontend to check if a version exists as part of a frontend fetch, and this behavior makes more sense for that use case. Updates golang/go#36811 Updates golang/go#37002 Updates golang/go#37106 Change-Id: I415a2730daa6edc023f80c0c615521047311f35b Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/744833 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-23all: rename module to golang.org/x/pkgsiteJulie Qiu
golang.org/x/discovery is renamed to golang.org/x/pkgsite. When the repository is open sourced, it will be hosted at go.googlesource.com/pkgsite. Change-Id: Ifc3b45b771a385b99179e785447f2a87afcacf87 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/724273 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-16internal/postgres: change version_map.module_id insertJulie Qiu
We're still seeing issues with UpsertVersionMap, even though the queue has been dialed down to insert only one module at a time. Change UpsertVersionMap to be two queries: (1) select moduleID: should be a relatively trivial query that just fetches the module_id for a given path and version from the modules table (2) upsert version_map: same as query before the data model changes, which upserts a row in the version_map table, but now it also upserts a module_id This will allow us to get more information on where in UpsertVersionMap the query is failing. Not that both the current query on master and the new one succeed when running the worker locally and connecting to the dev database. Change-Id: I3c1d0a621294fa0e38bdd2165a35460dbacda4e6 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/719980 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-14internal/postgres: insert module_id and refactor saveModuleJulie Qiu
licenses.module_id and version_map.module_id are now populated. saveModule is refactored so that inserting the module is done in multiple functions, instead of one large function, for readability. The functionality has not change and the pieces of a module are still being inserted in a single transaction. Change-Id: I3603a388be5dbb90ce8f05ae9c237989f965e6e4 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/717237 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-06internal/etl: insert into version_mapJulie Qiu
When a module version is processed, a row is now inserted into the version_map. A separate row is inserted for each requestedVersion, even if they resolve to the same version, and regardless of the status code. Updates b/135954292 Change-Id: I4ff6f8647e75327831cc4b58974b95414424c1a7 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/670267 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-03-27internal/postgres: read/write from version_mapJulie Qiu
Updates b/135954292 Change-Id: I358e2265f5114056f8b5da75f0b261a8bf29f56d Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/649158 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>