Implement a check for every storage whether a repository is available #3527
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
FrostyX
force-pushed
the
fix-pulp-storage
branch
from
682d6f5 to
ccbb6ac
Compare
backend/copr_backend/storage.py
Outdated
| @@ -172,6 +180,11 @@ def delete_builds(self, dirname, chroot_builddirs, build_ids): | |||
| self.log.debug("can't remove %s", log_path) | |||
| return result | |||
|
|
|||
| def repository_exists(self, chroot): | |||
| repodata = os.path.join(self.opts.destdir, self.owner, self.project, | |||
There was a problem hiding this comment.
Is self.project actually a self.coprdir?
There was a problem hiding this comment.
It's actually this:
copr/backend/copr_backend/actions.py
Line 100 in e123e4e
which comes from frontend
copr/backend/copr_backend/actions.py
Line 100 in e123e4e
so it should only be a project name, not CoprDir name. Which should IMHO be what we want because CoprDirs don't have their own repositories available (see #3369) and they use the main repo.
There was a problem hiding this comment.
Separate CoprDirs have their own repositories, and they should be created before build. We previously used self.chroot_dir, see #3571 how the variable looks like.
I believe we are changing the semantics now 🤷
It is truth that we do not necessarily need the CoprDir's metadata, because these are not used at build-time, but that is probably a "temporary" issue.
There was a problem hiding this comment.
Just to confirm that you are right. Previously, we were checking if this exists
/var/lib/copr/public_html/results/frostyx/test-pulp-devel-7:custom:foo/fedora-rawhide-x86_64/repodata/repomd.xml
and we check
/var/lib/copr/public_html/results/frostyx/test-pulp-devel-7/fedora-rawhide-x86_64/repodata/repomd.xml
I will fix it, so we don't change the behavior.
It is truth that we do not necessarily need the CoprDir's metadata, because these are not used at build-time, but that is probably a "temporary" issue.
This is the root of the problem. I didn't treat it as a "temporary issue" but rather "this is how it is". But I agree with you, we eventually want to get this fixed.
There was a problem hiding this comment.
I hope I addressed the issue correctly. PTAL.
There is also one more commit fixing semi-related issue.
…available
Otherwise, builds in Pulp projects will fail with this:
Waiting for copr_base repository
Waiting for copr_base repository
Backend process error: Giving up waiting for copr_base repository
So far it worked only because we didn't create Pulp projects directly
but rather migrated existing projects into Pulp and therefore this
path existed on the backend.
This method is called when backend receives a createrepo action. As you can see, the `dirname` parameter was unused. Which worked fine when the createrepo action was generated when creating a new project or when manually clicking the "Regenerate" button in a project overview. However, createrepo action is also emitted when somebody submits a build into a CoprDir and thereby creates it.
FrostyX
force-pushed
the
fix-pulp-storage
branch
from
ccbb6ac to
2075cac
Compare
Otherwise, builds in Pulp projects will fail with this:
So far it worked only because we didn't create Pulp projects directly but rather migrated existing projects into Pulp and therefore this path existed on the backend.