Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possibly broken EOL chroots removal #3580

Open
FrostyX opened this issue Jan 10, 2025 · 3 comments
Open

Possibly broken EOL chroots removal #3580

FrostyX opened this issue Jan 10, 2025 · 3 comments
Assignees
Labels

Comments

@FrostyX
Copy link
Member

FrostyX commented Jan 10, 2025

From matrix:

solopasha
Is removing outdated repositories broken? I clicked "Expire now" for this repo https://copr.fedorainfracloud.org/coprs/solopasha/nautilus-code/ in https://copr.fedorainfracloud.org/user/repositories/ some time ago and "Remaining time" shows -31 days

@FrostyX FrostyX added the bug label Jan 10, 2025
@github-project-automation github-project-automation bot moved this to Needs triage in CPT Kanban Jan 10, 2025
@praiskup praiskup moved this from Needs triage to In Progress in CPT Kanban Jan 13, 2025
@praiskup praiskup moved this from In Progress to In 3 months in CPT Kanban Jan 13, 2025
@praiskup praiskup moved this from In 3 months to In Progress in CPT Kanban Jan 13, 2025
@FrostyX
Copy link
Member Author

FrostyX commented Jan 20, 2025

From the logs:

2025-01-20 03:27:28,873 [ERROR][/usr/share/copr/coprs_frontend/commands/delete_outdated_chroots.py:32|delete_outdated_chroots:delete_outdated_chroots_function][SERVER] Refusing to delete solopasha/nautilus-code/fedora-39-x86_64 because any notification was sent about its deletion
There are ~100 chroots there weren't deleted
Refusing to delete @meta/netconsd/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete davidsch/preview/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete davidsch/scripts/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete salimma/ebranch/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete solopasha/torrserver/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete salimma/ebranch/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/netconsd/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete solopasha/nautilus-code/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete @meta/netconsd/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete @meta/netconsd/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete solopasha/torrserver/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete solopasha/nautilus-code/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete salimma/rust-coreutils/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-s390x because any notification was sent about its deletion
Refusing to delete poscat/mdbook-linkcheck/fedora-38-ppc64le because any notification was sent about its deletion
Refusing to delete poscat/mdbook-linkcheck/fedora-38-s390x because any notification was sent about its deletion
Refusing to delete poscat/mdbook-linkcheck/fedora-38-aarch64 because any notification was sent about its deletion
Refusing to delete salimma/rust-coreutils/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/netconsd/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-x86_64 because any notification was sent about its deletion
Refusing to delete poscat/mdbook-linkcheck/fedora-38-x86_64 because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-ppc64le because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-aarch64 because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-i386 because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/test/fedora-38-x86_64 because any notification was sent about its deletion
Refusing to delete davidsch/review/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete davidsch/review/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete @meta/netconsd/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete davidsch/preview/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete davidsch/scripts/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete salimma/ebranch/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete solopasha/torrserver/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete salimma/ebranch/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/netconsd/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete solopasha/nautilus-code/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete @meta/go2chef/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete @meta/netconsd/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete @meta/netconsd/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete @meta/drgn/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-s390x because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete solopasha/torrserver/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete solopasha/nautilus-code/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete salimma/rust-coreutils/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-s390x because any notification was sent about its deletion
Refusing to delete poscat/mdbook-linkcheck/fedora-38-ppc64le because any notification was sent about its deletion
Refusing to delete poscat/mdbook-linkcheck/fedora-38-s390x because any notification was sent about its deletion
Refusing to delete poscat/mdbook-linkcheck/fedora-38-aarch64 because any notification was sent about its deletion
Refusing to delete salimma/rust-coreutils/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete @meta/netconsd/fedora-39-x86_64 because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete salimma/golang-wip/fedora-39-i386 because any notification was sent about its deletion
Refusing to delete @meta/pystemd/fedora-39-ppc64le because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-x86_64 because any notification was sent about its deletion
Refusing to delete poscat/mdbook-linkcheck/fedora-38-x86_64 because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-ppc64le because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-aarch64 because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/fusion-remi/fedora-38-i386 because any notification was sent about its deletion
Refusing to delete goddabrouquita8280/test/fedora-38-x86_64 because any notification was sent about its deletion
Refusing to delete davidsch/review/fedora-39-aarch64 because any notification was sent about its deletion
Refusing to delete davidsch/review/fedora-39-x86_64 because any notification was sent about its deletion

@FrostyX
Copy link
Member Author

FrostyX commented Jan 20, 2025

One observation is that when a user click the "Expire now" button, we call this function

@classmethod
def expire(cls, copr_chroot):
"""
A `user` decided to expire some EOL chroot,
i.e. its data should be deleted ASAP
"""
delete_after_days = app.config["EOL_CHROOTS_EXPIRE_PERIOD"]
cls._update_copr_chroot(copr_chroot, delete_after_days)

which sets that the data should be deleted in 12 hours. We send the notifications daily through Cron, so this doesn't overlap perfectly, and a deletion can be delayed by a day.

But that is not the problem here. I am manually running

runuser -c 'copr-frontend notify_outdated_chroots' - copr-fe

and it doesn't set the delete_notify in the database.

@FrostyX
Copy link
Member Author

FrostyX commented Jan 20, 2025

I think the problem is here:

return (query.filter(models.CoprChroot.delete_after
>= datetime.datetime.now())

The notification script never tries to notify chroots that should have already been deleted. Hence ... we are deadlocked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: In Progress
Development

No branches or pull requests

1 participant