2026-03-26
Agenda
- Kubernetes Gateway API migration
In its continued war on software, Kubernetes has deprecated working features. We need to migrate to the new gateway API. Joe has already done some work with Envoy and will inform us.
Discussion
Envoy is crash looping. Joe deployed it, and he is asking why. He realizes that it is not really crash looping. It only restarted four times. Chris says that's fine, that's acceptable. Joe realizes that "it just fucking killed itself. Look at this shit."
Joe realizes the issue. We haven't paid for the high availability control plane. See rule 5.
Joe elaborates on what is missing.
Actions
Ticket #611 tracks further progress and got expanded with a checklist.
- Owl Corp Guix Area 51 on Turing
turing.box.pydis.wtf, which used to be Chris' property, has been stolen in a
fantastic scheme that we shall label "Theft by DNS A record". We have now
deployed Guix on it, to play around with fully
declarative host deployment.
We now want to figure out what to do with the host to expand our testing. The following suggestions have been made so far:
- Numbers station
- agents.pydis.wtf
- database backups
- Lovelace monitoring
Suggested actions
- Create a milestone for Area 51 initial setup
- Create issues for the bullet points above
Discussion
Accepted and ratified under Amrou Bellalouna Order in Absentia #125. Johannes actioned it.
- LKE IP address whitelisting
Right now the /etc/nftables IP whitelist on lovelace is only refreshed on
deployment. This is suboptimal, since worst case our resources may get
scheduled on a new node that is not whitelisted in the firewall.
The ideal solution would involve as little manual work as possible. nftables
has an include directive: we could write a timer / cronjob to update a
nftables file containing only the LKE ip addresses, which is then included
in our Ansible-managed main nftables.conf. We would have to take care of
setting up an initial IP whitelist in said file to prevent errors when
provisioning a new server (where the timer has not run yet).
Suggested actions
Create a ticket.
Discussion
Accepted. Actioned by Johannes.
- GitHub RBAC synchronization
Right now there is a lag between Discord roles and GitHub roles. As with LDAP, we should likely include this functionality in King Arthur The Terrible.
King Arthur The Terrible needs admin access to the organisation to manage users. We should call it Big Brother, because it upsets some people and is funny.
Suggested actions
Create a ticket for King Arthur The Terrible. Create a ticket to store GitHub usernames in LDAP.
Discussion
Accepted. Johannes will create a ticket. Points to note: we have to store all GitHub usernames in LDAP. Joe says that this makes him want to kill himself.
Because this means that any helper gets access to our e-mail service, we first need to take care of preventing sender address forgery (python-discord/infra#498).
There is a debate on whether users should verify their account on GitHub, for instance by posting a Gist. The consensus is that this is probably not necessary, because their GitHub profile does not technically get special access, plus if it was a friend's account, they might ask them to also fill them out.
Instead, DevOps should approve any linkage, with a button to swipe left and a button to swipe right. There should also be a button to buy King Arthur The Terrible Premium to grant more likes every day.
We should store the GitHub user ID, not the username.
Actions
Issues created by Johannes.