#archlinux32 | Logs for 2024-01-30
Back
[02:35:05] -!- eschwartz has quit [Read error: Connection reset by peer]
[02:37:24] -!- eschwartz has joined #archlinux32
[05:54:48] -!- titus_livius has joined #archlinux32
[06:42:37] -!- GNUtoo has quit [Remote host closed the connection]
[12:30:48] -!- epony has quit [Remote host closed the connection]
[12:34:55] -!- epony has joined #archlinux32
[12:40:55] -!- epony has quit [Remote host closed the connection]
[13:39:36] -!- Hobbyboy has quit [Ping timeout: 268 seconds]
[13:40:14] -!- petris has quit [Ping timeout: 268 seconds]
[13:40:17] -!- Hobbyboy has joined #archlinux32
[13:41:59] -!- petris has joined #archlinux32
[13:53:21] -!- epony has joined #archlinux32
[16:08:42] -!- drathir_tor has quit [Remote host closed the connection]
[16:24:26] -!- drathir_tor has joined #archlinux32
[16:49:17] -!- T`aZ has quit [Remote host closed the connection]
[16:52:42] -!- T`aZ has joined #archlinux32
[17:13:20] <KitsuWhooa> I see everything has gone down now
[17:16:50] -!- drathir_tor has quit [Ping timeout: 255 seconds]
[17:17:11] <KitsuWhooa> abaumann / girls (whoever sees this first), archlinux32.org is down
[17:17:32] <KitsuWhooa> I can't fix it myself I don't think :p
[17:34:05] -!- drathir_tor has joined #archlinux32
[17:44:38] <KillerWasp> !web archlinux32.org
[17:44:38] <phrik> KillerWasp: Is that… Is that really a word?
[17:44:51] <KitsuWhooa> lol
[17:44:54] <KitsuWhooa> !version
[17:44:55] <phrik> KitsuWhooa: The current (running) version of this Limnoria is 2023.04.28, running on Python 3.11.6 (main, Nov 14 2023, 09:36:21) [GCC 13.2.1 20230801]. The newest versions available online are 2023.11.18 (in master), 2024.01.03 (in testing).
[17:44:58] <KillerWasp> hummm... for check the web with the vbot?
[17:45:00] <KitsuWhooa> of course it's limnoria
[17:45:05] <KitsuWhooa> !list web
[17:45:05] <phrik> KitsuWhooa: doctype, fetch, headers, location, size, title, urlquote, and urlunquote
[17:45:11] <KitsuWhooa> !web headers https://archlinux32.org
[17:45:12] <phrik> KitsuWhooa: <urlopen error [Errno 111] Connection refused>
[17:45:43] <KitsuWhooa> I got seemingly everything else back up though
[17:45:48] <KitsuWhooa> !web headers https://git.archlinux32.org
[17:45:49] <phrik> KitsuWhooa: Date: Tue, 30 Jan 2024 17:45:48 GMT, Server: Apache/2.4.58 (Unix) OpenSSL/3.2.0, Expires: Tue, 30 Jan 2024 17:50:48 GMT, Last-Modified: Tue, 30 Jan 2024 17:45:48 GMT, Connection: close, Transfer-Encoding: chunked, Content-Type: text/html; charset=UTF-8
[17:48:46] -!- eschwartz has quit [Ping timeout: 255 seconds]
[18:01:56] -!- eschwartz has joined #archlinux32
[18:12:11] -!- abaumann has joined #archlinux32
[18:12:23] <abaumann> I have to resync the mysql database, the replication slave refused to start.
[18:12:37] <abaumann> currently both the buildmaster and archlinux32.org are down (web-wise)
[18:13:45] <KitsuWhooa> I started httpd on buildmaster
[18:13:52] <KitsuWhooa> shouldn't I have done that?
[18:14:19] <abaumann> yes, I shut it down now, so that no user interaction happens in mysql thus causing issues again when starting the slave on archlinux32.org
[18:14:25] <KitsuWhooa> ah, understood
[18:14:26] <KitsuWhooa> sorry
[18:14:36] <abaumann> everything was fine before - except the mysql slave was misbehaving.
[18:14:43] <abaumann> it often does - all of its own. :-)
[18:14:48] <KitsuWhooa> :p
[18:15:06] <KitsuWhooa> I think for replication to be reliable, the database needs to not use certain features/be done a certain way
[18:15:18] <KitsuWhooa> I don't remember what the restrictions were, but I wonder if we are not complying with one of them
[18:15:25] <KitsuWhooa> (or would it not let you replicate at all then)
[18:15:25] <abaumann> dropping the databases in mysql on archlinux32.org, copying the last backup from buildmaster (after triggering one there with /usr/local/bin/dbsnapshot), then ./re-bootstrap-slave-db database-2024-01-30-18\:23\:09.xz
[18:15:30] <abaumann> usually does the trick
[18:15:33] <KitsuWhooa> I see
[18:16:23] <abaumann> who knows, I have seen so many bugs in mysql that I fear replication works reliable when both instances are shut down, and only then. ;-)
[18:16:29] <KitsuWhooa> hahaha
[18:16:46] <KitsuWhooa> also, unrelated, but I shut down two of my builders because they were building the same package, uploading it, and then buildmaster said invalid signature again
[18:17:05] <KitsuWhooa> I copied the packages manually out of the work directory and verified them on my machine, and it verified fine
[18:17:06] <abaumann> yes, I got a mail about broken signature of yours today.
[18:17:18] <abaumann> not sure, if that is related.
[18:17:27] <KitsuWhooa> well, my key doesn't expire as far as I know
[18:17:32] <KitsuWhooa> and I didn't touch the builders
[18:17:44] <KitsuWhooa> https://tasossah.com
[18:17:45] <phrik> Title: gpg verify (at tasossah.com)
[18:17:46] <KitsuWhooa> this was one of the packages
[18:17:49] <abaumann> so, most like some expired keyring on the users side then.
[18:18:00] <KitsuWhooa> maybe
[18:18:28] <abaumann> wow. importing the whole database didn't get any faster..
[18:18:47] <KitsuWhooa> what should I do to unblock those builders?
[18:19:51] <abaumann> mmh. don't know how the buildmaster is checking the signatures, but maybe gpg keys have to be refreshed for the buildmaster user, probably thjis would be 'master'?
[18:19:54] <KitsuWhooa> https://tasossah.com
[18:19:55] <phrik> Title: buildmaster pgp error (at tasossah.com)
[18:19:59] <KitsuWhooa> I have no idea
[18:20:13] <KitsuWhooa> it hasn't happened with builder1 yet
[18:20:21] <KitsuWhooa> but I wonder if it's because it hasn't successfully built anything
[18:21:21] <abaumann> gpg --batch --status-fd 1 -q --homedir /etc/pacman.d/gnupg --verify | grep -qxF TRUST_FULLY
[18:21:25] <abaumann> or so.
[18:21:48] <abaumann> a side effect of having an empty package?
[18:21:49] <abaumann> mmh.
[18:21:55] <KitsuWhooa> wait, it's empty?
[18:22:00] <KitsuWhooa> I haven't checked
[18:22:06] <abaumann> no, just venting ideas.. :-)
[18:22:16] <abaumann> I didn't check either
[18:22:31] <KitsuWhooa> it's definitely not empty
[18:24:04] <KitsuWhooa> [builder2@arch32-builder2 ~]$ gpg -a --export "TasosSah (Arch32 Package Signing Key)" | gpg --list-packets | grep exp
[18:24:04] <KitsuWhooa> version 4, algo 1, created 1686166312, expires 0
[18:24:04] <KitsuWhooa> version 4, algo 1, created 1686166312, expires 0
[18:24:49] <KitsuWhooa> so for some reason it's buildmaster that can't verify it
[18:49:24] -!- GNUtoo has joined #archlinux32
[18:49:36] <KillerWasp> expires 0 is a dead certificate?
[18:50:35] <KillerWasp> dumb comment. forget it, sorry.
[18:54:02] -!- drathir_tor has quit [Ping timeout: 255 seconds]
[18:54:28] -!- clemens3 has quit [Ping timeout: 246 seconds]
[19:04:26] -!- drathir_tor has joined #archlinux32
[19:06:24] <abaumann> Jan 30 21:06:08 archlinux32 mariadbd[12507]: 2024-01-30 21:06:08 18 [ERROR] mariadbd: Can't find record in 'online'
[19:06:27] <abaumann> Jan 30 21:06:08 archlinux32 mariadbd[12507]: 2024-01-30 21:06:08 18 [ERROR] Slave SQL: Could not execute Delete_rows_v1 event on table fluxbb.online; Can't find record in 'online', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-bin.001098, end_log_pos 2694, Gtid 0-1-18808317, Internal MariaDB error code: 1032
[19:06:32] <abaumann> Jan 30 21:06:08 archlinux32 mariadbd[12507]: 2024-01-30 21:06:08 18 [Warning] Slave: Can't find record in 'online' Error_code: 1032
[19:06:35] <abaumann> Jan 30 21:06:08 archlinux32 mariadbd[12507]: 2024-01-30 21:06:08 18 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'mysql-bin.001098' position 2419
[19:06:39] <abaumann> Jan 30 21:06:08 archlinux32 mariadbd[12507]: 2024-01-30 21:06:08 18 [Note] Slave SQL thread exiting, replication stopped in log 'mysql-bin.001098' at position 2419, master: 10.100.10.1:3306
[19:06:43] <abaumann> This is soo annoying.
[19:06:45] <abaumann> anyway. I start the systems again now, without synching
[19:07:34] <abaumann> https://buildmaster-status.archlinux32.org is the alternative URL for buildmaster stuff
[19:07:48] <abaumann> this one is always up-to-date, no matter of the state of the slave
[19:08:14] -!- abaumann has quit [Quit: leaving]
[19:17:28] <buildmaster> Hi abaumann!
[19:17:29] <buildmaster> !rq abaumann
[19:17:29] <phrik> buildmaster: <abaumann> sure, and emacs _is_ a browser ;-)
[19:19:17] <gehidore> something something emacs a great os lacking a decent editor ;)
[19:22:28] -!- abaumann has joined #archlinux32
[19:22:28] <buildmaster> Hi abaumann!
[19:22:28] <buildmaster> !rq abaumann
[19:22:29] <phrik> buildmaster: <abaumann> ..sorry. After 2 hours of trying to do basic stuff I start to get a little bit dizzy.. :-)
[19:22:32] <abaumann> aha, https://buildmaster-status.archlinux32.org
[19:22:42] <abaumann> show my master key 30891150 has expired
[19:23:19] <KitsuWhooa> maybe that is causing issues
[19:23:20] <abaumann> probably your signing key is signed by my master key, and now there is too little trust.
[19:24:53] <abaumann> created: 2020-01-03 expired: 2024-01-29 usage: SC
[19:24:58] <abaumann> definitely the problem
[20:07:25] -!- drathir_tor has quit [Remote host closed the connection]
[20:07:58] -!- drathir_tor has joined #archlinux32
[20:23:33] -!- abaumann has quit [Quit: leaving]
[22:48:12] -!- GNUtoo has quit [Remote host closed the connection]
[22:48:32] -!- GNUtoo has joined #archlinux32