Show newer

We're sorry about the downtime that occurred yesterday. One of the drives in's server failed. We were running in RAID-1, but because the software raid was misconfigured we encountered corrupted filesystems due to high I/O.

All data was recovered successfully. The RAID array has been rebuilt with a new (0 hour) drive. One of our concerns was that the remaining drive in the replica would fail during the rebuild but that didn't happen, which we're grateful for. We will be putting that hard drive out of service within the next month (which will require scheduled downtime).

It seems like the drive died while processing the huge amount of strain imposed on it by Mastodon and Minio during the old media removal process. We will move to Backblaze B2 before running out again (which will require more scheduled downtime).

rip someone deleted their account and it caused 5000 jobs to get enqueued and I got an alert about it

registrations disabled on once again, someone tried to sign up using bots a bunch of times with what looks like hacked email addresses.

you can still sign up with the invitation link conveniently shown where the registration form used to be

Recently I made a shitty DDR pad to make up for the fact the local arcade sold their ITG cabinet.

The sensors are made out of tin foil lol. Eventually I'll make a improved version with real sensors (weight sensors).

Haven't made a bar for it yet, using an unused bedside table temporarily (it sucks since it's the wrong height and position)

Interestingly I still get follows even though I don't post very often

haven't had any drop outs or lag in the timelines on since we disabled cloudflare. glad we finally switched away from it

got a trackball mouse this week, the Kensington Orbit with scroll ring. it's nice to use so far, but my accuracy is worse than I expected. wondering if it's just something I need to get used to (like I did with my kinesis advantage 2 keyboard) or if I'm using it wrong is now running without cloudflare enabled. Eventually I'll move it to hurricane electric DNS or something

This happened after updating to the newest version, so we were convinced it was broken code but no one else was reporting similar issues.

For debugging, we undid our patches and added tracing to sidekiq and mastodon, which lead us to believe mastodon was filling memory because of elasticsearch failures. We'll reapply the patches shortly.

Show thread

We've been having a lot of troubles with mastodon recently (filling memory and swap), which we traced back to elasticsearch freaking out about there not being enough space on the storage drive.

We cleared off some old backups/unnecessary data from the disk, so everything should be fine now. Sorry about the recent downtime.

breaking news: world famous tija nee-san creates account on

registrations are fully open again. it's kinda disheartening to see displayed as a private instance on instance listing sites even though there's an invite on the homepage

mastodon moderation tools suck. performing moderation actions takes way longer than it should (it loads for a while), and for some reason disabled accounts can still post? I disabled 2 accounts a week ago and they somehow both posted today...

and to top it all off, after I suspended both of the accounts, the posts from one were still available on local timeline until I manually deleted the post.

I switched to fastmail today to try it out, they added labels in beta which was the last thing I needed to switch. gonna stay on the trial for a month but it seems pretty good so far.

the android app felt a bit sluggish but it's whatever. I can use a different mail app if it's a problem, plus I don't use my phone for viewing my mail very often.

proud owner of giant chocolate egg

registrations are behind an invite link due to a recent batch of bot signups. the registration link is on the homepage.

also, autofollows of aurieh and I are disabled now because you can finally turn it off

and since I couldn't get alsa to work, I have to attach my microphone to the VM to be able to call people from inside which sucks but it's whatever

doom eternal runs good, so all is well

Show thread

looking glass is working but the quality is ass, not sure why. maybe it's a wayland thing who knows. for now, I'm just using the GPU directly by switching inputs on my monitors, works fine. steam streaming works well too

setting up alsa audio with QEMU was a fucking nightmare, so I opted for scream, which is a windows virtual sound driver that just sends the pcm data over the network. multicast didn't work, so I had to change it to be unicast with my host's IP but it works great with very low latency

you'd expect the scream shit to be a pain in the ass and slow as hell, but I guess there's hardly any latency anyways since it's virtio net so whatever

Show thread

I set up a pci passthrough windows vm, had to go out and buy a 2070 super for it

bad experience at first, and the pci_acs_override patch didn't work for my nvme drive (???), but after enabling hugepages and changing the CPU model to be EPYC in QEMU (I have a Ryzen processor) it works great

sway was working pretty well for me but recently it's just been crashing, `dpms on` hasn't been working properly, desktops have been switching between monitors when using `dpms on`, and redshift-wayland has been randomly been turning off on a monitor

firefox still doesn't default to wayland either, extension menus sometimes don't appear, and there are random crashes (like when playing a video fullscreen on mastodon???)

pretty disappointing but I still believe in sway and wayland and I hope it gets better soon. I'll just put up with it

Show older

A public Mastodon instance run by the same people who run owo (the file sharing website). Everyone is welcome... as long as you like to uwu. Please read the rules before registering an account on this instance.