MacKuba

I’ve been still pretty busy with various Bluesky- and social-related projects recently, so here’s a small update on what I’ve been working on since my November post, if you’re interested:

Skythread – quote & hashtag search

I was missing one useful feature that’s still not available on Bluesky: being able to see the number of quote posts a post has received and looking up the list of those quote posts. The Bluesky AppView doesn’t currently collect and expose this info, so it’s not a simple matter of calling the API. But since everything’s open, anyone can build a service that does this, they just need to collect the data themselves.

Since I’m already recording all recent posts in a database for the purposes of feeds and other tools, I figured I could just add an indexed quote_id column and set it to reference the source post on all incoming posts that are quotes, and later look up the quotes using that field.

Skythread, my thread reading tool, seemed like a good place to add a UI for this. When you look up a thread there, it now makes a call to a private endpoint on my server which returns the number of quotes of the root post, and if there are any, it shows an appropriate link below the post. The link leads you to another page that lists the quotes in a reverse-chronological order, like this (it doesn’t currently do pagination though). You can open that page directly by appending the bsky.app URL of a post after the quotes= parameter here: https://blue.mackuba.eu/skythread/?quotes=.

In the same way, I also indexed posts including hashtags, since hashtags were being written into post records since the autumn, but it wasn’t possible to search for them in the app. However, this has now been added to the Bluesky app and search service, so you don’t need to use Skythread for that. I hope that the quote search also won’t be needed for much longer :)

Handles directory

One very cool feature of Bluesky is that you can verify the authenticity of your account by yourself, by proving that you own the domain name that you’ve used as your handle. So for official accounts like The New York Times, The Washington Post, or Alexandria Ocasio-Cortez, it’s enough if they just set their handle to their main website domain (or a subdomain of house.gov in AOC’s case) to prove they’re legit – they don’t need to apply anywhere to get a blue or gold tick on their profile.

I was thinking one day that it would be nice to see how many e.g. .gov handles there are and notice easily when new ones show up. So I grabbed a list of all custom handes from the plc.directory and started recording new and updated ones from the firehose.

In the end, I decided to build a whole catalog of all custom handles, grouped by TLD, and show which TLDs are the most popular. At first I only included the “traditional” main TLDs and country domains, but a lot of people liked it and I got a lot of requests to also include domains like .art, .blue, .xyz and so on, so in the next update I’ve added all other domains too. (I gave it an old-school tables-based design as a homage to the old “web directory” websites like Yahoo Directory 😉)

Apart from handles, the website now also tracks third party PDS servers that started to show up after the federation launch in February.

Bluesky activity charts

I’ve also made a page that shows some charts tracking Bluesky user activity – the number of daily posts and unique users that have posted in a given day. The activity has been gradually falling since October until February, then there was a huge spike when Bluesky opened up for registrations without an invite (when Japan suddenly took over), and then it’s been falling down again since then (currently around the level of the October top).

You can also see some other interesting stats on Jaz’s page and Eric Davis’s Munin charts, especially the one tracking daily/weekly/monthly active user count. (I also have a few more ideas for what to add to my charts.)

DIDKit

In the last few weeks, I’ve been updating the code that tracks custom handles again to adapt to some protocol changes. The #handle event in the firehose, which included handle info on every handle change, is now deprecated and being replaced with a new #identity event, which only tells you to go fetch the account info from the source again (source being usually plc.directory).

At the same time, I also implemented validation of custom handles – clients and services that display handles are supposed to verify the handle reference in both directions themselves, because some accounts may have handles in the PLC registry assigned to domains that they don’t actually own or which don’t exist (the Bluesky official app shows such accounts with an “⚠ Invalid Handle” label, which you’ve probably seen before). For example, the handles directory page initially listed an amongus.gov account under .gov TLD, which was loaded from plc.directory, but is not in fact a real domain.

This should ideally be done by not relying on Bluesky servers, and instead checking the DNS TXT entry and the .well-known URL of a given domain manually. There’s a bunch of pretty generic logic there that will be needed in most projects that need to convert between DIDs and handles, so I extracted it to another Ruby gem named DIDKit, which lets you do things like:

• get the DID of an account with a given handle
• load the DID JSON document, which includes info like assigned handle(s) or hosting PDS server
• check if any of the assigned handles from the document resolve back to the same DID
• fetch all updates to all DIDs in batches from the PLC directory

mackuba ∕ didkit

A library for handling DID identifiers used in Bluesky AT Protocol

Skyfall

I’ve also been making some minor updates to my Skyfall library for streaming data from the Bluesky relay firehose.

One thing I’ve been trying to fix is a rare but annoying issue with the websocket connection getting stuck. From time to time, it manages to get into a state where no data is coming, but the connection doesn’t time out and just waits for new packets for hours, until I notice it and restart it. It isn’t only happening to me, others have mentioned it too (and not only in Ruby code); but it happens rarely enough that it’s really hard to debug.

My proposed fix is adding a “heartbeat” timer, which runs with some interval like every 30 seconds, and checks if there have been any new packets in some period of time; if there haven’t been any in a while, then it will forcefully restart the connection. (This isn’t included in the latest release yet, I’m waiting for it to get triggered a few times first.)

Another thing I’ve added is being able to connect to a new kind of firehose exposed by “labellers” a.k.a. moderation services. Bluesky has released this new important piece of the federated architecture earlier this month – third party developers and communities can now set up independent moderation services, which manually or automatically add various “labels” to accounts or specific posts, flagging them e.g. as “racism” or “disinformation”. Anyone can subscribe to any labellers they choose, and they’ll see the labels from those selected services shown in the app. The new firehose (the subscribeLabels endpoint) allows you to connect to a specific labeller and stream all new labels that it’s adding.

I’m also tracking all new registered labeller services and keeping a list here (it’s not curated, just a dump from a database table, so it also includes various test servers etc.).

mackuba ∕ skyfall

A Ruby gem for streaming data from the Bluesky/AtProto firehose

The “Bluesky guide”

Last month I wrote a long blog post titled “Complete guide to Bluesky”, where I included various info and tips for beginners about Bluesky history, available apps, handles, custom feeds, privacy, or currently missing features. I’m now updating it every time Bluesky releases another big feature :) Check it out if you’ve missed it.

I have ideas for a few more Bluesky introduction posts with a developer focus – a general intro to the protocol and architecture, and about working with the XRPC API and the firehose. I hope I’ll be able to find time for that in the next few months.

Tootify – cross-posting to Mastodon

I still want to finish my Mac app for cross-posting to Twitter, Mastodon and Bluesky one day, but it’s a lot of work and I’ve got too many different things in progress at the same time, so it’s moving at a glacial pace… In the meantime, I started thinking if I could maybe quickly build something much simpler that also does the job. I’ve been mainly hanging out on Bluesky in recent months and posting on Mastodon only occasionally, because having to copy-paste things from one tab to another is annoying, especially if images and alt text are involved. But the folks I know from Twitter still mostly follow me on Twitter and Mastodon only and aren’t coming to Bluesky.

I also didn’t want to simply copy every single post from here to there, because a lot of things I post on Bluesky are specifically about Bluesky stuff, so it doesn’t always make sense to post them to Mastodon – I only want some selected ones to be copied. But at the same time, I wanted to minimize the amount of friction this would add.

So the idea I had one night was that I could mark the Bluesky posts to be copied to Mastodon by simply “liking” my own posts that I want copied; a service or a cron job would then periodically look at the list of my recent likes, and when it notices one made on my own post, it would copy that post to Mastodon (and remove the like).

I managed to build it in about a day and a half, complete with image support with alt text and copying of quote-posts as posts with plain links. It’s now running happily on a Raspberry Pi on my local network 😎

The code is published here, if you’re interested – but it’s a bit of a proof of concept at the moment, just enough to make it work for myself, so it’s probably not very user-friendly. But maybe I’ll build it up into something bigger if people find it useful. (Just to clarify, this is meant to be a one-way sync by design – syncing in the other direction would be harder for various reasons, e.g. because of the complex “facets” system that Bluesky uses for post record data, and because Mastodon’s post length limit is higher than on Bluesky.)

mackuba ∕ tootify

Toot toooooooot

And One More Thing ;)

Paul Frazee, Bluesky’s lead dev, has a lovely cat named Kit and often posts photos of her. I’m a big fan of Kit, so I made a feed named the Kit Feed, which only includes posts with these photos 🙂 Like and subscribe! 🐱