Search is complex - expectations

I don't mean the act of searching a database - that's routine - but I mean it is socially complex.

Lots of people left Twitter because it was too easy to search for them. For example, if you really hate people who support the wrong football team, it's trivial to search Twitter for people posting about that team. Then you can send them abuse. Or you can drag up old arguments. Or you can find something said years ago which sounds silly today.

So Mastodon was a haven. A place where it wasn't easy for twunts to pick a fight with you. Naturally, that means there has been some resistance to the idea of search - and that has informed its design decisions. By default, accounts cannot be searched. You have to make the positive choice to opt-in to search. And you can opt-out at any time.

That strikes me as a reasonable position. It adds a new feature for those that want it and keeps people safe if they'd rather not be bothered.

Search is complex - federation

The Fediverse is a collection of servers which share information with each other. You can only search the posts which your server knows about.

Here's what I mean. I post on mastodon.social and you post on whatever.int - because I subscribe to you, all your posts are sent to mastodon.social. That means anyone on my server can search for your posts. No one from fursuits.example follows me, so my posts aren't searchable by their members. But let's say @jeff@whatever.int is followed by @sue@fursuits.example - if Jeff reposts me, then that specific post is syndicated to the fursuits instance and that specific post can then be found in search results.

A bit confusing - but necessary. This isn't a monolithic service like Twitter with a single database. It is a loose collection of distributed entities which share data. You cannot search the entire Fediverse.

Search is complex - images

Six years ago, I noted that you couldn't search Twitter for Alt-Text. But, I am pleased to say, Mastodon allows for that! Search results include the description of images.

This can be a little confusing. You might search for a phrase and see dozens of posts which don't contain it in their body. It's only when you hover over the image that you'll find what you're looking for.

I don't know if this is something which could be improved with a better UI? Perhaps some way of highlighting that it is the image which is the result?

Search is complex - domains

Searching text is complicated. Should a search for "mastodon" find posts with the words "mastodons"? Probably. Should a search for the word "the" find "there", "their", or "them"? Probably not.

Should a search for "example.com" find "example.com/blog"? Maybe. At the moment, it doesn't. So you can't use search to find who shared your website. Nor can you find people who are talking about a specific page on the web.

Update! This works! A search for shkspr.mobi -from:edent finds all mentions of my website by people other than me.

Search is complex - operator discovery

As well as searching for words and phrases, you can also search metadata. For example, adding has:video to your query will find posts with an attached video. Using is:reply will find replies. And after:2023-08-27 will find posts after a specific date.

At the moment, there's no way to know what those options are. There's a discussion on GitHub which enumerates them but nothing for the casual user. Perhaps there needs to be an "Advanced Search" page? That's how most social networks do things.

Update! There's now a drop-down on the search box which shows you all the options.

Search is complex - what's missing?

Twitter let users search by number of likes & retweet - which was helpful for finding your most popular Tweets. That's not (yet) available on Mastodon.

Similarly, there's no way to search by geography. Twitter let users tag posts with their geolocation which could then be queried. Neither of those features are on Mastodon's roadmap.

I don't think it is possible to search previous edits of a post. That's probably a sensible decision.

Search is complex - unknown unknowns

One valid criticism of the Fediverse is that it is hard to search for interesting people, discussions, or posts. Hopefully this opt-in search makes it easier for people to meet new friends and form new ideas.

And, hopefully, the controls around it are strict enough that it will prevent abuse from running rampant.

I think searching is going to be primarily a good thing for the Fediverse. But what second and third order effects will this bring?

Gareth Hughes

@gh230277

Thank you for our compensation payment @Northpowergrid for the several days we were without power following #stormarwen Before I bank the cheque however, are you 100% certain you can afford this? #trillionpounds pic.x.com/z5MNc2Nxl1

---

I wondered whether 2324252080110 was a common sort of mistake - like a 32 bit overflow - so I Googled the number:

Huh! It is rare that a search doesn't provide any results. A search for a similar number gives one result - albeit a spam page

Type in any random number and there's usually something - even if it's just a glitch result.

Every integer under a billion seems to be there (I didn't check exhaustively) - but once you get into the trillions, they become a lot less likely to appear. See:

Of course, by writing 2230101000000 on this page, it will eventually find its way into Google all-seeing index - the original number I searched for now leads to page full of results linking to news stories. It's a little like the interesting number paradox.

If you find a lower positive integer, with no results, please let me know!

I remember, years ago, working on a mobile web service which had a URl bar - so users could tap in bbc.co.uk on their T9 keypads - and a separate search bar. I thought that was pretty nifty. But it turns out, users tried searching for URls and they tried going to "http:// When is the Next Bus?". Bloody users!!

Users find input bars confusing. A good example of this is Amazon's search bar. It faithfully records what everyone is searching for and uses that big data to suggest to other users what they might be interested in. Which leads to some "interesting" results.

Here's what happens if you start searching for https://

(And, yes, I used an incognito browser so it wouldn't be polluted with my own demented midnight searches for corrugated iron flavoured pogo sticks.)

There are a few lessons to take away from this.

• Users don't really understand interfaces
• Computers don't really understand users
• Big Data assumes that users are behaving in semi-rational manner

Wouldn't it make sense to use a QR code as well? That way people could quickly scan, and be taken straight to the discussion, rather than have to fire up Twitter and do a manual search.

As it happens, it's slightly tricky to make a QR code which searches for a Twitter hashtag.

There are two things to note:

1. Twitter's search URLs are annoyingly different from every other search URL on the planet.
2. You will need to take care of URL Encoding for special characters.

Building The Search Query

A typical Twitter search URL is

https://mobile.twitter.com/#!/search/

With the query at the end, so a search for "Transformers" would be

https://mobile.twitter.com/#!/search/Transformers

Encoding Correctly

Hash symbols (#) need to be URL Encoded. In this case, the hash becomes "%23"

https://mobile.twitter.com/%23!/search/%23Transformers

However, there is a "gotcha". Because of the... special... way Twitter constructs its search URLs, we have to URL Encode the percentage sign! So, the "%" becomes "%25".

So, our final URL becomes

https://mobile.twitter.com/%25%23!/search/%2523Transformers

Finally, if you want to use something like Google Charts to create your QR codes, you'll need the URL Encode all of the : / # ! and other special characters.

https://chart.googleapis.com/chart?chs=200x200&cht=qr&chl=
   https%3A%2F%2Fmobile.twitter.com%2F%23%21%2Fsearch%2F%2523Transformers

The Final Result

We end up with a QR code which can be scanned to take the user directly to the hashtag they are searching for,