@wowaname A script that parses the Mastodon profile static UI and saves every user and status it find into a DB

@matrix keeping db open is probably fine. sqlite db or something else?

@wowaname
Yeah. It's just SQLite. Should I do deduplication with the DB or in code? I want to make it multithreaded, so I think it's probably easier to just have the DB handle it.

@matrix dedup logic should be in db, yes, results in a cleaner structure

@matrix
Why duplicate what I'm already doing? ;)
@wowaname

@r000t @matrix no bully, maybe he just wants to learn how to use sqlite

@r000t
Experience. Your client is going to parse Mastodon's static UI? Awesome.
@wowaname

@matrix
Everyone is going to turn off the public API when that shit drops.

The static parser will be released shortly thereafter. I'm literally holding it back until admins start saying "phew! that client can't ever see my instance no matter what! I'm am very intelligent."

@wowaname

@r000t
That's a pretty great plan. I'm looking forward to the screeching. How are you going to handle Pleroma since it doesn't have a static UI?
@wowaname

@matrix
For now I'm going to naively assume that pleroma admins are less likely to try to play the "public to everyone but who i disagree with" game

Barring that, reinplement the web viewer in Python.

I played this game with losers who wrote imageboard software in their spare time, 10 years ago.
@wowaname

@r000t @matrix pleromafe uses api so you cant exactly filter requests to it

@r000t @matrix well, most you can do is set an instance's private mode to on, like i do on pl.wowana.me. but i only do that to cut down bandwidth use. i know stuff's still accessible directly, like post objects and whatnot

@wowaname @r000t I thought Pleroma has authorized fetch too

@matrix @r000t doesnt matter if your instance is set to public and people can use the frontend while logged out

@matrix @wowaname @r000t I haven't been following this very closely. Whats going on with all this fetch stuff and Mastodon devs...?

@shebang @matrix @r000t basically Pleroma Disrespect 2.0

@wowaname @matrix @r000t Alright, but what are the details, rationale? Like are they just deliberately making a Pleroma incompatibility or something? Isn't that just fucking up activitypub?

@shebang @matrix @r000t huh? both masto and pleroma are implementing authorised fetch from what i remember hearing

i dont know when you joined fedi, i was referring to something that happened years ago on fedi

@wowaname @matrix @r000t Hmm now what was it then, I think it was some big Mastodon server flipped on some option that made it incompatible with older versions. I thought it was fetch-y but now I don't recall. I assumed this was what it was about.

@shebang @matrix @r000t no, it was rareoma's pleroma fork that made everything (even DMs) public

@wowaname @matrix @r000t Oooh ok, nm then.

@shebang
Basically, there are a few larger Mastodon instances that suspend other instances for political reasons, but they don't do it to *not see things from that instance*, they do it with the expectation that *people on that instance cannot see their content*

They see it as some sort of divine punishment, that their drivel is something worth taking away.

So it stands to reason that they'd be upset if their content wound up in the servers they suspended. Never mind that it was posted publicly for public consumption, we want our message to go out to the whole world except these people.

So authenticated fetch became a thing.
@wowaname @matrix

@r000t @shebang @matrix authenticated fetch is actually useful with regard to voluntary association, but that point is moot when you realise that all these mastodon servers still have fucking public html pages for everyone to see. thats the kicker

@r000t
Yeah that sounds like the Mastodon devs. The guys who fled Twitter because it didn't censor enough for them. I don't know what their problem is, but it's not based on rationality, it's actually, and I mean this, deranged.

I like the software though. But yeah, thats dumb.
@wowaname @matrix

@matrix @r000t @wowaname private mode has different meaning in pleroma than mastodon. pleroma instance blocks are a joke compared to mastodon, the only work at the mrf, so when things arrive to your instance, they can't even stop people from mentioning users on blocked instances or stop the federator for sending messages to those

@r000t @wowaname
If my attempt turns out ok. I might deploy it and make a simple web that's going to allow everyone to search through it.

@matrix @r000t would be cool to crowdsource all instance histories into one cohesive database for searching

yes i said fedi's ephemeral but theres the pyromaniac inside me calling this a good idea

@wowaname
That all comes down to getting signatures for every status and if I recall, that's one of the things authenticated fetch disables.
@matrix