Lemmy Developer AMA and Dev Update, 2024-01-26, 1500 CEDT

This is a chance for any users, admins, or developers to ask anything they’d like to myself, @nutomic , SleeplessOne , or @phiresky about Lemmy, its future, and wider issues about the social media landscape today.

NLNet Funding

First of all some good news: We are currently applying for new funding from NLnet and have reached the second round. If it gets approved then @phiresky and SleeplessOne will work on the paid milestones, while @dessalines and @nutomic will keep being funded by direct user donations. This will increase the number of paid Lemmy developers to four and allow for faster development.

You can see a preliminary draft for the milestones. This can give you a general idea what the development priorities will be over the next year or so. However the exact details will almost certainly change until the application process is finalized.

Development Update

@ismailkarsli added a community statistic for number of local subscribers.

@jmcharter added a view for denied Registration Applications.

@dullbananas made various improvements to database code, like batching insertions for better performance, SQL comments and support for backwards pagination.

@SleeplessOne1917 made a change that besides admins also allows community moderators to see who voted on posts. Additionally he made improvements to the 2FA modal and made it more obvious when a community is locked.

@nutomic completed the implementation of local only communities, which don’t federate and can only be seen by authenticated users. Additionally he finished the image proxy feature, which user IPs being exposed to external servers via embedded images. Admin purges of content are now federated. He also made a change which reduces the problem of instances being marked as dead.

@dessalines has been adding moderation abilities to Jerboa, including bans, locks, removes, featured posts, and vote viewing.

In other news there will soon be a security audit of the Lemmy federation code, thanks to Radically Open Security and NLnet.

Support development

@dessalines and @nutomic are working full-time on Lemmy to integrate community contributions, fix bugs, optimize performance and much more. This work is funded exclusively through donations.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. Recurring donations are ideal because they allow for long-term planning. But also one-time donations of any amount help us.

9point6,

Firstly, thank you so much for providing the means for me to cut Reddit out of my life, I feel like I’m engaging with content in a much more deliberate way since, and honestly it’s been a massive improvement to my mental health in a way that I was completely oblivious to there even being a problem before.

Anyway, the question—regarding things happening entirely out of your control, what would be the best and worst things that could happen to lemmy from your perspectives? And as an extension, what are your goals for it?

nutomic,
@nutomic@lemmy.ml avatar

The best thing would be if Reddit goes the way of Digg. Seems that will happen sooner or later. The worst thing, maybe if funding stops and we are unable to keep working on Lemmy. But even then admins could still host Lemmy instances.

Tangentism, (edited )

The best thing would be if Reddit goes the way of Digg.

Well, it has already. The only reason it hasn’t fully imploded & all the users deserted for another site, is because there wasn’t an equivalent place to go to.

They were sort of parallel in development but digg blew up and Reddit didn’t then Digg took a quick hard turn towards enshitification.

Reddit has done the enshitification but like a parasitic infected spider, it’s wandering about and most of the users haven’t realised yet that it’s an empty shell.

It’s slow demise would be better in the long run than a quick collapse like Diggs so it’s now putrid culture is not transmitted with an enmass exodus.

can,

Couldn’t agree more. I’m still hopelessly addicted to the format but at leat now I’m sticking it to the man at the same time lol.

dessalines, (edited )

Thx! Its pretty wild to me how much these algorithms, and formats, affect our mental well-being. Those giant US tech companies employing Psychology PhDs to figure out how to keep people angry, engaged, and watching ads, is doing so much harm to so many people, not just in the US, but the whole world, and unfortunately very few countries are doing enough to protect their people from these companies (who also act as surveillance arms of the US state) by blocking facebook and the rest.

I’ve seen two professors I respected turn into angry children on twitter, in a way that would never happen in real life. Reddit, twitter, and Youtube platform reactionary rage-bait to get people trapped in a downward spiral of negativity. These companies do not care how much damage they do; all that matters to them is their profits.

We don’t have those same incentive structures, so we can and should be doing everything we can to make this a positive and enjoyable experience, not about arguing constantly, but about learning, laughing, and understanding.

what would be the best and worst things that could happen to lemmy from your perspectives? And as an extension, what are your goals for it?

The best thing would be that we continue our slow and steady growth. Every user that migrates away from big tech to the fediverse is victory, so while we shouldn’t emphasize growth at any cost, its still a good thing when we can get people away from all that negativity.

The biggest concern for me about Lemmy, would be a centralization onto one big server, that tries to replicate all the worst things and behaviors about reddit: its combativeness, xenophobia, bigotry, pro-US-foreign policy agendas, and advertising. There is a noticeable chunk of Lemmy’s users who don’t really see any problem with those things, they just want a reddit that lets them use 3rd party apps again.

calavera,

Were you ever approached by any kind of organization making some weird proposal regarding lemmy?

nutomic,
@nutomic@lemmy.ml avatar

Some company (dont know which) wanted to make a one-time donation of 500 Euros to get listed as donor on join-lemmy.org. Rejected because thats only for recurring donors. Does this count as weird?

dessalines, (edited )

A few, mostly harmless tho, just about working on pet features they’d like to see in Lemmy. None panned out.

I imagine the more parasitic companies avoid us as soon as they see the AGPLv3 license.

jawa21,

I post a fair amount of video edits. I’ve had quite a few people say that video playback is far from ideal for not just Lemmy, but the Fediverse as a whole. Is this mostly a 3rd party app thing, or a backend issue? I haven’t had much issue myself, but enough people have mentioned it that there is likely an issue somewhere down the line.

simple,

but the Fediverse as a whole.

FWIW video playback on Mastodon has been seamless for me.

nutomic,
@nutomic@lemmy.ml avatar

It works pretty well on Peertube, only lacks users and quality content.

jawa21,

I’m sure. However, I am not in a position to self host and videos are huge unless you post them as webm which some people might even report you for.

dessalines, (edited )

I have ambitious plans about how to turn lemmy into a true Youtube / Spotify killer. But that might not be what you’re asking, which might just be simple browser / app video support (IE mp4s, webm, etc).

If that app / web UI has trouble with videos, you might need to open up an issue there.

threelonmusketeers, (edited )

Where is the best place to propose new features for Lemmy?

Edit: And as potential follow-up, where is the best place on Lemmy to propose new features for Lemmy? (Not every Lemmy user has or wants a GitHub account)

can,

I’m not sur eif it’s bad form to answer these questions but for this one I think the github issue page is the answer.

dessalines,

This is correct, its our issue tracker and its what we work from to close issues.

Die4Ever, (edited )

if you don’t want to make a github account, then probably !lemmy

but you can still search and read the github issues without making an account, so you could check if it already exists, or link the issue if you’re starting a lemmy discussion about it

BlanK0,

Are there any plans on adding features that enable easier interaction with other federated platforms like mastodon and peertube (for example being able to comment/interact with peertube videos and mastodon posts)?

nutomic,
@nutomic@lemmy.ml avatar

You can already interact with Peertube videos and follow their channels. Thats possible because Peertube also federates groups (communities). With Mastodon thats not possible because it doesnt have groups, and Lemmy doesnt support content outside of communities. At least not without a full rewrite, which doesnt make sense considering that KBin and dozens of different microblogging platforms already exist.

syd, (edited )
@syd@lemy.lol avatar

Sorry for taking up all the space :) But please skip the ones you wish.

  1. How old are you guys? Can be a range if you want e.g. 10-15, 15-20 :)
  2. Do you know each other IRL?
  3. Which region do you live in? Like America, Europe, Asia etc.
  4. If Lemmy had 100 million users, would you still try to remain anonymous?
  5. Is there an instance admin you hate but can’t say?
  6. What percentage of users should ideally be in the largest Lemmy instance do you believe?
  7. If you had the chance to change the name Lemmy, would you?
possiblylinux127,

I’d hope that they are at least 18

nutomic,
@nutomic@lemmy.ml avatar
  1. Im 31
  2. Not yet, hopefully that will change in the future
  3. North of Spain
  4. Im not really anonymous, it would just feel weird to post here with my full name
  5. No
  6. Maybe less than 50%
  7. Im terrible with names so no
OsrsNeedsF2P, (edited )

So many apps die before getting any users. For Lemmy however, when was the first time you really thought “Damn, this thing really might actually take off”?

nutomic,
@nutomic@lemmy.ml avatar

Lemmy was meant to be a Reddit replacement from the beginning, so it was always supposed to take off. Even in the early days the tech was working quite smoothly and users were happy so there was no real doubt about it. The only thing missing were more users. However I had no idea how a real migration would actually look like, so it was really overwhelming when last year people started to flood in and everything got overloaded and broke down.

dessalines, (edited )

For me it was long before the reddit migration (which was ~7 months or so ago). I noticed lemmy slowly but surely gaining traction. It felt more dead than it does now, but the trend was slow and steady growth, which is always a great sign. People were using lemmy, liking it, and sticking around.

At the same time, it was clear that we weren’t making the mistake of all the other reddit alternatives, by promising to be a free speech haven for bigoted communities. Those people actively did our work for us by warning their communities to stay away from Lemmy and its tankie devs, thereby making Lemmy a much more enjoyable place from the very beginning. That was a crucial test: we were not willing to sacrifice our values for growth’s sake.

It’s great to see that positivity confirmed by a researcher who did a qualitative and quantitative analysis about Lemmy migration, and finding that >90% of people saw themselves using Lemmy in the long term. We can all be very proud of that, and it means we have a bright future.

SubArcticTundra, (edited )
@SubArcticTundra@lemmy.ml avatar

Do you have any estimate of how much storage (in GB) all the posts ever posted across Lemmy have taken up, to date? (Excluding media)

1984,
@1984@lemmy.today avatar

Something tells me lemmynsfw has the largest disks. :)

Omega_Haxors,

They call me Bigdisk Johnson.

WanderingVentra,

I should donate to them. I mean… lemmynsfw, what’s that?

Blaze,
@Blaze@discuss.online avatar

The porn instance

morrowind,
@morrowind@lemmy.ml avatar

*A porn instance

WanderingVentra,

I’m just playing dumb. I know what it is lol

JaymesRS,

I was told it’s not about the size of the disk but how it’s used that really matters.

1984,
@1984@lemmy.today avatar

Yeah we are told a lot… :)

nutomic,
@nutomic@lemmy.ml avatar

The SQL table for posts is 1.6 GB on lemmy.ml, and 5.7 GB for comments. That probably accounts for a majority of content on the Lemmyverse.

SubArcticTundra,
@SubArcticTundra@lemmy.ml avatar

1.6GB is impressively small for anything by modern internet standards

dessalines,

To add, lemmy.ml’s entire DB compressed as a xz with -0 strength is about 3.7 GB. But that also includes the activity tables which aren’t vital.

phiresky,

I don’t think it’s that large. Text is very small and compressible compared to images. Well it depends on if you mean the actual database storage (uncompressed, with indexes) or a compressed copy of all the posts. You can see the post number in the URL, which on lemmy.world for this post is 11169622. That means there’s around 11 million posts total in lemmy.world’s database. If you assume each of them takes 0.5kB of storage that would be only ~ 5 GB of posts.

A_A,
@A_A@lemmy.world avatar

… is 11169622 …

Maybe 9 post out of 10 are deleted by the few checks I made manually …or am I missing something ?

OsrsNeedsF2P, (edited )

What are some cool “Lemmy Adjacent” projects you know of and want to share? (Things like LemmySchedule or Toast.ooo’s Canvas)

dessalines,

One that I can think of rn, is @CannotSleep420 's lemmy-bot, as well as ridoukousage’s TLDR bot.

With the web being so ad-infested and completely owned by google, people have noted how the TLDR bot means they often don’t have to leave their lemmy app at all, and can stay behind its privacy shield.

While of course I do think we can code a lot of functionality directly in to lemmy in a way that we couldn’t with reddit, there’s undeniably a lot of potential with bots that can do different things for us.

OsrsNeedsF2P,

What’s coming in the Lemmy 0.20 release?

nutomic,
@nutomic@lemmy.ml avatar

Any breaking changes which get implemented in the meantime. There are no specific plans.

OsrsNeedsF2P,

Not a question, just wanted to let you know I how much we appreciate and love you all for making Lemmy happen 🥰🥰

nutomic,
@nutomic@lemmy.ml avatar

Thank you! Its great that you have been around all these years.

dessalines,

Thx a ton!

OsrsNeedsF2P,

What other ideas do you have to increasing funding for Lemmy development?

nutomic,
@nutomic@lemmy.ml avatar

Wait for Reddit to implode, more users to migrate and donations to increase. It worked last year :D

syd,
@syd@lemy.lol avatar

lol

dessalines,

We could definitely use some help with ideas there. Lemmy currently has ~40k active users, and it should be able to support more than ~1 average dev salary, especially if we want to take on a multi-billion dollar company with hundreds of employees like reddit.

Ludrol,
@Ludrol@szmer.info avatar

I think short event or campaign with push for donations with a pop up that you actually can dismiss. An ad like banner. The biggest problem would be community organization as Lemmy isn’t only decentralized horizontally but also vertically. Different front ends, different apps different instances. Most of them wouldn’t want to implement an ad that wouldn’t benefit them directly. They also have costs with running their piece of lemmy. So some cut for them should be included.

I think a dedicated trustworthy person should be responsible for organizing this campaign as developer time is best spent elsewhere.

A_A, (edited )
@A_A@lemmy.world avatar

Wikipedia, about once a year, has those donation pledge boxes at the top of every article …they must be somewhat effective since they come back year after year …better keep them small though to avoid disheartening users. Maybe start small like this trending community line at the top of the user feed.
P.S. : Since we don’t want the user to get habituated it’s better if it’s just a few days once a year.

OsrsNeedsF2P,

First, I want to say thank you for the incredible job you already have done in this area. However, do you have any thoughts on further improving some fundamental Lemmy UX painpoints? Examples such as:

  • Migrating accounts between instances
  • Tagging users across instances
  • Linking communities across instances
  • Finding communities across instances
Blaze,
@Blaze@discuss.online avatar

Migrating accounts between instances

Isn’t that implemented in the 19.2 and later versions? I just migrated using that feature a few days back, worked quite well

WanderingVentra, (edited )

That would be awesome if true. It’s progressing faster than I thought. I’m still just learning about the scaled sort and enjoying that new feature lol.

Blaze,
@Blaze@discuss.online avatar

I’m pretty sure it is there, you can export and import your subscriptions in the settings

can,

It’s only subscriptions, blocks, and user settings iirc. Your posts and comments don’t migrate for example.

WanderingVentra, (edited )

Still that’s not bad. Wish they could get saved posts to transfer, too. That would be useful.

Blaze,
@Blaze@discuss.online avatar

Importing posts and comments could cause a security risk if someone would to abuse that function.

Even Mastodon doesn’t support it

Mastodon currently does not support importing posts or media due to technical limitations, but your archive can be viewed by any software that understands how to parse Activity Streams 2.0 documents.

docs.joinmastodon.org/user/moving/

nutomic,
@nutomic@lemmy.ml avatar

More importantly it would make exports extremely large and would cause a lot of server load to import/export. Plus you would end up with duplicate posts and comments which seems like a bad idea.

Blaze,
@Blaze@discuss.online avatar

Yes, that’s what I meant by security risk, that’s like a DoS feature.

JackbyDev,

Can you add one to your list? Linking posts across instances? Like you can do !community@instance and the community will open viewed through your instance. But for linking posts there is no such equivalent. Like if I make an HTTP link it will be through my instance or possibly the one the community is hosted on which would be annoying for users of other instances.

Also, linking communities across instances is possible already, but you can leave it up since it’s confusing. I still see a lot of folks try to do the reddit approach if c/community

spaduf, (edited )

Turning the fediverse button into an “open on my instance” with similar functionality to subscribing may also be a solution here. Bonus points if it’ll also open a comment on mastodon.

Die4Ever,

github.com/LemmyNet/lemmy-ui/issues/2318

spaduf,

Yup that’d be sick

Die4Ever,

github.com/LemmyNet/lemmy/issues/2987

spaduf, (edited )

Really like your protocol handlers contribution here. Seems tough to square with multiple accounts though.

phiresky, (edited )

For migration we recently added a feature to export your user data. But “real” migrating accounts is something I put on our “todo” list, though it probably also first needs a proposal to define how it should work exactly (should it still work when the original instance is down?) As soon as we start giving users more control over their private key issues start appearing like not having any infrastructure for key rotation / revocation. Without that it will only work when the original instance still exists.

I’m not sure if by tagging users you mean linking / mentioning them? Or adding tags to them like you can tag posts / users on other platform. For tagging in general there’s a pending proposal github.com/LemmyNet/rfcs/pull/4 . So far it focuses on post tagging though to reduce the scope. I think the goal is going to be to start with one kind of tagging and add more kinds of tagging later.

For improving cross-instance linking (both communities, posts, and users) we also have a open milestone. There’s a few spitballing issues about it, but no real concrete proposal on how to build it yet.

dessalines,

As @phiresky mentioned we have improvements coming down the pipe for linking content across instances.

Community linking and user linking do work currently (for example I just linked phiresky above), and a community example would be !risa , but we could improve this by extending it to posts and comments, as well as creating a url link standard that would work across apps.

OsrsNeedsF2P,

Back when the first Reddit exodus happened, there was a group heavily DDOSing many of the popular Lemmy instances. While it was a great opportunity to optimize Lemmy, did you ever find out who that attacker was?

phiresky,

I don’t think we found any specific groups of people attacking Lemmy. I personally just saw one or two what looked like individuals trying (and succeeding) to take Lemmy down with a few very simple requests that forced Lemmy to do lots of compute (something like fetching the next million posts from page 10000). The fixes for those were simple because it was just missing limits checking.

I’m not sure if there actually was a larger organized attack. Lots of performance issues in Lemmy simply appeared simultaneously and compunded each other with a rapidly growing number of active users and posts.

OsrsNeedsF2P, (edited )

How do you feel about extreme right-wing instances like the late Wolfballs using Lemmy to promote and spread hate?

WanderingVentra,

I hope they’re not too against it. I know they’re extremely left wing, which scared a lot of the centrists on Reddit. If they allow even right wing instances, then it emphasizes the project’s lakc of political bent, and encourages more mainstream people to join. The politics can be up to each individual instance to decide whether to defederate with those other instances or not.

But that’s just my opinion, I’m also curious how the devs will answer.

bdonvr,

I’m sure they’re probably not okay with it but also there’s not much they can do about it other than defederate .ml - such is the nature of open source software.

calavera,

Not that I’m suggesting it, but they could hardcode the defederation of those instances on the code and most admins wouldn’t bother to fork lemmy to remove it. Like in the past they had a hardcoded slur filter, but I think they disabled it because many slurs in English were regular words in other languages

possiblylinux127,

Yeah that’s a very slippery slope

Kaboom,

Wolfballs? Was it straight up hate or just run of the mill GOP talking points?

Moonguide,

Wasn’t here back when it happened, but from what I know, it was straight up white supremacy shit. It got defed to death.

possiblylinux127,

I think you should be careful with political hate. Its a slippery slope and should not be handled by a developer.

Maybe we could simply focus on having clear rules for each community.

nutomic,
@nutomic@lemmy.ml avatar

They were posting spicy memes but thats how the internet works. If you dont like it then dont visit there, just like you wouldnt visit 4chan. Lemmy is open source so anyone can use it for any purpose.

dessalines, (edited )

I very much dislike it obviously, and I’m happy that one shut down. There have been others, but for the most part they’ve stayed away from Lemmy as “that software made by tankies.”

Outside of making sure that we don’t platform them anywhere, there isn’t much we can do. Lemmy is open-source software after all, and a tool can be used for good or ill. As @CannotSleep420 mentioned, coordinating on adding them to our blocklists and isolating them is the best option.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • announcements@lemmy.ml
  • localhost
  • All magazines
    •
    Loading…
    Loading the web debug toolbar…
    Attempt #

    Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 18878464 bytes) in /var/www/kbin/kbin/vendor/symfony/http-kernel/Profiler/FileProfilerStorage.php on line 171

    Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 10502144 bytes) in /var/www/kbin/kbin/vendor/symfony/error-handler/Resources/views/logs.html.php on line 35