Lemmy Developer AMA and Dev Update, 2024-01-26, 1500 CEDT

This is a chance for any users, admins, or developers to ask anything they’d like to myself, @nutomic , SleeplessOne , or @phiresky about Lemmy, its future, and wider issues about the social media landscape today.

NLNet Funding

First of all some good news: We are currently applying for new funding from NLnet and have reached the second round. If it gets approved then @phiresky and SleeplessOne will work on the paid milestones, while @dessalines and @nutomic will keep being funded by direct user donations. This will increase the number of paid Lemmy developers to four and allow for faster development.

You can see a preliminary draft for the milestones. This can give you a general idea what the development priorities will be over the next year or so. However the exact details will almost certainly change until the application process is finalized.

Development Update

@ismailkarsli added a community statistic for number of local subscribers.

@jmcharter added a view for denied Registration Applications.

@dullbananas made various improvements to database code, like batching insertions for better performance, SQL comments and support for backwards pagination.

@SleeplessOne1917 made a change that besides admins also allows community moderators to see who voted on posts. Additionally he made improvements to the 2FA modal and made it more obvious when a community is locked.

@nutomic completed the implementation of local only communities, which don’t federate and can only be seen by authenticated users. Additionally he finished the image proxy feature, which user IPs being exposed to external servers via embedded images. Admin purges of content are now federated. He also made a change which reduces the problem of instances being marked as dead.

@dessalines has been adding moderation abilities to Jerboa, including bans, locks, removes, featured posts, and vote viewing.

In other news there will soon be a security audit of the Lemmy federation code, thanks to Radically Open Security and NLnet.

Support development

@dessalines and @nutomic are working full-time on Lemmy to integrate community contributions, fix bugs, optimize performance and much more. This work is funded exclusively through donations.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. Recurring donations are ideal because they allow for long-term planning. But also one-time donations of any amount help us.

silas,
@silas@programming.dev avatar

What has been the most rewarding part of working on Lemmy for you guys?

nutomic,
@nutomic@lemmy.ml avatar

The fact that there is no boss telling me what to work on. Instead I get to decide myself whats most important. Last year before the Reddit migration I was temporarily working for a company, and it was extremely demotivating to be told how to do every little thing as if I were a junior developer.

dessalines,

As a software dev, mainly that people enjoy using it.

99% of the proprietary software work I did for companies was work that was societally useless, and eventually thrown in the trash. Here I get to make software that improves peoples lives in a tiny way, and is a form of social media that hopefully 🤞 doesn’t destroy people’s mental well-being: is easy to put down, and enjoyable to use.

morrowind,
@morrowind@lemmy.ml avatar

If you haven’t read the book “bullshit jobs” yet, I really recommend it. Encapsulates this mentality very well

pocketman_stuck, (edited )

This is for all Lemmy devs:

Talk a bit about yourself, likes and dislikes.

Musical taste, movies, whatever, just willing to know a lil more about you as persons

nutomic,
@nutomic@lemmy.ml avatar

I mainly like Hip-hop, but also good music from other genres. I like all sorts of movies if they are well made, but especially adventure and comedy. Since the quality of Hollywood movies has gone steeply downhill, I mainly watch movies from different countries across the world, and older movies from the 70s or so. I also like video games, at the moment Im playing Baldurs Gate 3 as everyone was praising it on Lemmy (and they were right its a great game).

shootwhatsmyname,
@shootwhatsmyname@lemm.ee avatar

In your opinion, what are the top strengths and weaknesses of Lemmy at this point in time?

nutomic,
@nutomic@lemmy.ml avatar

Strengths: Its open source, decentralized and working quite reliably

Weaknesses: Theres not enough funding/developers to keep up with all the issues

dessalines,

Strengths are federation, comment threads, mod tools, app ecosystem.

Weaknesses are lack of notifications, DB code, and lack of funding / donations. We need to support much more that just one or two devs if we want to take on reddit, which is a multi-billion dollar company with hundreds of employees.

I discuss more things I’m going to work on this year in this comment

OsrsNeedsF2P, (edited )

How do you feel about extreme right-wing instances like the late Wolfballs using Lemmy to promote and spread hate?

WanderingVentra,

I hope they’re not too against it. I know they’re extremely left wing, which scared a lot of the centrists on Reddit. If they allow even right wing instances, then it emphasizes the project’s lakc of political bent, and encourages more mainstream people to join. The politics can be up to each individual instance to decide whether to defederate with those other instances or not.

But that’s just my opinion, I’m also curious how the devs will answer.

bdonvr,

I’m sure they’re probably not okay with it but also there’s not much they can do about it other than defederate .ml - such is the nature of open source software.

calavera,

Not that I’m suggesting it, but they could hardcode the defederation of those instances on the code and most admins wouldn’t bother to fork lemmy to remove it. Like in the past they had a hardcoded slur filter, but I think they disabled it because many slurs in English were regular words in other languages

possiblylinux127,

Yeah that’s a very slippery slope

Kaboom,

Wolfballs? Was it straight up hate or just run of the mill GOP talking points?

Moonguide,

Wasn’t here back when it happened, but from what I know, it was straight up white supremacy shit. It got defed to death.

possiblylinux127,

I think you should be careful with political hate. Its a slippery slope and should not be handled by a developer.

Maybe we could simply focus on having clear rules for each community.

nutomic,
@nutomic@lemmy.ml avatar

They were posting spicy memes but thats how the internet works. If you dont like it then dont visit there, just like you wouldnt visit 4chan. Lemmy is open source so anyone can use it for any purpose.

dessalines, (edited )

I very much dislike it obviously, and I’m happy that one shut down. There have been others, but for the most part they’ve stayed away from Lemmy as “that software made by tankies.”

Outside of making sure that we don’t platform them anywhere, there isn’t much we can do. Lemmy is open-source software after all, and a tool can be used for good or ill. As @CannotSleep420 mentioned, coordinating on adding them to our blocklists and isolating them is the best option.

OsrsNeedsF2P,

Back when the first Reddit exodus happened, there was a group heavily DDOSing many of the popular Lemmy instances. While it was a great opportunity to optimize Lemmy, did you ever find out who that attacker was?

phiresky,

I don’t think we found any specific groups of people attacking Lemmy. I personally just saw one or two what looked like individuals trying (and succeeding) to take Lemmy down with a few very simple requests that forced Lemmy to do lots of compute (something like fetching the next million posts from page 10000). The fixes for those were simple because it was just missing limits checking.

I’m not sure if there actually was a larger organized attack. Lots of performance issues in Lemmy simply appeared simultaneously and compunded each other with a rapidly growing number of active users and posts.

OsrsNeedsF2P,

First, I want to say thank you for the incredible job you already have done in this area. However, do you have any thoughts on further improving some fundamental Lemmy UX painpoints? Examples such as:

  • Migrating accounts between instances
  • Tagging users across instances
  • Linking communities across instances
  • Finding communities across instances
Blaze,
@Blaze@discuss.online avatar

Migrating accounts between instances

Isn’t that implemented in the 19.2 and later versions? I just migrated using that feature a few days back, worked quite well

WanderingVentra, (edited )

That would be awesome if true. It’s progressing faster than I thought. I’m still just learning about the scaled sort and enjoying that new feature lol.

Blaze,
@Blaze@discuss.online avatar

I’m pretty sure it is there, you can export and import your subscriptions in the settings

can,

It’s only subscriptions, blocks, and user settings iirc. Your posts and comments don’t migrate for example.

WanderingVentra, (edited )

Still that’s not bad. Wish they could get saved posts to transfer, too. That would be useful.

Blaze,
@Blaze@discuss.online avatar

Importing posts and comments could cause a security risk if someone would to abuse that function.

Even Mastodon doesn’t support it

Mastodon currently does not support importing posts or media due to technical limitations, but your archive can be viewed by any software that understands how to parse Activity Streams 2.0 documents.

docs.joinmastodon.org/user/moving/

nutomic,
@nutomic@lemmy.ml avatar

More importantly it would make exports extremely large and would cause a lot of server load to import/export. Plus you would end up with duplicate posts and comments which seems like a bad idea.

Blaze,
@Blaze@discuss.online avatar

Yes, that’s what I meant by security risk, that’s like a DoS feature.

JackbyDev,

Can you add one to your list? Linking posts across instances? Like you can do !community@instance and the community will open viewed through your instance. But for linking posts there is no such equivalent. Like if I make an HTTP link it will be through my instance or possibly the one the community is hosted on which would be annoying for users of other instances.

Also, linking communities across instances is possible already, but you can leave it up since it’s confusing. I still see a lot of folks try to do the reddit approach if c/community

spaduf, (edited )

Turning the fediverse button into an “open on my instance” with similar functionality to subscribing may also be a solution here. Bonus points if it’ll also open a comment on mastodon.

Die4Ever,

github.com/LemmyNet/lemmy-ui/issues/2318

spaduf,

Yup that’d be sick

Die4Ever,

github.com/LemmyNet/lemmy/issues/2987

spaduf, (edited )

Really like your protocol handlers contribution here. Seems tough to square with multiple accounts though.

phiresky, (edited )

For migration we recently added a feature to export your user data. But “real” migrating accounts is something I put on our “todo” list, though it probably also first needs a proposal to define how it should work exactly (should it still work when the original instance is down?) As soon as we start giving users more control over their private key issues start appearing like not having any infrastructure for key rotation / revocation. Without that it will only work when the original instance still exists.

I’m not sure if by tagging users you mean linking / mentioning them? Or adding tags to them like you can tag posts / users on other platform. For tagging in general there’s a pending proposal github.com/LemmyNet/rfcs/pull/4 . So far it focuses on post tagging though to reduce the scope. I think the goal is going to be to start with one kind of tagging and add more kinds of tagging later.

For improving cross-instance linking (both communities, posts, and users) we also have a open milestone. There’s a few spitballing issues about it, but no real concrete proposal on how to build it yet.

dessalines,

As @phiresky mentioned we have improvements coming down the pipe for linking content across instances.

Community linking and user linking do work currently (for example I just linked phiresky above), and a community example would be !risa , but we could improve this by extending it to posts and comments, as well as creating a url link standard that would work across apps.

OsrsNeedsF2P,

What other ideas do you have to increasing funding for Lemmy development?

nutomic,
@nutomic@lemmy.ml avatar

Wait for Reddit to implode, more users to migrate and donations to increase. It worked last year :D

syd,
@syd@lemy.lol avatar

lol

dessalines,

We could definitely use some help with ideas there. Lemmy currently has ~40k active users, and it should be able to support more than ~1 average dev salary, especially if we want to take on a multi-billion dollar company with hundreds of employees like reddit.

Ludrol,
@Ludrol@szmer.info avatar

I think short event or campaign with push for donations with a pop up that you actually can dismiss. An ad like banner. The biggest problem would be community organization as Lemmy isn’t only decentralized horizontally but also vertically. Different front ends, different apps different instances. Most of them wouldn’t want to implement an ad that wouldn’t benefit them directly. They also have costs with running their piece of lemmy. So some cut for them should be included.

I think a dedicated trustworthy person should be responsible for organizing this campaign as developer time is best spent elsewhere.

A_A, (edited )
@A_A@lemmy.world avatar

Wikipedia, about once a year, has those donation pledge boxes at the top of every article …they must be somewhat effective since they come back year after year …better keep them small though to avoid disheartening users. Maybe start small like this trending community line at the top of the user feed.
P.S. : Since we don’t want the user to get habituated it’s better if it’s just a few days once a year.

OsrsNeedsF2P,

Not a question, just wanted to let you know I how much we appreciate and love you all for making Lemmy happen 🥰🥰

nutomic,
@nutomic@lemmy.ml avatar

Thank you! Its great that you have been around all these years.

dessalines,

Thx a ton!

OsrsNeedsF2P,

What’s coming in the Lemmy 0.20 release?

nutomic,
@nutomic@lemmy.ml avatar

Any breaking changes which get implemented in the meantime. There are no specific plans.

OsrsNeedsF2P, (edited )

What are some cool “Lemmy Adjacent” projects you know of and want to share? (Things like LemmySchedule or Toast.ooo’s Canvas)

dessalines,

One that I can think of rn, is @CannotSleep420 's lemmy-bot, as well as ridoukousage’s TLDR bot.

With the web being so ad-infested and completely owned by google, people have noted how the TLDR bot means they often don’t have to leave their lemmy app at all, and can stay behind its privacy shield.

While of course I do think we can code a lot of functionality directly in to lemmy in a way that we couldn’t with reddit, there’s undeniably a lot of potential with bots that can do different things for us.

SubArcticTundra, (edited )
@SubArcticTundra@lemmy.ml avatar

Do you have any estimate of how much storage (in GB) all the posts ever posted across Lemmy have taken up, to date? (Excluding media)

1984,
@1984@lemmy.today avatar

Something tells me lemmynsfw has the largest disks. :)

Omega_Haxors,

They call me Bigdisk Johnson.

WanderingVentra,

I should donate to them. I mean… lemmynsfw, what’s that?

Blaze,
@Blaze@discuss.online avatar

The porn instance

morrowind,
@morrowind@lemmy.ml avatar

*A porn instance

WanderingVentra,

I’m just playing dumb. I know what it is lol

JaymesRS,

I was told it’s not about the size of the disk but how it’s used that really matters.

1984,
@1984@lemmy.today avatar

Yeah we are told a lot… :)

nutomic,
@nutomic@lemmy.ml avatar

The SQL table for posts is 1.6 GB on lemmy.ml, and 5.7 GB for comments. That probably accounts for a majority of content on the Lemmyverse.

SubArcticTundra,
@SubArcticTundra@lemmy.ml avatar

1.6GB is impressively small for anything by modern internet standards

dessalines,

To add, lemmy.ml’s entire DB compressed as a xz with -0 strength is about 3.7 GB. But that also includes the activity tables which aren’t vital.

phiresky,

I don’t think it’s that large. Text is very small and compressible compared to images. Well it depends on if you mean the actual database storage (uncompressed, with indexes) or a compressed copy of all the posts. You can see the post number in the URL, which on lemmy.world for this post is 11169622. That means there’s around 11 million posts total in lemmy.world’s database. If you assume each of them takes 0.5kB of storage that would be only ~ 5 GB of posts.

A_A,
@A_A@lemmy.world avatar

… is 11169622 …

Maybe 9 post out of 10 are deleted by the few checks I made manually …or am I missing something ?

OsrsNeedsF2P, (edited )

So many apps die before getting any users. For Lemmy however, when was the first time you really thought “Damn, this thing really might actually take off”?

nutomic,
@nutomic@lemmy.ml avatar

Lemmy was meant to be a Reddit replacement from the beginning, so it was always supposed to take off. Even in the early days the tech was working quite smoothly and users were happy so there was no real doubt about it. The only thing missing were more users. However I had no idea how a real migration would actually look like, so it was really overwhelming when last year people started to flood in and everything got overloaded and broke down.

dessalines, (edited )

For me it was long before the reddit migration (which was ~7 months or so ago). I noticed lemmy slowly but surely gaining traction. It felt more dead than it does now, but the trend was slow and steady growth, which is always a great sign. People were using lemmy, liking it, and sticking around.

At the same time, it was clear that we weren’t making the mistake of all the other reddit alternatives, by promising to be a free speech haven for bigoted communities. Those people actively did our work for us by warning their communities to stay away from Lemmy and its tankie devs, thereby making Lemmy a much more enjoyable place from the very beginning. That was a crucial test: we were not willing to sacrifice our values for growth’s sake.

It’s great to see that positivity confirmed by a researcher who did a qualitative and quantitative analysis about Lemmy migration, and finding that >90% of people saw themselves using Lemmy in the long term. We can all be very proud of that, and it means we have a bright future.

syd, (edited )
@syd@lemy.lol avatar

Sorry for taking up all the space :) But please skip the ones you wish.

  1. How old are you guys? Can be a range if you want e.g. 10-15, 15-20 :)
  2. Do you know each other IRL?
  3. Which region do you live in? Like America, Europe, Asia etc.
  4. If Lemmy had 100 million users, would you still try to remain anonymous?
  5. Is there an instance admin you hate but can’t say?
  6. What percentage of users should ideally be in the largest Lemmy instance do you believe?
  7. If you had the chance to change the name Lemmy, would you?
possiblylinux127,

I’d hope that they are at least 18

nutomic,
@nutomic@lemmy.ml avatar
  1. Im 31
  2. Not yet, hopefully that will change in the future
  3. North of Spain
  4. Im not really anonymous, it would just feel weird to post here with my full name
  5. No
  6. Maybe less than 50%
  7. Im terrible with names so no
BlanK0,

Are there any plans on adding features that enable easier interaction with other federated platforms like mastodon and peertube (for example being able to comment/interact with peertube videos and mastodon posts)?

nutomic,
@nutomic@lemmy.ml avatar

You can already interact with Peertube videos and follow their channels. Thats possible because Peertube also federates groups (communities). With Mastodon thats not possible because it doesnt have groups, and Lemmy doesnt support content outside of communities. At least not without a full rewrite, which doesnt make sense considering that KBin and dozens of different microblogging platforms already exist.

threelonmusketeers, (edited )

Where is the best place to propose new features for Lemmy?

Edit: And as potential follow-up, where is the best place on Lemmy to propose new features for Lemmy? (Not every Lemmy user has or wants a GitHub account)

can,

I’m not sur eif it’s bad form to answer these questions but for this one I think the github issue page is the answer.

dessalines,

This is correct, its our issue tracker and its what we work from to close issues.

Die4Ever, (edited )

if you don’t want to make a github account, then probably !lemmy

but you can still search and read the github issues without making an account, so you could check if it already exists, or link the issue if you’re starting a lemmy discussion about it

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • announcements@lemmy.ml
  • localhost
  • All magazines
    •
    Loading…
    Loading the web debug toolbar…
    Attempt #

    Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 18878464 bytes) in /var/www/kbin/kbin/vendor/symfony/http-kernel/Profiler/FileProfilerStorage.php on line 171

    Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 10502144 bytes) in /var/www/kbin/kbin/vendor/symfony/error-handler/Resources/views/logs.html.php on line 33