Lemmy Developer AMA and Dev Update, 2024-01-26, 1500 CEDT

This is a chance for any users, admins, or developers to ask anything they’d like to myself, @nutomic , SleeplessOne , or @phiresky about Lemmy, its future, and wider issues about the social media landscape today.

NLNet Funding

First of all some good news: We are currently applying for new funding from NLnet and have reached the second round. If it gets approved then @phiresky and SleeplessOne will work on the paid milestones, while @dessalines and @nutomic will keep being funded by direct user donations. This will increase the number of paid Lemmy developers to four and allow for faster development.

You can see a preliminary draft for the milestones. This can give you a general idea what the development priorities will be over the next year or so. However the exact details will almost certainly change until the application process is finalized.

Development Update

@ismailkarsli added a community statistic for number of local subscribers.

@jmcharter added a view for denied Registration Applications.

@dullbananas made various improvements to database code, like batching insertions for better performance, SQL comments and support for backwards pagination.

@SleeplessOne1917 made a change that besides admins also allows community moderators to see who voted on posts. Additionally he made improvements to the 2FA modal and made it more obvious when a community is locked.

@nutomic completed the implementation of local only communities, which don’t federate and can only be seen by authenticated users. Additionally he finished the image proxy feature, which user IPs being exposed to external servers via embedded images. Admin purges of content are now federated. He also made a change which reduces the problem of instances being marked as dead.

@dessalines has been adding moderation abilities to Jerboa, including bans, locks, removes, featured posts, and vote viewing.

In other news there will soon be a security audit of the Lemmy federation code, thanks to Radically Open Security and NLnet.

Support development

@dessalines and @nutomic are working full-time on Lemmy to integrate community contributions, fix bugs, optimize performance and much more. This work is funded exclusively through donations.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. Recurring donations are ideal because they allow for long-term planning. But also one-time donations of any amount help us.

hendrik, (edited )

Since I read a few comments here… What is your oppinion on more democratic platforms? I mean something like electing moderators. (Or dropping them in a democratic process.) Or voting for other things in a community.

(This is more a hypothetical question. I guess with the architecture as is, it can easily be exploited. And there is no way to implement this properly without severe changes and consequences.)

dessalines,

We’ve talked about it elsewhere, on github and one of these Q and A’s, but the main thing is that these things can be easily gamed. IE: making a ton of fake users to vote for a malicious mod.

There’s a ton of ways that could be exploited that I can’t think of rn. I’d rather not stress test them in lemmy.

mlg,
@mlg@lemmy.world avatar

Is there any new p2p/decentral technology that is trying to advance beyond federation?

It would be cool to have a generic framework to make web resources that are inherently decentralized without the need for sponsor and hosting.

Like IPFS but as a social site. Everyone helps partially host content in exchange for access to all the content.

Danterious, (edited )

There are different software that could help this happen but I’m not sure how they’d handle all of lemmy’s content

There is Gun.js and OrbitDB (OrbitDB runs on IPFS using Helia)

and there is FreeNet which allows hosting of freesites on their network if you have something downloaded to your computer.

Some of the problems could be mitigated by volunteers running relay nodes hosting the kind of content they want to stay up (this would work with Gun.js and OrbitDB since it allows subscribing to specific data) or just general relay nodes that gives the network more capacity (this would be the method if you are working with FreeNet)

Again not sure how easy or hard this would be so I hope the developers share their thoughts.

Edit: When I say relay nodes hosting I don’t mean they have control over the data but instead keeps the data online. The data is still stored on the users own device and is shared between users if both are online at the same time.

phiresky,

There is a ton of decentralized projects that no one has really ever heard of, new ones pop up all the time (I was watching multiple of them in the past). Sadly in most cases it seems like most authors stop working on their projects after a while.

The same ideas have existed for a long time but both decade old projects (ever heard of Freenet? Probably no) and new ones . Many of them are very ambitious and try to replace huge swaths of things (not just file storage but also social aspects, web of trust, etc) but then collapse under the complexity. IPFS is the most well known new project and (good imo) has limited its scope, but sadly (still) suffers from huge scalability issues, some of which are deep in the design.

I think it’s really hard to align incentives there - the nicer it is the harder it is to make money with it. So either these projects tend towards control by one entity or they tend towards death.

Really the only one that seems to have a long lasting life so far is torrents. Which are amazing. And Email if you want to count that.

dessalines,

Torrents are truly one of the best inventions of the internet.

They’ve fully solved the static data distribution problem, in a way that’s resilient and practical. I do a few torrent-related side projects, and I’m also super-interested in how we could integrate them into lemmy UIs and apps in order to take on YouTube.

Omega_Haxors,

I’m fed up to shit with youtube’s bullshit. Between their excessive advertising, unmoderated comments and far-right bias I want so bad to leave that shit and never come back.

dessalines,

I take a deep breath before I enter any youtube comment section that isn’t a cooking video lol.

Danterious,

I’d love to see that happen.

jawa21,

I post a fair amount of video edits. I’ve had quite a few people say that video playback is far from ideal for not just Lemmy, but the Fediverse as a whole. Is this mostly a 3rd party app thing, or a backend issue? I haven’t had much issue myself, but enough people have mentioned it that there is likely an issue somewhere down the line.

simple,

but the Fediverse as a whole.

FWIW video playback on Mastodon has been seamless for me.

nutomic,
@nutomic@lemmy.ml avatar

It works pretty well on Peertube, only lacks users and quality content.

jawa21,

I’m sure. However, I am not in a position to self host and videos are huge unless you post them as webm which some people might even report you for.

dessalines, (edited )

I have ambitious plans about how to turn lemmy into a true Youtube / Spotify killer. But that might not be what you’re asking, which might just be simple browser / app video support (IE mp4s, webm, etc).

If that app / web UI has trouble with videos, you might need to open up an issue there.

OsrsNeedsF2P,

A long time ago, @Dessalines made Jerboa as an Android Native client for Lemmy as an alternative to Boost for Reddit. How happy are you that the OG Boost developer came and made a Lemmy client?

WanderingVentra,

He did? That’s awesome. I’ve been using Jerboa and Sync mostly. How does the Boost app compare to those? Sorry if too unrelated lol.

Blaze,
@Blaze@discuss.online avatar

Feel free to have a look at all the others: lemmyapps.netlify.app

dessalines,

Its a great thing, and its exciting to have @rmayayo creating an app for Lemmy.

Blaze,
@Blaze@discuss.online avatar

Looking forward to it. Hopefully people will stay respectful.

iso,
@iso@lemy.lol avatar

Seeing my PR here made me feel good. 3 months and ~60 commits for only one lil field was too much 😅

It’s very nice and reassuring that all the commits are audited subtly though.

hendrik, (edited )

When and how are you going to address the thousands of open issues in the Github repository, that contain UI bugs, missing error messages (something looks as if it was sent for example if you send a direct message with too many characters, but actually isn’t), backend issues and other assorted bugs?

nutomic,
@nutomic@lemmy.ml avatar

When we have about a dozen more developers. So far only dessalines and i work on Lemmy fulltime, and besides solving issues we also have to review pull requests, prepare releases and much more. So its just not enough time to keep up with all the new issues let alone resolve the whole backlog.

hendrik, (edited )

Thank you. A follow-up question: You sound like most things have to be done by full-time developers. Is there a healthy open-source community around Lemmy development? Do people submit enough pull-requests to fix bugs? Do people from the community contribute a substancial amount? features?

dessalines, (edited )

We’re no different from 99% of open source projects: there are a lot of one-off contributors that just do a feature or two they’d like to have, but the vast majority of work is done by a handful of core devs. This is why you should always base your infrastructure and decisions to support those devs, rather than cater to one-off contributors.

hendrik, (edited )

I hope those wants and needs aren’t mutually exclusive. I think most open source projects do a good job in catering for both. I’m not involved in Lemmy development so I don’t really know what’s going on here. But I’ve sent one-off contribution to various projects, sometimes contributed single features or helped to sort something out. It always felt appreciated.

Sure, a drive-by commit every now and then and no responsibility is a completely different level than maintaining a (large) project and putting in that effort and dedication. I think a healthy open source project has both. Maintenance and the responsibility/decisions by a core team. And the community contributions make up by adding diversity, being close to what the user needs and adding manpower by a larger group of people, meaning the individual contributions might be smaller, but by many more people. Good communication between the devs and the community usually helps to get quality contributions.

nutomic,
@nutomic@lemmy.ml avatar

Of course contributions by volunteers are also welcome. However there are very few of those who are consistently contributing (particularly phiresky and sleepless one mentioned in op). And because they have a fulltime job their contributions are much smaller than mine or dessalines’. After the Reddit migration lots of people opened pull requests to implement new features, but most of them were abandoned after noticing how much work it takes to address review comments and actually get the pr merged. So fulltime devs seem very much preferable because they can put their full attention to Lemmy, and get a lot more done.

J12,

Will there be flairs?

phiresky,

I think flairs would be the same as user-tagging. There’s an open proposal for post-tagging github.com/LemmyNet/rfcs/pull/4 and the discussion there was so far to add tagging for one type of thing and then later expand to others (like user tagging).

It’s a bit of a complicated feature because it needs decision who can tag whom, and what is the scope (who is it federated two), and how does it transfer / interact with other ActivityPub software.

Danterious,

What is currently in the works to help admins locate spammers and problematic users on their instance?

Right now I believe it relies heavily on users reporting and admins looking through a users history however I think that is really inefficient.

Are there any better visualization tools that could be made to aid admins?

nutomic,
@nutomic@lemmy.ml avatar

How would you suggest to make it more efficient?

Danterious, (edited )

Well I am not 100% sure on what the best method would be.

I think the addition of letting moderators see who is voting on what posts is a nice addition since they would tend to be closer to where the content is.

More generally I was thinking about creating a dashboard that gave the admins statistics about that user.

Similar to how we as users can see how many comments and posts any user makes, it might be useful to allow admins to see statistics about which communities the user interacts with or some rudimentary similarity score with other users?

These are some ideas I thought of on the fly and have not thought about the implications but the gist is something that allows the admins to get a better understanding of what is going on from like a bird eye’s view.

Edit: and where -> on what posts

nutomic,
@nutomic@lemmy.ml avatar

The main implication is that someone would have to implement it, and we dont have enough developers for that.

Danterious,

Yeah that’s fair. It was just more of a possible what if.

Hopefully you’re able to get more people that can help develop the features that are in the backlog.

Anyways thx for answering. I was half expecting it to be skipped.

dessalines,

Hrm… I remember my time as a reddit mod, community statistics were very useful in making mod decisions. But that was only because reddit hosts so many vile communities, which isn’t a problem we really have at the moment.

dessalines,

Registration applications, and user reports are the best way to handle trolls. The first stops 90% of them, the second means we can ban and remove all their spam at the click of a button.

I don’t see how you could prematurely know about spammers or trolls until someone reports them. We don’t plan on adding any text-analyzing AI or anything like that into lemmy’s codebase.

Danterious, (edited )

I don’t see how you could prematurely know about spammers or trolls until someone reports them.

I don’t think you can. My suggestion was more focused on how admins make decisions after a report. Right now they have to do a manual scan of the person’s comment history and that is the part I find inefficient. If it was possible to just show extra high level information on the user it might make it easier for the admin to make a decision.

We don’t plan on adding any text-analyzing AI or anything like that into lemmy’s codebase.

Yeah using AI to try and analyze comments would be overkill and probably prone to manipulation anyways.

Edit: I’m sorta talking more specifically towards banning a user or seeing if what a user is doing is a repeated pattern.

dessalines,

I don’t think there’s a way you could avoid going into their history. I do that as an admin to verify that the account in question is indeed repeatedly breaking rules. I’m open to suggestions tho.

OsrsNeedsF2P,

What other ideas do you have to increasing funding for Lemmy development?

nutomic,
@nutomic@lemmy.ml avatar

Wait for Reddit to implode, more users to migrate and donations to increase. It worked last year :D

syd,
@syd@lemy.lol avatar

lol

dessalines,

We could definitely use some help with ideas there. Lemmy currently has ~40k active users, and it should be able to support more than ~1 average dev salary, especially if we want to take on a multi-billion dollar company with hundreds of employees like reddit.

Ludrol,
@Ludrol@szmer.info avatar

I think short event or campaign with push for donations with a pop up that you actually can dismiss. An ad like banner. The biggest problem would be community organization as Lemmy isn’t only decentralized horizontally but also vertically. Different front ends, different apps different instances. Most of them wouldn’t want to implement an ad that wouldn’t benefit them directly. They also have costs with running their piece of lemmy. So some cut for them should be included.

I think a dedicated trustworthy person should be responsible for organizing this campaign as developer time is best spent elsewhere.

A_A, (edited )
@A_A@lemmy.world avatar

Wikipedia, about once a year, has those donation pledge boxes at the top of every article …they must be somewhat effective since they come back year after year …better keep them small though to avoid disheartening users. Maybe start small like this trending community line at the top of the user feed.
P.S. : Since we don’t want the user to get habituated it’s better if it’s just a few days once a year.

calavera,

Hi again,

1 - Now that there are multiple apps for lemmy, wouldn’t be better to handover jerboa to someone else and focus more on core lemmy?

2 - How was the inception moment for lemmy? When and how did you decided to leave reddit and create something new and how was the first couple months?

Thanks and come to Brazil ;)

dessalines,

Thx!

  1. Jerboa has another major contributor now besides me. But I’d also like to keep a hand in it, simply because I made it and its also the main app I use for lemmy. Besides a few other people, no one has stepped up to get involved with it enough that I’d. Plus I’d eventually like to add it to the LemmyNet repo, as it will soon implement all of the features in lemmy-ui, making it an “official” and native app.
  2. Reddit has been banning communist / leftist communities for a while, so a lot of us were looking at different reddit alternatives. I found that none of them had a great tech stack, none of them had federation, and a few were unwelcoming to contributions. It made sense to do a “what if reddit, but open source, made in rust, and self-hostable”. It was a great opportunity to teach myself rust also.
Inui, (edited )

I’m not sure if this will entirely make sense, but do you believe there could be some kind of feature to authenticate with a specific instance, but make your ‘home’ instance another?

In other words, to create your account on lemmy.ml, but visit lemmy.world and be signed into your account, effectively changing what your home page and ‘local’ looks like without needing to subscribe to all communities on another instance if you’re interested in having them curate your feed? Piped has something like this, where you auth with one server, but proxy video through another.

The reason I ask is that this helps to get around tricky defederation situations and can improve privacy. Perhaps I want to auth from a server I trust more with my data, but I want to make .world or .ml my home page because they have more content that is relevant to me and that helps me discover it. Or I want to auth from lemmy.ml so I have access to instances that were defederated from .world, meaning I can still use them as my ‘home page’ without needing to actually make an account there.

nutomic,
@nutomic@lemmy.ml avatar

So basically Oauth support? Theres an open PR for that.

github.com/LemmyNet/lemmy/pull/4238

threelonmusketeers, (edited )

Where is the best place to propose new features for Lemmy?

Edit: And as potential follow-up, where is the best place on Lemmy to propose new features for Lemmy? (Not every Lemmy user has or wants a GitHub account)

can,

I’m not sur eif it’s bad form to answer these questions but for this one I think the github issue page is the answer.

dessalines,

This is correct, its our issue tracker and its what we work from to close issues.

Die4Ever, (edited )

if you don’t want to make a github account, then probably !lemmy

but you can still search and read the github issues without making an account, so you could check if it already exists, or link the issue if you’re starting a lemmy discussion about it

silas,
@silas@programming.dev avatar

What has been the most rewarding part of working on Lemmy for you guys?

nutomic,
@nutomic@lemmy.ml avatar

The fact that there is no boss telling me what to work on. Instead I get to decide myself whats most important. Last year before the Reddit migration I was temporarily working for a company, and it was extremely demotivating to be told how to do every little thing as if I were a junior developer.

dessalines,

As a software dev, mainly that people enjoy using it.

99% of the proprietary software work I did for companies was work that was societally useless, and eventually thrown in the trash. Here I get to make software that improves peoples lives in a tiny way, and is a form of social media that hopefully 🤞 doesn’t destroy people’s mental well-being: is easy to put down, and enjoyable to use.

morrowind,
@morrowind@lemmy.ml avatar

If you haven’t read the book “bullshit jobs” yet, I really recommend it. Encapsulates this mentality very well

interdimensionalmeme,

When will there be default view agglomeration of posts sent to identically named communities. For example /c/books. The current setup cntralizes power into the hands of whoever gets traction first on the platform. If I go to /c/books on any server, all posts of all federated servers’ /c/books should be visible. This way no server owner gets the stranglehold on the community that they host.

ademir,
@ademir@lemmy.eco.br avatar

I think this is interesting but should not be an automatic feature.

interdimensionalmeme,

If it is not the default and automatic, then lemmy is a pointless reddit clone.

You have to filter out what you don’t want because it is not possible to undelete what has already been deleted.

Users will just circulate ready made blacklists of spammer and thoughtcriminal communities to automatically remove them all from their feed.

The alternative is that only the biggest instance and the biggest community will matter and writing everywhere else is just a exercise in pointlessness

Blaze,
@Blaze@discuss.online avatar

The current setup cntralizes power into the hands of whoever gets traction first on the platform.

There are other factors at hand, such as the moderation and the instance politics

interdimensionalmeme,

Which is another centralization incentive.

Don’t want to be ostracized because your user is registered on the wrong politic instance ? Join biggest instance instead.

Going to the biggest local community of the biggest instance is always the way of least resistance.

And that’s how you make a worse reddit with extra steps.

Blaze,
@Blaze@discuss.online avatar

Don’t want to be ostracized because your user is registered on the wrong politic instance ? Join biggest instance instead.

There are plenty of politically neutral instance. Most of them are, actually, the only ones that come to mind as politically oriented are hexbear, lemmygrad and to an extend, lemmy.ml.

That leaves lemm.ee, sh.itjust.works, all the feddit.country, discuss.tchncs.de, sopuli.xyz, reddthat.com, lemmy.zip as neutral alternatives

spaduf,

Another option here is FEP-d36d which is a standard for group-to-group following. This looks to me like a slightly more organic and opt-in approach.

interdimensionalmeme,

Any opt-in approach will be irrelevant. Most user never change the defaults.

Example are “multireddit” feature. Statistically speaking, nobody used them and they never mattered.

Imagine a combination of /r/books /r/books2 and /r/books3

Owner of /r/books goes mad with power (as tgey all do) abd sells out the community.

So you post in /r/books2 because you use the multireddit, and if everyone else did, the defective owner would be transparently bypassed.

But what actually happens ?

To 99% if users in /r/books, you have simply ceased to exist. New users still to biggest /r/books and never know of the alternatives.

Multireddits are socially irrelevant.

The default MUST whole fediverse aggloneration which the users filters out what they don’t like out of

By manual removal of individual communities

By including or subscribing to circulating blacklists of communities (think spamfilter lists)

And by the owner of their instance defederating from other servers.

spaduf, (edited )

I think one thing you’re missing here is that under such a system the defaults would likely become your locally hosted /c/books rather than the largest one. Even still you’d probably see posts from the largest books communities because /c/books@your_instance follows multiple /c/books@big_instance. Community blocking would likely still work as it currently does so any books communities that you were not fond of could still be blocked.

There is still the issue of where do you post and I think the answer looks something like:

  • Post in /c/books@your_instance if you want to talk to your neighbors
  • Post in /c/books@big_instances if you want to talk to everybody

Which is more or less how most people would decide where to post book stuff anyway.

interdimensionalmeme,

Yes, the majority of content would still come from bigcommunity/c/books, the crucial difference in that system is that posting in otherserver/c/books would get the same probabibility of being viewed by random and non logged users.

I cannot emphasize enough how important that is. It is the only way to break the stranglehold that bigcommunity/c/books will always have over almist every lemmy users.

Without this, this is just reddit all over again. Meet the new boss, same as old boss.

Blaze,
@Blaze@discuss.online avatar

Meet the new boss, same as old boss.

Who’s Lemmy’s spez?

The main difference of Lemmy compared to Reddit is the ability that communities have to walk away, as I explained in another comment: discuss.online/comment/5393546

hendrik, (edited )

I’d like that. I think some other platforms/projects have features like this. And on Lemmy some instances duplicate everything. For example beehaw.

Blaze,
@Blaze@discuss.online avatar

And on Lemmy some instances duplicate everything. For example beehaw

Are they not allowed to?

Beehaw exists for people who wanted a heavily-moderated space, and they seem to be doing well activity-wise. Do you want to force them with the rest of the instances?

hendrik, (edited )

Sure, that’s not the point at all. But wouldn’t it be great if the knitting community (for example) on beehaw.org, lemmy.ml, lemmy.world and feddit.de would be merged for me into one entity for a better browsing experience? And people wouldn’t post the same breaking news 3 times and the cross-posts always showed up 3 times in my timeline? (And sometimes it’s the same 30 people anyways that are subscribed to all of them so the cross-posting doesn’t add anything?)

I currently don’t have a good idea for a UI design for that. But I think a feature like that would add to federated platforms (if done right.) But nobody said you’re not allowed or it’s bad to open a dozen communities with the same name and topic on different servers. That’s perfectly alright. In the real world we also sometimes discuss the same topic with different people at different locations.

Blaze,
@Blaze@discuss.online avatar

But wouldn’t it be great if the knitting community (for example) on beehaw.org, lemmy.ml, lemmy.world and feddit.de would be merged for me into one entity for a better browsing experience?

Why wouldn’t they merge on one instance? Seems easier, and can be done today compared to having to ask the developers to implement a complex feature.

hendrik, (edited )

Is there a client that does that? Sorry I lost track of the different clients. But I’d like to try. I know Eternity (which I use on my Android phone) and the default webui can’t do that. But I haven’t tried all the options.

I don’t quite get your wording. If you mean similar communities should be merged in all cases, I think I’d disagree. People might want to subscribe to a specific community. And it’d be complex to figure out moderation etc, since the root of the platform is a federated architecture and this somewhat goes against that. I think it’d be more a UI / client feature, tied into a cross-posting mechanism.

nutomic,
@nutomic@lemmy.ml avatar

Then who would moderate this? And what if lemmygrad.ml/c/books wants to have different discussions from lemmy.world/c/books?

interdimensionalmeme,

Lemmygrad still can send all the kulaks to the gulags. But only when the discussion happening inside their hard drive. Aka “I take my ball and go home”

They do not get to silence the rest of the fediverse/c/books

dessalines,

See here.

Die4Ever,

see github.com/LemmyNet/lemmy/issues/818

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • announcements@lemmy.ml
  • localhost
  • All magazines
    •
    Loading…
    Loading the web debug toolbar…
    Attempt #

    Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20975616 bytes) in /var/www/kbin/kbin/vendor/symfony/http-kernel/Profiler/FileProfilerStorage.php on line 171

    Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 10502144 bytes) in /var/www/kbin/kbin/vendor/symfony/error-handler/Resources/views/logs.html.php on line 31