In looking up suggestions made already I found 2 other projects that might be useful. Does anyone have comments about these? I have just looked at them a little bit.
OfflineIMAP
OfflineIMAP is software that downloads your email mailbox(es) as local Maildirs. OfflineIMAP will synchronize both sides via IMAP.
There are a few different overlapping projects by same developer(s). It is a bit messy.
OfflineIMAP/offlineimap3 - I think this repo is the one with the most active and up to date version of the software
Imapsync is an IMAP transfer tool. The purpose of imapsync is to migrate IMAP accounts or to backup IMAP accounts.
Imapsync is a command-line tool that allows incremental and recursive IMAP transfers from one mailbox to another, both anywhere on the internet or in your local network. Imapsync runs on Windows, Linux, Mac OS X. “Incremental” means you can stop the transfer at any time and restart it later efficiently, without generating duplicates.
But Wayland’s technical merits are relevant in a subtle way. Wayland is maintainable. Xorg isn’t. That’s it, the single most important technical merit. Everyone works on Wayland. Nobody works on Xorg. If people decide to use X11 today, their issues are wontfix with the solution to use Wayland instead. They can’t fix the issues themselves because X11 is an unmaintainable mess. Xorg is on life support with the only purpose to serve Xwayland.
They hated her for she spoke the truth. We (DIY people) hate to acknowledge that not everyone sees value in investing as much time and energy into perfecting their workspaces as the nerds have, and would rather have their tools Just Work™ so they can get to work on the projects they do care about. I say this as someone who still gets frustrated and argumentative when my friends say they prefer spyware-ridden OSes that remove control from the end-user because they don’t require end-user micro-management to maintain and work.
X vs Wayland might as well be Grub vs rEFInd or systemd vs SysVinit to most end-users: it matters from a technical perspective, but most people just want something that will allow them to go about their business without sinking hours into getting the “correct” option to work. And it’s important to remember that we all fall on either side of this divide with some aspects of our life, even if it’s not computer-related. How often do we agonize over finding the “correct” pipe wrench when our sink is leaking, despite what the plumbing nerds would criticize you for using? Do you sink hours into picking the right books on conflict resolution when you argue with your spouse, or do you post on AITA and hope they give good advice? Do you agonize over having all the right utensils and ingredients so you can eke out the most subtle flavors from your cooking, or do you use the pan that you got at the local superstore?
For the four groups enumerated in the article, this still is not important. It may affect them in ways they are unaware of, but you will not be able to change their minds using technical arguments at this point if they have not already been convinced by the wealth of information and support that is readily available.
Protocols are fine. Clients may speak one or another protocol. But protocols aren’t broken when clients designed to speak one protocol fail to speak a different protocol. It’s like saying English is broken because my friend only knows German, except English is Wayland, German is X11 and my friend is clients. Wayland is always ready to listen to clients that speak Wayland.
Yeah, especially the argument about Wayland compositors taking down all apps with them when they crash: that’s just bad, no need to sugarcoat that.
A better argument would be that it’s not true: KWin keeps Qt apps alive, and they’re working on extending that to all apps. As a result, in the only crash I’ve experienced, I only lost my Firefox window, and zero data as all my tabs and form entry values got restored when I started it again.
@theshatterstone54 > "Reason for that being that Wayland is built with Linux in mind and would not work under FreeBSD without a lot of effort bwing put in as it uses some Linux-specific components or libraries."
Not so. FreeBSD is 100% Linux compatible and has Linux Kernel emulation built in. Wayland support is also built in to FreeBSD. FreeBSD is a much superior operating system compared to Linux. But the FreeBSD team only cares about the server aspects and really does not care about a graphical desktop. They tend to use Macs.
The state of Linux Desktop interface is a schizophrenic flustercuck with far too many cooks spoiling the stew. They're not just spoiling the stew; they're pooping in it. And a bunch of noveu-riche trust-fund baby nerds think this is cool. They don't give a rat's ass about the end user being able to get work done. They would rather we all waste our time filing bug reports rather than getting things done.
I think the real issue with audio work on Linux is the complexity of getting things working. MacOS and Windows are both much easier to work in with dealing with audio stuff and much like Adobe 's stranglehold on potential converts having to jump through so many hoops for an arguably worse experience really keeps some people at bay.
Oh yeah you’re right there, but what I’m getting at is having a system that does everything you want is, I would assume, preferred. I run Pop and have it setup with my Audient EVO and it works well with Reaper, but getting it to that point was a pain.
If you're choosing to do audio production in Linux, the odds are that "easy" wasn't your top decision criteria. lol
Personally, I recently hooked up my Berhinger USB audio interface to Mint, and Ardour and Audacity saw it immediately. I was impressed. I was ready to google around for how to use lusb and dmseg and shit because I never remember what I'm doing.
It is the POP3 workflow, not IMAP. Maybe setup your client to use POP3 and remove mails from server after receiving? However I don’t recommend Thunderbird, its POP3 support was very buggy when I used it (many years ago). Try Sylpheed or Claws Mail, for example.
I want to keep mail on the server at about 80-90% of quota. Because when I am outside of my home, that will continue to be what I have access to. So the local copy will only be as a backup in case I delete something that I later realize I need to refer to. Since most emails are very small individually I should be able to keep the majority of them on the server. I will selectively delete either very large emails, or emails which there are so, so many of like notifications, which I will probably never need to look at.
I have used Sylpheed a bit in the past. I prefer it and a very similar project called Interlink to tbird. I just said tbird because I figured everyone would know it. But also I thought all of those were forks of tbird and wouldn’t differ much in how they work. Do they have much different internals?
You are wrong, there are no widely used forks of Thunderbird AFAIK. Thunderbird is based on Mozilla and has a huge codebase that is very hard to maintain. All other popular email clients have totally different code and based on other libraries. They can be similar in how they appear, but not in what bugs they have.
A while ago I was trying out betterbird which actually is a TB fork and I guess I kinda just generalized from that. But looking through a list of linux email clients it is clear that only a couple are related to TB.
It is not the question of funding. Thunderbird has always had a number of long standing bugs. Speaking about such rare use cases, I don’t think someone care about them. Anyway, I recommend using software that I know it worked correctly, not that worked incorrectly and could be fixed but requires further testing.
So I think the way I would want to do this is with something like mailpiler (www.mailpiler.org). It’s been on my long list of things to dive into for a while.
Well it is literally exactly what I was asking for. :) But as you allude to the setup is not trivial and would be a bit of a project. It is useful to know about because it could help find a somewhat simpler alternative. And I will add it to my own list in case I find none.
edit:
Led me to https://github.com/polo2ro/imapbox. Which is a different take on the same problem. I am not sure if I like the email all being converted to html like this. It could be a really nice addition but somehow I feel that keeping more original-formatted emails would be wise too. It does also create for each message “A gziped version of the email in .eml format” alongside the html but I would have to look more into what can be done with that.
Alas, no! Things seemed to be going well: I got >90k messages imported from my Google Takeout mbox file before the import was interrupted (not mailpiler’s fault). At this point, I logged into the “auditor” account and was able to see my emails and search them. But, then I resumed the import. By the end of today, the import was finished (~150k messages total). When I logged in with the auditor account, I got some error “No search results” and nothing I could do about it. This is actually what happened last time I tried mailpiler, too, now that I recall. All seemed fine, but, it seems, the database got corrupted or something along the way… So, now it’s useless. I might try it one more time over the next few days. I’ll keep y’all posted.
This kind of tool needs to be something you can rely on if it’s to be used in the way I am intending. If there is a master copy of the mail (as it sounds like you are working from) it’s not as big a deal as you can always go back to that. But if the application is relied upon to be doing its job, possibly in silence for long stretches, it can’t just combust.
I am not sure I really like the word “database” in this context. I don’t understand them and I can’t fix them. Am feeling that maildir, where each email is simply a text file, should be the primary storage. If there is another tool that can index or interact with the maildir then that’s handy, but the mail itself should stay in a plain, interoperable filetype. (Unless that is how mailpiler works? I might be mis understanding.)
I also see that mailpiler encrypts everything. I do not love that. My hdd is already encrypted. I do not want things further encrypted because it also means I am unlikely to be be able to fix any problems.
I think this application is too complex for me. I need something that I can easily administer. Hopefully set up and leave it to be for a long time and not have too much to relearn if something needs to be fixed. It is perhaps suitable for a more advanced user/admin.
Yes, I’m coming to similar conclusions myself. To be fair, encryption is a configurable option with Mailpiler. But, yes, it is all digested and stored in a mysql database, which is definitely more opaque than plaintext in the filesystem. I might try the mutt + notmuch solution described by @marty_relaxes below. Sounds like it might be a challenge to set up but would work great forever after. I’ll need to figure out how to convert my mbox files to maildir, but Google suggests there are tools for that. Good luck to you, let us know what you ultimately figure out! I’ve been working on this off-and-on for a few months now without figuring our a good solution!
Edit: I guess, if you want fast full-text search, a database will have to enter the equation somewhere, though.
Honestly i could live without fast. If its a text file there is always grep, ripgrep, silver searcher etc. But there is nothing in my deleted email demanding immediate attention. Any situation i forsee would accommodate waiting hours or days. I was kind of hoping to continue interacting with it in a webmail kind if way because piling up too many new things for something i wont be working on regularly is just asking for a mess.
The mutt/notmuch proposal is a solid solution for the right person. To me, learning like 5 new major tools just for one project is a big risk. I played around with this stuff a couple years ago and failed at creating even a simple setup to do regular mail stuff. It is absolutely not clear.
So i might try one if the intermediate solutions mentioned elsewhere. A solution that digests mail be acceptable as an addon extra.
Well, I’ve solved it! I now have a web interface (accessible via VPN, although, in principle, I could expose it to the internet) that allows fast, full-text search of all my old emails. Here is the recipe:
Installed notmuch via my distro’s repository and set it up (notmuch setup & notmuch new). This creates a new folder in your maildir directory containing full-text search info.
Installed netviel via python3 -m pip install netviel and then ran it via python3 -m netviel
That’s it! This let’s you search locally. I actually did a few more steps because I wanted to containerize this thing so I could run it on my NAS. I’d be happy to go into detail about that too, if you’re interested. One hiccup was that, for some reason, netviel binds to 127.0.0.1 instead of 0.0.0.0, and there is no way to change that without compiling the project yourself. But, I found a workaround for my Docker container where you can use socat bound to 0.0.0.0 to redirect requests to netviel, so that requests from other computers appear local to netviel.
Anyway, that makes it all sound more complicated than it is. I am super-pleased to have solved this problem at last!
@leo KDE with Wayland was all crashy when I tried it. If Wayland windowing is as buggy and crashy as their browser we'll all need to switch to Windows or Mac just to get any work done.
I’m daily driving Firefox with Wayland on KDE Plasma since years, not on Xwayland, and can’t remember it not working well. This on two different distributions (Arch and NixOS). Not saying this is your fault but your experience is not representative for everyone
@Laser My experience is representative for enough people to show that Linux Desktop is a mess and is not suitable for production work. I don't identify myself by my choice of software. I just want to get work done.
@crypto@Laser Linux desktop is not one thing. If you have a company that standardizes on Gnome, then the software you need to work will work as they will likely have been tested to work. As for work, well, not everyone uses it for work.
I suppose it really depends on when you tried it. Ubuntu 23.10 has been working quite well on Wayland. I haven’t once failed down to X, and the only papercut I run into now is with differently scaled displays (100% and 150%) where OBS will crash the session when moving back and forth.
Everything else seems good as I haven’t really seen anything else break at all and I use Firefox, Kdenlive, Audacity, lots of chat apps, and played some games. Specifically, playing Vivaldia 2 while I was remotely compiling Gentoo using OBS to livestream.
When piping output of find to xargs, always use -print0 option of find and -0 option of xargs. This allows processing files with any allowed characters in names (spaces, new lines etc.). (However I prefer -exec.)
There’s an i command to insert a line in sed, it is better to use it instead of s/^/…n/. It makes code more readable (if we can talk about readability of sed code, huh).
If you want to split a delimiter separated line and print some field, you need cut. Keep awk for more complicated tasks.
agree with one and two and younger me would have agreed with your third point but I think I don’t anymore.
yes cut is the simpler and mostly functional tool you need for those tasks.
but it is just so common to need a slight tweak or to want to substitute something or to want to do a specific regex match or weird multi character delimiter or something and you can do it all easily in awk instead of having to pipe three extra times to do everything with the simplest tool.
Nvidia appeared fairly buggy as of nvidia 535 and kernel 6.3 with both sway and Plasma 5.27. Notably of all the possible choices for Wayland support ONLY KDE in relatively recent releases supports proper scaling of apps using xwayland which are apt to be a thing for a while now. This is a huge point in KDE’s favor despite loving the idea of an i3 like experience with sway.
If prior experience bears out plasma 6.0 will be buggy as fuck and 6.2 will be excellent.
Nouveau has NEVER been a particularly good choice and its primary developer just resigned www.phoronix.com/news/Nouveau-Maintainer-Resigns I wouldn’t pin my hopes on it in the future becoming usable. I sure as hell wouldn’t say its a useful choice NOW because you suppose it may become so in the future. I’d rather look at nvidias official open source effort.
If I had a crystal ball to look in I bet it would say a lot of folks with existing Nvidia hardware are best off sticking with X11 in 2023 but looking again at KDE’s wayland session in 2024.
Although do bear in mind people using stable distros like Ubuntu/Mint/Debian will be a lot longer seeing new useful features pushed out.
I always interpreted that as a factor of the Plasma team being willing to offer compatibility for things that broke the freedesktop spec.
Whereas Gnome / Mutter for example appear to believe that if they don’t strictly follow spec it’ll perpetuate the fragmentation.
I tend to side with the latter perspective but use KDE + kwin on my desktop for gaming for Wayland + vrr (it’s amazing how smooth and responsive this is). Gnome really shines on the notebook form factor so I use it there.
NVidia user here, it most certainly is not working well. My external monitor for my laptop is getting black boxes shadowing the kde menu and most of my windows on that screen, and often block boxes trailing the mouse.
Personal anecdote: I connected my guitar to my shitty sound card a few weeks ago, ran guitarix (because real DAWs are overwhelmingly complicated and I just want an amp, a compressor, and some reverb), and thanks to PipeWire and pipewire-jack everything ran perfectly. Low latency, no crackling, no messing with jackd or ALSA, no restarting audio daemons, I could simultaneously play audio through Firefox and hear my guitar. I dare say that that part of the audio stack is now a solved problem.
I’m not a musician though so I can’t comment on hardware support for exotic sound/midi cards or the maturity of FOSS DAWs.
As far as exotic stuff goes, I only buy stuff that’s class-compliant so I don’t have to worry about the manufacturer sunsetting support in the future. Supporting those sorts of devices should be a priority over anything with weird proprietary issues (fuck you IK Multimedia!).
It was definitely easier to create music with my current tools when I used Windows and Cubase! I left Windows behind for good and haven’t been able to scratch that itch.
Would love to be able to get back to low-latency and tools I understand. I haven’t been breaking my back trying, but I spent a few days with different DAWs and not really getting anywhere close. If it became easier, I’d get my MIDI kit back out and my USB audio interface and mics and start creating music again. I’m no Dev, but a creative lacking the tools for expression.
Good to hear that something like this is possibly getting moving.
Thanks I am looking at these. Do you think maildir format is the best to try to work with? When I was researching I find there are other formats such as mbox, or more program-specific formats. I was not having an easy time discerning which is the most portable, robust format.
I havent looked into these other formats because maildir works for me. I can saxe local backups, remoxe mail from the serveg, and even put it back later. All plain text.
Mutt (and neomutt) has very nice search capabilities, supporting regex search within specific mailboxes. However, it is a relatively slow search - unbearably slow for full text search in large mailboxes.
Here, notmuch is usually used to complement mutt. It’s a very fast (full-text) mail indexer, which can be directly integrated in mutt and allows much faster searching (among other things such as advanced mail tagging, virtual mailboxes and more).
It is generally a royal pain to set up with so many moving parts but once you do it is a very fast, comfortable mail environment if you’re comfy with the terminal.
Using un*x since the 90s, this is all I know. I like awk but it can go fucking complicated, I once maintain a 5000 lines script that was parsing csv to generate JavaScript…
Someone used the wrong tool for the job. If an awk script gets more than a few dozen lines, it’s time to use another language/tool to process that data.
At that point I'd be looking for languages that have libraries that do what I need. Both Python and Perl have online repositories full of pre-written things. Some that can read CSV and others that can spit out JSON. It's then a matter of bolting things together, which, hopefully, is a few lines of code rather than 5000.
There are even awk repositories, but I'm not sure there's a central, official one like PyPI or CPAN.
I now have perfect wayland setup with a Nvidia GPU. I just use my AMD Apu as main gpu and the nvidia one as secondary GPU. The DE runs on Amd and games run on Nvidia. Thanks for nothing Nvidia, making me work around your bs.
linux
Hot
This magazine is from a federated server and may be incomplete. Browse more on the original instance.