Iām currently watching the progress of a 4tB rsync file transfer, and iām curious why the speeds are less than the theoretical read/write maximum speeds of the drives involved with the transfer. I know thereās a lot that can effect transfer speeds, so I guess iām not asking why my transfer itself isnāt going faster....
SATA III is gigabit, so the max speed is actually 600MB/s.
What filesystem? For example, on my ZFS pool I had to let ZFS use a good chunk of my RAM for it to be able to cache things enough that rsync would max out the throughput.
Rsync doesnāt do the files in parallel so at such speeds, the process of open files, read chunks, write chunks, close files, repeat can add up. So you want the kernel to buffer as much of it as possible.
If you look at the disk graphs of both disks, you probably see a read spike, followed by a write spike on the target, instead of a smooth maxed out curve. Then the solution is increasing buffers and caching. Depending on the distro thereās a sysctl that may be on by default that limits the size of caches to prevent the āI wrote a 4GB file to my USB stick and now thereās 4GB of RAM used for it and it takes hours after finishing the transfer before itās flushed to the stickā.
Hi everyone. I have found many ghost comments in posts. Like one of the posts has 300+ upvotes and 28 comments but when I opened it, there were no comments. I tried different Lemmy apps and itās the same in all of them. Which leads me to believe that it has something to do with defederation done by Lemmy.ml. Which instance has...
Keep in mind, defederation is bidirectional. You can end up on an instance that doesnāt defederate anybody but is being defederated by some major instances and end up worse off. Also, communities are bound to an instance so even if your instance doesnāt defederate with another, the instance that hosts the community might, which also doesnāt solve anything.
Also lemmy.ml had to restore from backup monday because postgres shat itself, so if the post is from monday or around, itās possible it was simply lost due to the technical problems.
Thereās also some federation problems with 0.19.0 and 0.19.1, so itās possible itās been attempted to be delivered to lemmy.ml but failed due to load or whatever.
You didnāt give any details or examples so we can only speculate. We troubleshoot federation by establishing patterns, like from what instance are the missing comments from, what instance hosts the community.
Addendum: Iāve also been experiencing occasional ghost posts, and Iām on my own instance, so there might be some stuff going on thatās unrelated, because I sure didnāt do anything. If they were deleted or retracted I would see them because Iām admin, I see everything.
The ads come from an ad network where there is very little visibility into whatās going to be displayed in your app. And bad people also keep managing to get their ads published even though the ad network doesnāt allow them
And it all ties into the whole targeted advertising, where they also make sure very few people get the bad ad, and tries to target people they think may be more susceptible to these kinds of tactics. Depending on the amount of interactivity allowed, the ad can even display two different things if it deems you too savvy to fall for it.
Itās basically unescapable unless you only use apps without ads, or pay for the ad-free versions.
The whole advertising industry is sketchy, more news at 10.
If we allow derivatives, Iād say SteamOS despite being Arch. Itās putting Linux in non-technical peopleās literal hands and itās not a locked down and completely different platform that happens to run Linux like Android is. Itās almost designed by Valve to give people a taste of Linux by the addition of its desktop mode, and people that would be modding consoles are now modding SteamOS and learning how much fun an open platform can be. Iāve seen people from sales talk about their Decks on my work Slack.
Otherwise, NixOS, no contest. Itās been a really long time since weāve last seen a fundamentally different distro thatās got some real potential. For the most part, Arch, Debian and Fedora do similar things with varying degrees of automation and preconfiguring your packages, but theyāre still very package oriented. Weāve been mostly slapping tools like Ansible to really configure them to our liking reproducibly, answer files if your package manager has something like that. And then NixOS is like, what if the entire system was derived from evaluating a function, and and the same input will always result in the exact same system? Itās incredibly powerful especially when maintaining machines at scale. Updates are guaranteed to result in the exact same configuration, and theyāre atomic too, no halfway updated system the user unplugged the system in the middle of.
Yes but by doing so youāre using the same principles as MBR boot. Thereās still this coveted boot sector Windows will attempt to take back every time.
Whatās nice about EFI in particular is that the motherboard loads the file from the ESP, and can load multiple of them and add them to its boot menu. Depending on the motherboard, even browse the ESP and manually go execute a .efi from it.
Which in turn makes it a lot less likely to have bootloader fuckups because you basically press F12 and pick GRUB/sd-boot and youāre back in. Previously the only fix would be boot USB and reinstall syslinux/GRUB.
Itāll definitely run Kali well, Windows will be left without hardware acceleration for 2D/3D so itāll be a little laggy but itās usable.
VMware has its own driver that converts enough DirectX for Windows to run smoother and not fall back to the basic VGA path.
But VMware being proprietary software, changing distro wonāt make it better so itās either you deal with the VMware bugs or you deal with stable but slow software rendering Windows.
That said on the QEMU side, itās possible to attach one of your hostās GPUs to the VM, where it will get full 3D acceleration. Many people are straight up gaming in competitive online games, in a VM with QEMU. If you have more than one GPU, even if itās an integrated GPU + a dedicated one like is common with most Intel consumer non-F CPUs, you can make that happen and itās really nice. Well worth buying a used GTX 1050 or RX 540 if your workflow depends on a Windows VM running smoothly. Be sure your CPU and motherboard support it properly before investing though, it can be finicky, but so awesome when it works.
Well thereās your mistake: using VMware on a Linux host.
QEMU/KVM is where itās at on Linux, mostly because itās built into the kernel a bit like Hyper-V is built into Windows. So it integrates much better with the Linux host which leads to fewer problems.
Ubuntu imho is unstable in and of itself because of the frequent updates so Iām looking for another distro that prioritizes stability.
Maybe, but itās still Linux. Thereās always an escape hatch if the Ubuntu packages donāt cut it. But I manage thousands of Ubuntu servers, some of which are very large hypervisors running hundreds of VMs each, and they also run Ubuntu and work just fine.
Appimage for me ticks all the boxes for cross distro package as its very portable, simple to run, what are devs trying to do when creating snaps and flatpack?
It is very unfortunate. Itās fine to point out problems, but then when you become part of the problem, thatās not amazing.
Heās had the same meltdown with fuse2 being deprecated in favor of fuse3 which, guess what, also broke AppImage and we had a huge rant for that too.
Flatpak has a better chance of being forward compatible for the foreseeable future. Linux generally isnāt a very ABI/API compatible platform because for the most part youāre expected to be able to patch and recompile whatever you might want.
Distro packages and to some extent Flatpaks, use shared libraries which can be updated independently of your app.
So for example, if a vulnerability is discovered in say, curl, or imagemagick, ffmpeg or whatever library an app is using: for AppImages, this wonāt be fixed until you update all of your AppImages. In Flatpak, it usually can be updated as part of a dependency, or distributed as a rebuild and update of the Flatpak. With distro packages, you can usually update the library itself and be done with it already.
AppImages are convenient for the user in that you can easily store them, move them, keep old versions around forever easily. It still doesnāt guarantee itāll still run in distros a couple years for now, it guarantees that a given version will forever be vulnerable if any of its dependencies are because theyāre bundled in, it makes packages that are much much bigger than they need to be, and you have to unpack/repack them if you need library shims.
Different kinds of tradeoffs and goals, essentially. Flatpak happens to be a compromise a lot of people agree on as it provides a set of distro-agnostic libraries while also not shifting the burden entirely onto the app developers. The AppImage developer is intentionally keeping Wayland broken on AppImage because he hates it and wants to fulfil his narrative that Wayland is a broken mess that wonāt ever work, while Flatpak developers work hard on sandboxing and security and granular permission systems.
RAM is the kind of thing youāre better off having too much than not enough. Worst case the OS ends up with a very healthy and large file cache, which frees up your storage and makes things a bit faster/lets it spend the CPU on other things. If anything, your machine is future proofed against the ever increasing RAM hungriness of web apps. But if you run out of it, you get apps killed, hangs or major slowdowns as it hits the swap.
The thing with RAM is that itās easy for 99% of your workload to fit comfortably, and then thereās one thing you temporarily need a bit more and youāre screwed. My machine usually uses 8-12/32GB of RAM but yet I still ended up needing to add swap to my machine. Just opening up the Lemmy source code and spinning up the Rust LSP can use a solid 8+GB alone. Iāve compiled some AUR packages that needed more than 16GB of RAM. I have 16 cores so compiling anything with -j32 can very quickly bring down a machine to its knees even if each compile thread is only using like 256-512MB each.
Another example: my netbook has 8GB. 99% of the time itās fine, because itās a web browsing machine, and I probably average on 4GB usage on a heavy day with lots of tabs open. But if I open up VSCode and use any LSP be it TypeScript or Rust, the machine immediately starts swapping aggressively. I had to log out of my graphical session to compile Lemmy, barely.
RAM is cheap enough these days itās nice to have more than you need to not ever have to worry about it.
Yeah, itās not really advertised as an init system anymore. Itās an entire system management suite, and when seen from that angle, itās pretty good at it too. All of it is consistent, itās fairly powerful, and itās usually 10-20 lines of unit files to describe what you want. I wanted that for a long time.
I feel like the hate always comes from the people that treat the UNIX philosophy like religion. And even then, systemd is very modular, just also well integrated together: networkd manages my network, resolved manages my DNS, journald manages my logs, timesyncd manages my NTP, logind manages my logins and sessions, homed mounts my users profiles on demand.
Added complexity, yes, but Iāve been using the hell out of it. Start services when a specific peripheral is plugged in? Got it. Automatically assign devices to seats? Logindās got you covered, donāt even need to mess with xorg configs. VM network? networkd handles it. DNS caching? Out of the box. Split DNS? One command. Donāt want 2000 VMs rotating their logs at exactly midnight and trashing your ceph cluster? Yep just slap a RandomizedDelaySec=24h to the units. Isolate and pin a VM to dedicated cores dynamically? Yep itāll do that. Services that needs to run on a specific NUMA node to stay close to PCIe peripherals? Yep easy. All very easily configurable with things like Ansible or bash provisioning scripts.
Sure it may not be for everybody, but it solves real problems real Linux admins have to deal with at scale. If you donāt like it, sysvinit still works just fine and I heard good things about runit too. Itās an old and tired argument, itās been over 10 years, we can stop whining about it and move on. Thereās plenty of non-systemd distros to use.
Title. ālmao internet pointsā and all, but what is the point of participating in a community that sees assumptions and other commonly non-harmful commentaries/posts as ābadā this easily? Do folks in here are really that needy of self-validation, even if it means seeking such from something completely insignificant like...
I expected this to be āanother one of thoseā but actually from what my instance has about you, you were indeed correct. Gaming distros with exclusive features lmao.
IMO thatās some of the gamer logic bleeding over in the Linux side, now that Linux gaming is taking off. Theyāll do anything including install dubious Linux distros barely hanging together with duct tape for a perceived extra 2 FPS. Download software exclusively distributed on Discord? Hell yeah. Iām sure at least one of them boots with mitigations=off and itās not clearly indicated that it does.
Weāre seeing the same thing on the Windows side with modified Windows ISOs like the whole AtlasOS, that rightfully made some security experts sound the alarm. Some did things like completely strip off the updates, antivirus and firewall. Unless your system is exclusively running Steam and firewalled off the network, this is a certified bad idea.
Iād probably trust Nobara because the guy clearly knows his shit, but some of them really are just some other guyās riced up Arch snapshot. They may give the impression everything just works at first but Iāve definitely seen examples of it falling apart. Even bigger distros like Pop_OS! had major snafus like the whole Steam uninstalls your DE thing, and Manjaro still fucks up something basic every now and then. I tried some of them in a VM and they didnāt even install or boot correctly. Oh my fault that one only works for NVIDIA graphics cards not AMD, my bad.
Itās not worth arguing, itās a user base with vastly different goals than I do, just let them have their Bedrock Linux completely blow up in multi package manager hell and soon enough theyāll come running for a saner more reliable distro.
I always see new GTK apps popup on Flathub. I dont really care and think GTK looks fancy, although CSD suck a bit and they waste space and often functionality....
C bindings and APIs generally work much better in Rust because the language works a lot more like C than it does C++.
Qt depends a lot on C++ class inheritance, and even does some preprocessing of C++ files to generate code in those classes. Thatās obviously not possible when using Rust. And it looks like you need a fair bit of unsafe there and there to use it at all too.
Meanwhile, GTK being a C library, its integration with Rust is much more transparent and nice.
So if youāre making a GUI Rust app, youāre just kind of better off with GTK at the moment. Itās significantly easier and nicer.
Iāve been daily driving Linux for 17 months now (currently on Linux Mint). I have got very comfortable with basic commands and many just works distros (such as Linux Mint, or Pop!_OS) with apt as the package manager. Iāve tried Debian as a distro to try to challenge myself, but have always ran into issues. On my PC, I could...
Arch is actually not as bad as many say. Itās pretty stable nowadays, I even run Arch on some servers and I never had any issues.
Not even just nowadays. My desktop is running a nearly 10 year old install. Itās so old, it not only predates the installer, it predates the ātraditionalā way and used the old TUI installer. It even predates the sysvinit to systemd switch! The physical computer has been ship of thesisād twice.
Arch is surprisingly reliable. Itās not āstableā as in things change and you have to update some configs or even your own software. But itās been so reliable I never even felt the need to go look elsewhere. It just works.
Even my Arch servers have been considerably more reliable and maintenance-free than the thousands I manage at work with lolbuntu on them. Arch does so little on its own, thereās less to go wrong. Meanwhile the work boxes canāt even update GRUB noninteractively, every now and then we have a grub update that pops a debconf screen and hangs unattended-upgrades until manually fixed and hoses up apt as a whole.
For me what planted the Linux seed is when I tried Mandrake Linux when I was 9-10ish. I didnāt end up sticking with it for all that long, but I absolutely loved trying out all those DEs. I had downloaded the full fat 5 CD version and checked almost everything during setup, so it came jam packed with all sorts of random software to try out. The games were nice, played the shit out of Frozen Bubble. I really liked Konqueror too, coming from Internet Explorer. It was pretty snappy overall. And thereās virtual desktops for more space! People were really helpful on IRC, even though I was asking about installing my Windows drivers in Wine. Unfortunately I kinda wanted games and my friends were getting annoyed we couldnāt play games on my computer.
It stuck with me however, so later on when some of my online friends were trying it out, I wanted to try it out again too. I wasnāt much into games anymore, had started coding a little bit. So on my computer went Kubuntu 7.10, and Iām still on Linux to this day.
But that seed is what taught me thereās more. I didnāt hate Windows, I wasnāt looking to replace it. I hadnāt fallen in love with FOSS yet. It was cool and different and fun. It wasnāt as sterile and asā¦ grey as Windows 98. You could pop up some googly eyes that followed your mouse, because you could. There were all those weird DEs with all sorts of bars and features.
The only advice I have is to try to make it interesting for them and not just additional practical information they have to memorize. You donāt want to be the weird dad that insists on using stuff nobody else does, you have to show them whatās cool about it, and also accept maybe theyāll just stick with Windows for now.
I also think the main takeaway they should have out of it is that thereās many ways of doing the same thing and none is āthe correct and only wayā. They should learn to think critically, navigate unfamiliar user interfaces, learn some more general concepts and connect the dots on how things work, and that computers are logical machines, they donāt just do random things because theyāre weird. Teach them the value of being able to dig into how it works even if it doesnāt necessarily benefit them immediately.
Maybe set up a computer or VM with all sorts of WMs and DEs with the express permission to wreck it if they want, or a VM they can set up (even better if they learn they can make their own VMs as well!). Probably have some games on there as well. Maybe tour some old operating systems for the historical context of how we got where we are today. Show them how you can make the computers do things via a terminal and it does the same thing as in the GUI. Show different GUIs, different file managers, different text/document editors, maybe different DEās, maybe even tiling vs floating. What is a file, how are ways you can organize them, how you can move them around, how some programs can open other programās files.
Teach them the computer works for them not the other way around. They can make the computer do literally anything they want if they wish so. But itās okay to use other peopleās stuff too.
Basically the idea is that if you have a lot of data, HDDs have much bigger capacities for the price, whereas large SSDs can be expensive. SSDs have gotten cheap, but you can get used enterprise drives on eBay with huge capacities for incredibly cheap. Thereās 12TB HDDs for like $100. 12TB of SSDs would run you several hundreds.
You can slap bcache on a 512GB NVMe backed by a 8TB HDD, and you get 8TB worth of storage, 512GB of which will be cached on the NVMe and thus really fast. But from the userās perspective, itās just one big 8TB drive. You donāt have to think about what is where, you just use it. You donāt have to be like, Iām going to use this VM so Iāll move it to the SSD and back to the HDD when done. The first time might be super slow but subsequent use will be very fast. It also caches writes too, so you can write up to 512GB really fast in this example and itāll slowly get flushed to the HDD in the background. But from your perspective, as soon as itās written to the SSD, the data is effectively commited to disk. If the application calls fsync to ensure data is written to disk, itāll complete once itās fully written to the SSD. You get NVMe read/write speeds and the space of an HDD.
So one big disk for your Steam library and whatever you play might be slow on the first load but then as you play the game files gets promoted to the NVMe cache and perform mostly at NVMe speeds, and your loading screens are much shorter.
The Canonical-developed Netplan has served for Linux network configuration on Ubuntu Server and Cloud versions for years. With the recent Ubuntu 23.10 release, Netplan is now being used by default on the desktop. Canonical is committing to fully leveraging Netplan for network configuration with the upcoming Ubuntu 24.04 LTS...
Itās been slow for quite a while for me too, and it canāt even be the ads because I have Premium.
Itās usually fine when itās loaded but it does take quite a while to load for some reason, and Iāve got gigabit fiber and 16 cores to process it.
I heard YouTube falls back to a very slow path on Firefox because it uses features that Chrome implemented and never made it to the standard and something else was adopted instead.
Itās mostly better, but not in every way. It has a lot of useful features, at a performance cost sometimes. A cost that historically wasnāt a problem with spinning hard drives and relatively slow SATA SSDs but will show up more on really fast NVMes.
The snapshots, it has to keep track of whatās been modified. Depending on the block size, an update of just a couple bytes can end up as a few 4k write because itās Copy-on-Write and it has to update a journal and it has to update the block list of the file. But at the same time, copying a 50GB file is instantaneous on btrfs because of the same CoW feature. Most people find the snapshots more useful than eeking out every last bit of performance out of your drive.
Even ZFS, often considered to be the gold standard of filesystems, is actually kinda slow. But its purpose isnāt to be the fastest, its purpose is throwing an array of 200 drives at it and trusting it to protect you even against some media degradation and random bit flips in your storage with regular scrubs.
Precisely. Itās not just āit worksā, itās third-party hardware that Canonical tests, certifies and commits to support as fully compatible. Theyāll do the work to make sure everything works perfectly, not just when upstream gets around to it. Theyāll patch whatever is necessary to make it work. The use case is āwe bought 500 laptops from Dell and weāre getting a support contract from Canonical that Ubuntu will run flawlessly on it for the next 5 years minimumā.
Otherwise, most Linux OEMs just focus on first party support for their own hardware. They all support at least one distro where they ensure their hardware runs. Some may or may not also have enterprise support where they commit to supporting the hardware for X years, but for an end user, it just doesnāt matter. As a user, if an update breaks your WiFi, you revert and itās okay. If you have 500 laptops and an update breaks WiFi, you want someone to be responsible for fixing it and producing a Root Cause Analysis to justify the downtime, lost business and whatnot.
Lemmy has many nearly abandoned instances. Over the entire period of its existence - several posts. Shouldnāt the instance owner post content to attract users?
My instance exists for me and my friends to use. Itās not meant to attract anybody, itās meant to serve me.
It costs me nothing and Iām permanently in control of my data, and itāll live however long I want it to live, it updates when I decide I want to update it, if I want features I can just patch them in. When I make a PR, it goes on my instance first to try it out properly. I can post 10GB files from my instance if I want to, Iām the one that will pay for the bandwidth in the end.
I bet if you look at the profile of the admin of those āabandonedā instances, youāll find theyāre active on Lemmy. They just have their own private instance just for themselves.
Doesnāt matter if lemmy.world or lemmy.ml or beehaw.org goes down: I still got all the content and theyāll eventually federate out when they come back up.
Lemmy instance admin snooping at votes (monero.town)
So I was going through /all and this admin is snooping at vote counts for posts in his instance and then posting it publicly....
Linux file transfer speed bottlenecks?
Iām currently watching the progress of a 4tB rsync file transfer, and iām curious why the speeds are less than the theoretical read/write maximum speeds of the drives involved with the transfer. I know thereās a lot that can effect transfer speeds, so I guess iām not asking why my transfer itself isnāt going faster....
Lemmy instance which has not defederated with any other instance.
Hi everyone. I have found many ghost comments in posts. Like one of the posts has 300+ upvotes and 28 comments but when I opened it, there were no comments. I tried different Lemmy apps and itās the same in all of them. Which leads me to believe that it has something to do with defederation done by Lemmy.ml. Which instance has...
The Boost android client for Lemmy is displaying these dark pattern ads pretending to be system notifications. What security/privacy conscious Lemmy clients do you recommend? (lemmy.ml)
What's your current favorite distro that isn't Arch, Debian or Fedora?
Iām wondering what the current favorite distros are besides the most popular ones like Arch, Debian and Fedora.
Can't relate to be honest, I still use MBR boot (sh.itjust.works)
Which distro in your opinion is the best for virtualization (Windows 10 on either KVM or VMware), stability, and speed?
Flatpack, appimage, snaps..
Appimage for me ticks all the boxes for cross distro package as its very portable, simple to run, what are devs trying to do when creating snaps and flatpack?
As a normal, boring user that does nothing special other than browse the internet and the occasional "casual coding" -- what am I supposed to do with 32GiB of ram?
Title. Besides setting tmpfs to use 10GiB of it to store downloads.
Fedora 40 Will Enable Systemd Service Security Hardening (fedoraproject.org)
Summary...
Alright, I'm gonna "take one for the team" -- what is with the "downvote-happy" users lately?
Title. ālmao internet pointsā and all, but what is the point of participating in a community that sees assumptions and other commonly non-harmful commentaries/posts as ābadā this easily? Do folks in here are really that needy of self-validation, even if it means seeking such from something completely insignificant like...
Why are there so many (rust) GTK apps and so little Qt ones?
I always see new GTK apps popup on Flathub. I dont really care and think GTK looks fancy, although CSD suck a bit and they waste space and often functionality....
Wanting to improve my Linux skills after 17 months of daily driving Linux
Iāve been daily driving Linux for 17 months now (currently on Linux Mint). I have got very comfortable with basic commands and many just works distros (such as Linux Mint, or Pop!_OS) with apt as the package manager. Iāve tried Debian as a distro to try to challenge myself, but have always ran into issues. On my PC, I could...
Any experience with teaching kids Linux?
Any one here has any experience with teaching 8 to 12 years old kids Linux?
Bcache is amazing!: Making HDD way faster!
Okay, I love Linux. But I always surprised that I can love Linux ever more everyday. The possibility is basically endless!...
Ubuntu 24.04 LTS Committing Fully To Netplan For Network Configuration (www.phoronix.com)
The Canonical-developed Netplan has served for Linux network configuration on Ubuntu Server and Cloud versions for years. With the recent Ubuntu 23.10 release, Netplan is now being used by default on the desktop. Canonical is committing to fully leveraging Netplan for network configuration with the upcoming Ubuntu 24.04 LTS...
deleted_by_author
Why btrfs gets huge perf hit with background IO work? (lemmy.world)
Why aren't linux hardware shops on Ubuntu's certified hardware list? (ubuntu.com)
Where are Purism, System76, Tuxedo Computers, Starlabs, SlimbookES, and others? Instead thereās Dell, HP, ASUS, and Fujitsuā¦
Why create an instance if you are not ready to post in it?
Lemmy has many nearly abandoned instances. Over the entire period of its existence - several posts. Shouldnāt the instance owner post content to attract users?
Spotify spotted prepping a $19.99/mo 'Superpremium' service with lossless audio, AI playlists and more | TechCrunch (techcrunch.com)