See my reply to a sibling post. Nextcloud can do a great many things, are your dozen other containers really comparable? Would throwing in another “heavy” container like Gitlab not also result in the same outcome?
that endlessly duplicating services across containers causes no overhead: you probably already have a SQL server, a Redis server, a PHP daemon, a Web server, … but a docker image doesn’t know, and indeed, doesn’t care about redundancy and wasting storage and memory
that the sum of those individual components work as well and as efficiently as a single (highly-optimized) pooled instance: every service/database in its own container duplicates tight event loops, socket communications, JITs, caches, … instead of pooling it and optimizing globally for the whole server, wasting threads, causing CPU cache misses, missing optimization paths, and increasing CPU load in the process
that those images are configured according to your actual end-users needs, and not to some packager’s conception of a “typical user”: do you do mailing? A/V calling? collaborative document editing? … Your container probably includes (and runs) those things, and more, whether you want it or not
that those images are properly tuned for your hardware, by somehow betting on the packager to know in advance (and for every deployment) about your usable memory, storage layout, available cores/threads, baseline load and service prioritization
And this is even before assuming that docker abstractions are free (which they are not)
Most containers don’t package DB servers, Precisely so you don’t have to run 10 different database servers. You can have one Postgres container or whatever. And if it’s a shitty container that DOES package the db, you can always make your own container.
that those images are configured according to your actual end-users needs, and not to some packager’s conception of a “typical user”: do you do mailing? A/V calling? collaborative document editing? … Your container probably includes (and runs) those things, and more, whether you want it or not
that those images are properly tuned for your hardware, by somehow betting on the packager to know in advance (and for every deployment) about your usable memory, storage layout, available cores/threads, baseline load and service prioritization
You can typically configure the software in a docker container just as much as you could if you installed it on your host OS… what are you on about? They’re not locked up little boxes. You can edit the config files, environment variables, whatever you want.
Most containers don’t package DB programs. Precisely so you don’t have to run 10 different database programs. You can have one Postgres container or whatever.
You can typically configure the software in a docker container just as much as you could if you installed it on your host OS…
True, but how large do you estimate the intersection of “users using docker by default because it’s convenient” and “users using docker and having the knowledge and putting the effort to fine-tune each and every container, optimizing/rebuilding/recomposing images as needed”?
I’m not saying it’s not feasible, I’m saying that nextcloud’s packaging can be quite tricky due to the breadth of its scope, and by the time you’ve given yourself fair chances for success, you’ve already thrown away most of the convenience docker brings.
Nothing to do with efficiency, more because the containers are come with all dependencies at exactly the right version, tested together, in an environment configured by the container creator. It provides reproducibility. As long as you have the Docker daemon running fine on the host OS, you shouldn’t have any issues running the container. (You’ll still have to configure some things, of course)
I host a Plex server for streaming and my apple TV 4k 2021 would refuse to play high bit rate media. Kept displaying an error message telling me I’ve exceeded the limit. Started searching online and everyone consistently called Nvidia shield pro the best one can buy. Bought it, love it, now have 3. But all I use it Plex. I’ve made my server basically all streaming services combined to one.
If I may ask, what are you using to host the Plex server? I’ve read about people using NAS devices (Synology, etc. which has Plex available natively) and running a PC with a lot of storage. Appreciate the comment!
I’m absolutely at that point with Nextcloud. I kind of didn’t want to go the syncthing route, but I’ll probably give it a shot anyway since none of the NC alternatives seem any better.
Any guidance on this? I looked into Synthing at one time to backup Android phones and got overwhelmed very quickly. I’d love to use it in a similar fashion to NextCloud for syncing between various computers too.
Well, it works in a different way than NextCloud. You don’t have a server, instead you just make a share between your computers and they are all peers.
It takes some getting used to the idea, but it’s actually much simpler than NextCloud.
@squidspinachfootball@marcos Syncthing syncs. It does one way syncs, but if your workflow is complex and depends on one way syncs that's probably not what you want.
Sync things between operational systems, then replicate to nonoperational systems, and backup to off site segregated systems.
I was very intimidated as well, I’ll try to simplify it, but as always check the documentation ;)
This is the process I used to sync between my Windows PC and Android phone to sync retroarch saves (works well, would recommend, Pokemon is awesome) I’ve never done it on a Linux, though i assume it’s not too different
I downloaded the Synctrazor program so that it would run in the tray, again I’m not sure what the equivalent/if this would be necessary on Linux.
No shade to the writers, but the documentation isn’t super noob friendly, as I figured out. I’d recommend trying to cut out all the fluff, and boil it down to bare essentials. Download the program (whichever one seems right for your device, there’s an app for Android) and follow the process for syncing stuff (I believe I used a video guide, but it’s not actually as complicated as it seems)
If you need specific help I’d be happy to answer questions, though I only understand a certain amount myself XD
It really wasn’t all that complicated for me. Install the client on two devices set a share up on one device go to the other device Hit add device put the share ID in. Go back to the first devices admin and say allow the share
i have been running the new owncloud (ocis) and, with some quirks and very basic functionality, it’s been running for 2+ years and survived multiple updates without major complications
I run it and mariaDB in docker and they run perfectly when left alone, but everything breaks horribly if I try to do an update. I recently figured out that you need to do updates for NC in steps, and docker (unRAID’s, specifically) defaults to jumping to the latest version. I think I figured out how to specify version now so fingers crossed I won’t destroy it the next time I do updates.
This is probably what I’m doing wrong. I’m using linuxserver’s docker which should be okay to auto update, but it just continuously degrades over time with updates until it becomes non-functional. Random login failures, logs failing to load, file thumbnails disappearing, the goddamn Collabora office docker that absolutely refuses to work for more than one week, etc.
I just nuke the NC docker and database and start from scratch every year or so.
You absolutely need to move from patch to patch and cannot just do a multiple version jump safely. You also need to validate the configs between versions, especially major release updates or you risk breaking. New features and optimizations happen and you also may need to change our update your reverse proxy configuration on update, or modify db table configuration (just puking this from memory as I’ve had to do it before). I don’t know that there’s automation for each one of those steps.
Because of that, I run nextcloud in a VM and install it from the binary package. I wrote a shell script that handles downloading, moving the files, updating permissions and copying the old config forward, symlinking and doing the upgrade. Then all I have to do is log in as administrator, check out the admin dashboard and make sure there aren’t new things I have to address in the status page. It’s a pain, but my nextcloud uses external db and redis and PHP caching so it’s not an easy out of the box setup. But it’s been solid for a long time once I adopted using this script.
There could probably be some additional refactoring here, but it works for my setup. I’m using default nginx paths, so they probably look different than other installs that use custom stuff like /var/www, etc.
Use it by putting it in a shell script, make it executable, then call it:
sudo scriptName.sh 28.0.1
Replace the version with whatever version you’re upgrading to. I would highly recommend never upgrading to a .0, always wait for at least a .1 patch. I left some sleeps in the when I was debugging a while back, those are safe to remove assuming it works in your setup. I also noticed some variables weren’t quoted, I’m not a bash programmer so there’s probably some consistency issues that could be addressed if someone is OCD.
I’ve been a proponent here for a few months on using postgres/redis every time someone shits on NC for performance. While I agree the database change itself isn’t a huge improvement, it pays for itself long term in larger volume installs when you and your organization/group get using it heavily. The redis connected on socket like the AIO mastercontainer sets up is where the real juice comes from, but only on an install that gets used so it caches properly. The first time you fire it up, it’s pretty slow but as it gets used, things are much better.
I consider the 'good enough' level to be, if I didn't pixel peep, I couldn't tell the difference. The visually lossless levels were the first crf levels where I couldn't tell a quality difference even when pixel peeping with imgsli. I also included VAMF results, which say that the quality loss levels are all the same at a pixel level.
I know that av1, x264, and x265 all have different ways of compressing video. Obviously, the whole point of this was to get a better idea of what that actually looked like. Everything on the visually lossless section is completely indistinguishable to my eyes, and everything on the good enough section has very minor bits of compression only noticed when i'm looking for it in a still image. This does not require the same codec to compare and contrast with.
Frankly, for anything other than real-time encoding, I don't actually consider encoding time to be a huge deal. None of my encodes were slower than 3fps on my 5800x3d, which is plenty for running on my media server as overnight job. For real-time encoding, I would just grab a Intel Arc card, and redo the whole thing since the bitrates will be different anyways.
Frankly, for anything other than real-time encoding, I don’t actually consider encoding time to be a huge deal. None of my encodes were slower than 3fps on my 5800x3d, which is plenty for running on my media server as overnight job. For real-time encoding, I would just grab a Intel Arc card, and redo the whole thing since the bitrates will be different anyways.
Encoding speed heavily depends on your preset. Veryslow will give you better compression than medium or fast, but at a heavy expense of encoding speed. You’re not gonna re-encode a movie overnight on slow preset. GPU encoding will also give you worse result than CPU encode so that’s something one would have to take into consideration. It’s not a big deal when you’re streaming, but if it’s for video files, I’d much prefer using the CPU.
I consider the ‘good enough’ level to be, if I didn’t pixel peep, I couldn’t tell the difference. The visually lossless levels were the first crf levels where I couldn’t tell a quality difference even when pixel peeping with imgsli. I also included VAMF results, which say that the quality loss levels are all the same at a pixel level.
I was mostly talking about how you organised your table by using CRF values as the rows. It implies that one should compare the results in each row, however that wouldn’t be a comparison that makes much sense. E.g. looking at row “24” one might think that av1 is less effective than h264/5 due to greater file size, but the video quality is vastly different. A more “informative” way to present the data might have been to organise each row by their vmaf score.
Hopefully I don’t come across as too cross or argumentative, just want to give some feedback on how to present the data in clearer way for people who aren’t familiar with how encoding works.
GPU encoding uses (relatively) simpler fixed function encoders that do it much faster than the CPU which uses its general purpose transistors to run an encoding algorithm. End result is GPU encoding is speedy at the cost of visual quality per bitrate; the file size is bigger for same visual quality as a CPU encode. Importantly for storing your videos - CPU encoding, while much slower, will get your file size smaller at the same visual quality threshold you desire, so you can save more videos per drive!
Very anecdotally, I saw a little speed improvement but not all that much. DB size increased a bit. I’ll be sticking with it for the time being because why not.
Can you explain what you mean by “visually lossless”? Is this a purely subjective classification, or is there a specific definition or benchmark you used?
Visually lossless means I couldn't tell an image difference even when pixel peeping with imgsli. Good enough means I couldn't tell a difference in video, but could occasionally see a compression artifact in imgsli.
Thanks. I didn’t realize syslog would help. Just configured it to send to my grafana/loki server. Not sure if it’s really helping, but seems like maybe it’s a bit faster. I’ve long since done everything listed here and more, but in the last couple months my nextcloud has seemed a bit sluggish for some reason.
It’s not both, it’s Akamai Connected Cloud. The Linode brand has been retired. The vestiges of the Linode brand that are still visible are merely due to Akamai’s sloppy and disordered integration effort. They’ll likely retain the Linode.com domain so as not to break existing API calls, but the switch away from Linode.com as the primary domain, and removal of the name Linode from technical documents and design elements is ongoing.
That makes sense, it does sound better to keep it within nixos! I’ve mostly been using nixos to bootstrap servers that run nomad+docker, so beyond the system-level config, I haven’t done a lot with additional software yet.
selfhosted
Newest
This magazine is from a federated server and may be incomplete. Browse more on the original instance.