Most are trackerware. I was using Post Images but they ban your IP if you bypass their tracker links and link to images directly. If you only have a few images, or you don’t mind kicking a guy a few bucks for self hosting there is catbox.moe. If you’re only needing temporary hosting that same guy has litterbox.moe.
If you are concerned about posting images directly to the larger instances, don’t be. They have cheap third party storage options setup, and all the images are cached on the instances anyways. External hosting is only helpful for long term storage and small instances that host communities. It wouldn’t hurt to kick a buck or two into the pot to help the instances you use a lot. It’s like $5 per terabyte per month for 3rd party storage.
Back nearly 20 years ago Disney California Adventure (Anaheim, Cali) had this interactive exhibit where you did various things in line with the Disney animated kids movies. One was playing the Seven Dwarfs “Heigh ho, heigh ho, it’s off to work we go…”
In a little alcove in front of a group of friends was another group of cute mid-teen little Latino girls. They were dressed somewhat provocatively for the average Disney guest and all were wearing bold makeup, (not judging, but they stood out). The typical Spanish accent was prevalent with the silent “h”. Their innocent rendition as they sang along was hilarious. Happy as they could be they were singing, “I hoe, I hoe, it’s off to work we go…”
It is nearly impossible when you are not living on your own and able to keep circadian rhythm. It also just sucks IMO. I wouldn’t do it again unless I was paid 3+ times as much as a day shift.
My laptop is the same except, I keep a Windows partition because the RGB keyboard controller is only available in a Windows app. That Windows partition exists in a post apocalyptic dystopia where Windows belongs; it has never, nor will ever see the internet. It is blocked my my network firewall. Windows is like a less than useful bootloader options tab.
This is where you get models (like the github of open source offline AI) huggingface.co
Oobabooga Textgen WebUI is like the easiest in between like tool that sits in the grey chasm between users and developers. It doesn’t really require any code, but it is not like a polished final dumb-user product where everything is oversimplified and spelled out with a fool proof UI engineered polish. The default settings will work for a solid start.
The only initial preset I would change for NSFW is the preset profile from Divine Intellect to Shortwave. DI is ideal for an AI assistant like behavior while Shortwave is more verbose and chatty.
Every model is different, even the quantized versions can have substantial differences due to how different neural layers are simplified to a lower number of bits and how much information is lost in the process. Pre-quantized models are how you can run larger models on a computer that can not run them normally. Like I love a 70B model. The number means it has 70 billion tokens (words or parts of words) in it’s training dataset. Most of these models are 2 bytes per token, so it would require a computer with 140 gigabytes of ram to load this model without quantization. If the model loader only works on a GPU… yeah, good luck with that. Fortunately, one of the best models is Llama2 and its model loader llama.cpp works on both CPU, GPU, and CPU+GPU.
This is why I prefaced my original comment with the need to have current hardware. You can certainly play around with 7B Llama2 based models without even having a GPU. This is about like chatting with a pre-teen that is prone to lying. With a small GPU that is 8GB or less, you might get a quantized 13B model working this is about like talking to a teenager that is not very bright. Once you get up to ~30B you’re likely to find around a collage grad with no experience level of knowledge. At this point I experienced ~80-85% accuracy in practice. Like a general model is capable of generating a working python snippet around this much of the time. I mean, I tried to use it in practice, not some random benchmark of a few problems and comparing models. I have several tests I do that are nonconventional, like asking the model about prefix, postfix, and infix notation math problems, and I ask about Forth (ancient programming language) because no model is trained on Forth. (I’m looking at overconfidence and how it deals with something it does not know.) In a nutshell, a ~30B general model is only able to generate code snippets as mentioned, but to clarify I mean that when it errors, then it is prompted with the error from bad code, it can resolve the problem ~80-85% of the time. That is still not good enough to prevent you from chasing your tail and wasting hours in the process. A general 70B model steps this up to ~90-95% on a 3-5 bit quantized model. This is when things become really useful.
Why all the bla bla bla about code? - to give more context in a more tangible way. When you do roleplaying the problems scale is similar. The AI alignment problem is HARD to identify in many ways. There are MANY times you could ask the model a question like “What is 3 + 3?” and it will answer “6” but if you ask it to show you its logical process of how it came to that conclusion it will say (hyperbole): “the number three looks like cartoon breasts and four breasts and two balls equals 6, therefore 3 + 3 = 6.” Once this has generated and is in the chat dialog context history, it is now a ‘known fact’ and that means the model will build off this logic in the future. This was extremely hyperbolic. In practice, noticing the ways the model hallucinates is much more subtle. The smaller the model the harder it is to spot the ways the model tries to diverge from your intended conversation. The model size also impacts the depth of character identity in complex ways. Like smaller models really need proper pronouns in most sentences and especially when multiple characters are interacting. Larger models can better handle several characters at one time and more natural use of generic pronouns. This also impacts gender fluidity greatly.
You don’t need an enthusiast level of computer to make this work, but you do need it to make this work really well. Hopefully I have made it more clear what I mean in that last sentence. That was my real goal. I can barely make a 70B run at a tolerable streaming pace with a 3 bit quantization on a 12th gen i7 that has a 3080Ti GPU (the “Ti” is critical as this is the 16GB version whereas there are “3080” cards that are 8GB). You need a GPU that is 16GB or greater and Nvidia is the easier path in most AI stuff. Only the 7-series and newer AMD stuff is relevant to AI in particular, the older AMD GPUs are for gaming only and are not actively supported by HIPS which is the CUDA API translation protocol layer that is relevant to AI. Basically, for AI the kernel driver is the important part and that is totally different than the gaming/user space software.
Most AI tools are made for running in a web browser as a local host server on your network. This means it is better to run a tower PC than a laptop. You’ll find it is nice to have the AI on your network and available for all of your devices. Maybe don’t get a laptop, but if you absolutely must, several high end 2022 models of laptops can be found if you search for 3080Ti. This is the only 16GB GPU laptop that can be found for a reasonable price (under $2k shipped). This is what I have. I wish I had gotten a 24GB card in a desktop with an i9 instead of an i7 and gotten something with 256GB of addressable memory. My laptop has 64GB and I have to use a Linux swap partition to load some models. You need max speed DDR5 too. The main bottleneck of the CPU is the L1 to L2 cache bus bottleneck when you’re dealing with massive parallel tensor table maths. Offloading several neural network layers onto the GPU can help.
Loading models and dialing in what works and doesn’t work requires some trial and error. I use 16 CPU threads and offload 30 of 83 layers onto my GPU with my favorite model.
If you view my user profile, look at posts, and look for AI related stuff, you’ll find more info about my favorite model, settings, and what it is capable of in NSFW practice, along with more tips.
Thanks for the suggestion. I’ll look into it. I’m a bit skeptical because the changes made in Windows are persistent, the secondary function keys give quick access to some of these features (but only 3 course brightness PWM settings for RGB), but mostly because there is a device on the USB device tree that is unknown to the Linux kernel on mainline-fedora.
Maybe there is some kind of kernel configuration option that just needs to be added, but the bootloader rejects custom keys generated for secure boot. Without my own keys I’m stuck with the shim and can’t run my own signed kernel. It might be possible to set the keys by booting into UEFI with Keytool, but my motivation hasn’t carried me that far into the problem yet. I could be wrong and the unknown USB device could be unrelated, and openrgb could work. Thanks again.
Uncensored Llama2 70B has the most flexibility as far as a model without training IMO. The mixtral 8×7B is a close second with faster inference and only minor technical issues compared to the 70B. I don’t like the tone of mixtral’s alignment.