Its like a less capable Emacs wannabe that is designed to frustrate you when nothing works in examples using proprietary crap in VS Code, then leverage your familiarity to get you to give in to using VS Code and its MASSIVE stalkerware stream of constant information across your network with no rhyme or reason for that traffic to exist.

Everything I run is behind a whitelist firewall on an external device largely for this reason, but also learning curiosity.

Tailor Swift

Ash comes from a busybox… sh…

deleted by creator

No work in micropython on uno? pls help. Need thesis in Rust do tomorgh. Removed French bloat rm -fr catz tut. Why

Issue: … filled& coated …

Pettifogger’s Pedantic Pull Request: … filled & coated …

Octal. Start expressing it in maga Octal with thoughts prayers and bullets for ones

BSD for the win

deleted by creator

deleted by creator

There is no such thing as justice in courts. Lawsuits are corporate weapons.

Like (first hand experience in California) if you do not have 100k in liquid assets available right now, and you get injured by someone else, you’re not going to get jack shit for some lawsuit. You can’t navigate it on your own because of all the bullshit, and you’ll need lots of “expert witnesses”. Here is a little secret, expert witnesses are all academic opinion mercenaries that cost around 6k-10k each on an open market. You’ll need a bunch of them to counter whatever the other side does. In the USA it is all a formality in court. Whoever buys the most mercenaries wins the game. The insurance company or firm on retainer gets a bulk discount on mercenaries. The Supreme Court is not the only extremely corrupt institution here. Right, wrong, it’s all irrelevant. We are all worthless serfs in neo feudalism.

The Pi in any form is a much larger system with a whole lot more clock cycles, larger architecture, and more peripherals like a full memory management unit, graphics hardware, etc.

On the flip side IIRC most ESP32’s are 210MHz and just dual core. It is microcontroller versus microprocessor, so probably 10× less power or more.

I’m no expert. Have you looked at the processors that are used and the RAM listed in the OpenWRT table? That will tell you the real details if you look it up. Then you can git clone OpenWRT, and use the gource utility to see what kind of recent dev activity has been happening in the source code.

I know, it’s a bunch of footwork. But really, you’re not buying brands and models. You’re buying one of a couple dozen processors that have had various peripherals added. The radios are just integrated PCI bus cards. A lot of options sold still come with 15+ year old processors.

The last time I looked (a few months ago) the Asus stuff seemed interesting for a router. However, for the price, maybe go this route: https://piped.video/watch?v=uAxe2pAUY50

I am only one of these. I need a drink. The old water into wine glass trick should work

Thanks. I went down this rabbit hole already with large FreeCAD assemblies where single threaded operations had an enormous amount of non voluntary context switching. It helped some in that instance. I don’t have a benchmark, but I reduced the NVCS by an order of magnitude using Tuna to set affinity and pinning. Editing a large assembly in the middle of the tree went from 5++ minutes to around 3-4 minutes.

I’m not claiming I know better, or that this is a bright idea. As mentioned, it’s an abstract curiosity. Playing with FreeCAD got me interested in the OS on a deeper level, and got me reading as much as I can about the CFS and others. The LLM toolchain I’m working on is to help me follow the CS curriculum posted freely online from UC Berkeley. Ultimately I plan to use a langchain database to help me follow along with the books and transcribed lectures. It is just a hobby interest.

Thanks for the insights!

Seems like avoiding context switching and all the overhead associated would make a big difference when pretty much everything in cache is critical data.

I’m more curious about running something like Bark TTS where the delay is not relevant, but it would be cool to have the instructional clarity of the Halo Master Chief voice read me technical documentation, or test the effects of training my own voice, tuned to how I hear it, reading me stuff I find challenging. If the software is only able to process around 10 seconds at a time, just script it and let it run. The old machine will just collect dust otherwise.

Anyways, what’s the best scheduler with affinity/isolation/pinning?

There may be other out of the box type solutions. This setup really isn’t bad. You can find info on places like YT that are step by step for Windows.

If you are at all interested in learning about software and how to get started using a command line, this would be a good place to start.

Oobabooga is well configured to make installation easy. It just involves a few commands that are unlikely to have catastrophic errors. All of the steps required are detailed in the README.md file. You don’t actually need to know or understand everything I described in the last message. I described why the model is named like x/y/z if you care to understand. This just explained details I learned by making lots of mistakes. The key here is that I linked to the model you need specifically and tried to explain how to choose the right file from the linked model. If you still don’t understand, feel free to ask. Most people here remember what it was like to learn.

Originally posted this to beehaw on another account:

Oobabooga is the main GUI used to interact with models.

https://github.com/oobabooga/text-generation-webui

FYI, you need to find checkpoint models. In the available chat models space, naming can be ambiguous for a few reasons I’m not going to ramble about here. The main source of models is Hugging Face. Start with this model (or get the censored version):

https://huggingface.co/TheBloke/llama2_7b_chat_uncensored-GGML

First, let’s break down the title.

• This is a model based in Meta’s Llama2.
• This is not “FOSS” in the GPL/MIT type of context. This model has a license that is quite broad in scope with the key point stipulating it can not be used commercially for apps that have more than 700 million users.
• Next, it was quantized by a popular user going by “The Bloke.” I have no idea who this is IRL but I imagine this is a pseudonym or corporate alias given how much content is uploaded by this account on HF.
• This model is based on a 7 Billion parameter dataset, and is fine tuned for chat applications.
• This is uncensored meaning it will respond to most inputs as best it can. It can get NSFW, or talk about almost anything. In practice there are still some minor biases that are likely just over arching morality inherent to the datasets used, or it might be coded somewhere obscure.
• Last part of the title is that this is a GGML model. This means it can run on CPU or GPU or a split between the two.

As for options on the landing page or “model card”

• you need to get one of the older style models that have “q(numb)” as the quantization type. Do not get the ones that say “qK” as these won’t work with the llama.cpp file you will get with Oobabooga.
• look at the guide at the bottom of the model card where it tells you how much ram you need for each quantization type. If you have a Nvidia GPU with the CUDA API, enabling GPU layers makes the model run faster, and with quite a bit less system memory from what is stated on the model card.

The 7B models are about like having a conversation with your average teenager. Asking technical questions yielded around 50% accuracy in my experience. A 13B model got around 80% accuracy. The 30B WizardLM is around 90-95%. I’m still working on trying to get a 70B running on my computer. A lot of the larger models require compiling tools from source. They won’t work directly with Oobabooga.

This is a general list that was shared recently (has google analytics though):

• https://llm.extractum.io/

PrivateGPT is on my list to try after someone posted about it weeks ago with this how to article (that has a view limit embedded before a pay wall)/github project repo:

• https://medium.com/generative-ai/how-to-setup-and-run-privategpt-a-step-by-step-guide-ab6a1544803e
• https://github.com/imartinez/privateGPT

What software do you want to run?

I’ve been doing a lot of research on this over the last 2 weeks. I have my machine in the mail, but have not tried anything myself on my own hardware.

For Stable Diffusion, 8GBV is usually considered absolute minimum to do very basic stuff only. 16GBV or more is the basic need for a decent workflow.

For AMD I have seen multiple sources saying to avoid it, but there are a few people that have working examples in the wild. Apparently, AMD only supports the 7k series of GPUs officially with ROCm/hips/AI stuff.

Officially with Stable Diffusion, only nvidia is supported.