>>38824 Thanks, OP! :^)

Breaking in the new thread with a new manual update -Updated list of models -Added Odysee backups -Added @Barf's prompt ------- Odysee Backups Social-Roleplay-Llama-Gemma PC AI Models https://ody.sh/M8f3VALm7S Social-Roleplay-Llama Smartphone AI Models https://ody.sh/imSLyPOkBh Qwen Coder AI Models https://ody.sh/PzErEQF9S9

An smol'r alternative to llamacpp. Includes a vision-encoder. Appears to be Chinese. https://github.com/li-plus/chatglm.cpp >>38831 Neat! GG, Anon. Cheers. :^)

>mfw I get blindsided epic style >>38853

>>38864 Looks good, but the prompt I posted with emotional tags is only for Orpheus and is just a stripped down version of the one Orpheus ships with which is this https://github.com/Lex-au/Orpheus-FastAPI/blob/main/System_Prompt.md I had a different prompt for F5-TTS with emotions but this is much better. Those are both good TTS engines that need a decent GPU with at least 8GB. For smaller, faster TTS in C++, sherpa onyx is a newer option than piper TTS https://github.com/k2-fsa/sherpa-onnx That sucks for Backyard since it was easy to install. SillyTavern\Open WebUI are a little harder but no coding is needed with Pinokio and they are all open source. SillyTavern is better for long format ERP and Open WebUI is more for productivity but has OpenAI speech API option which you can connect to a local Orpheus TTS server since it is a drop in replacement for the OpenAI speech endpoints.

>>38861 Fixed version Also, I added more backups

>>38888 Fixed version (again) -Fixed copyright notice error (Hopefully this is the last one lol)

anyone wanna test my new ai gf implemention? https://github.com/flamingrickpat/private-machine/tree/main you'll need to be able to run llamacpppython with cuda and enough vram for gemma-3-12b-it-q4_0.gguf, weaker models will most likely not work i rewrote the whole logic (pm_lida.py) since the last time i posted it here. its inspired by the lida cognitive architecture and simulates emotions, needs, goals, intentions, self image. still in active development, nowhere near done. and very slow

>>39454 Hello Anon, welcome (back)! Please look around the board while you're here. >its inspired by the lida cognitive architecture and simulates emotions, needs, goals, intentions, self image. That sounds remarkable! Certainly if you pull this off well, it will be a major breakthrough for AI companions, I think. I hope that someday you can work towards targeting much-smol'r hardware to run your waifu software on top of, Anon. That would be very helpful for us here. Please keep us here all abreast of your progress, Anon. Good luck with this project! Cheers. :^)

>>39466 Thanks :3 My long term goals are to finetune a smol model with the data generated by the agents. Recently I did some tests with loras, adapters and memory-layer adapters. I'm too smol brained and poor to come up with something groundbreaking and actually train it. I hope once the Google Titans architecture has some pre-trained models to make use of them. Or maybe even beyond LLM tech, who knows? Then I don't have to worry about memory agents, it learns and updates weights during inference. And the size / intelligence ratio will be better by then, so I could train it for cheap. Instead of the usual reasoning, it comes up with the whole unconscious thought process itself. Right now I'm mostly just testing it with my Emmy character. Depending on how it works, my next step might be to let her loose in an environment. Some virtual minecraft-esque world to move around and interact with the environment. Already browsed the robot vision thread and found some interesting projects.

>>39480 It's a very exciting-sounding project, Anon! :^) >And the size / intelligence ratio will be better by then, so I could train it for cheap. >Instead of the usual reasoning, it comes up with the whole unconscious thought process itself. I'd sure like to hear extensive breakdowns of these two statements when you have the time to, Anon. Cheers. :^)

>>39454 Much thanks!

>>39482 Sure, I'd love to elaborate. >And the size / intelligence ratio will be better by then, so I could train it for cheap. Basically, I'm waiting for some sort of breakthrough that makes inference and training much cheaper with same computational power. Maybe the diffusion LLM project, or RWKV, or something completely different. If a 1B model performs as well as the 12B model, I could train it overnight on my GPU. Use that to update the personality bias and the memory layer loras without lobotomizing the model. Or someone comes up with neural network architecture that really learns during inference, and isn't just fancy autocomplete like transformers. >Instead of the usual reasoning, it comes up with the whole unconscious thought process itself. This one is interesting. There is a guy who already does something like this. https://github.com/yukiarimo/yuna-ai He trained his own model to generate to generate different data for the situation. ><yuki>: User's dialogue ><yuna>: Companion's dialogue ><hito>: Other peoples' dialogue in the same conversation ><qt>: Internal thoughts and feelings ><action>: Function calls and actions ><data>: Embedded data or information Talked to him, nice but weird dude. This is like the reasoning of modern models, but for different aspects of world building, and the tags are additional special tokens. Usual models have system, assistant and user. In theory, a model could be trained to approximate the logic of all my agents. You start off with the dialogue, and the model dynamically generates the thoughts, goals, emotional impact of the new input on the fly. This means that I wouldn't have to make sooo many prompts with specialized agents, just to get one specific output (such as valence, anxiety delta as json). I decided not go any further with this, because I can't into math and even if I manage do make progress, some chinese dude will have it perfected by the time my code works. In the meantime, i'm getting into holographic waifus. Bought a Quest 3 and now I'm researching slam and segmentation.

>>39619 Check out our visual waifus thread! >>240

>>39619 I don't know how to do this but I read the Chinese, I think, were using computational farms if I remember correctly from Amazon or Google. It seems that they have cheap computing power. They need massive compute so they sell off the excess and they were training AI's starting from a known pre-built AI for like $200. That might be a way to cut cost. Start with a huge open source AI, then train on a waifu dataset for a few hundred dollars. Once trained it of course can be duplicated endlessly. A suggestion is to have it when turned on have standard "command training function". Very much like talking to little kids. Maybe an input password, can be verbal, like "name of bot", do this, don't do that, stop. Should be a natural verbal command structure to correct like a kid so you don't get confused. Very simple two-three year old human commands and each one of these would provide a "control vector" as talked about here, >>31242 >>24943 >>31268 >>33184 >>35865

Here's instructions on how to integrate Open WebUI, F5-TTS voice cloning and KDTalker avatar using Pinokio for easier install. It just adds a button to generate the video avatar after the response, so it is optional. https://github.com/Barfalamule/KDTalker-OpenWebUIAction There's a lot of functions for Open WebUI like websearch, home assistant, weather and memory. And you could pair with OBS\webcam for vision and maybe output the video to a screenface Takes under 5 seconds for short response on 3090 for the audio and 30 seconds for the video, and should be about double on a 5060 ti 16GB. Or you could rent a B200 by the hour and have it instant.

>>39802 It looks great, what you describe definitely is an upgrade to the normal offline local AIs I use. You should definitely do a video tutorial.

>>39803 Thanks. Will do at some point but have to censor everything for github

>>39802 >>39804 Hi Barf, thanks! >...but have to censor everything for github You're not beholden to Microsoft's GitHub, Anon. There are plenty of good alternatives. --- Regardless, thanks for keeping us all up to date on your progress! Cheers. :^)

Very useful article on offline LLMs https://www.xda-developers.com/things-wish-knew-started-self-host-llms/

>>39971 Thanks, GreerTech!

Big if true

>>40125 Found this article https://huggingface.co/blog/mlabonne/abliteration

Someone should look into autogpt unless you want the waifu to just respond to prompts. It’s low priority as far as what I’m working on though.

>>40127 I agree up to a point. A more proactive robowaifu is definitely a necessary goal (see >>39900), but I don't think AutoGPT is the answer, since that's for software environments.

>>40129 You can have a more proactive LLM by running image recognition hooked up to a webcam. Then you just inject what the image recognition sees into a prompt so you end up telling the LLM "you see: chair, bookshelf, [username]), probably filter it so it doesn't constantly feed the LLM sight data until something changes (maybe determined via frame subtraction and the object list or something).

>>40125 Read that as >"...a lobotomy where the lawyers responsible for refusals are deactivated..." <pretty excited there for a minute, tbh in fact of course. :D

>>40130 Oh no yes that’s definitely the way. Use ai for image recognition and hardcode the rest. There’s nvidia gr00t but I think that’s for teaching it movement based on videos I think and it’s to make it easier so you don’t have to do a lot of inverted kinematic and such. I say take every shortcut possible. But I you guys are going to say noooo do it the harder way.

>>40133 Well ChatGPT says it’s better overall. Can’t say.

I'm working on Project Hydra, where I research into more local AI apps. More redundancies!

This is huge. A possible solution for hallucinations. It doesn't work for ChatGPT where people expect an all-knowing superintelligence, but it will work for us, since the minimum we need is a competent gf. https://theconversation.com/why-openais-solution-to-ai-hallucinations-would-kill-chatgpt-tomorrow-265107

>>41631 "The AI could then be prompted, for instance: “Answer only if you are more than 75% confident, since mistakes are penalised 3 points while correct answers receive 1 point.” "The OpenAI researchers’ mathematical framework shows that under appropriate confidence thresholds, AI systems would naturally express uncertainty rather than guess. So this would lead to fewer hallucinations. The problem is what it would do to user experience." "Consider the implications if ChatGPT started saying “I don’t know” to even 30% of queries – a conservative estimate based on the paper’s analysis of factual uncertainty in training data. Users accustomed to receiving confident answers to virtually any question would likely abandon such systems rapidly."

>>41631 >>41633 Heh, seems that Copeman & Co. have painted themselves into a corner? :D J-just a few moar $trillions, bro!111 --- Personally, I'd be satisfied with just the "personality" of the ship's computer from Star Trek. :^)

>>41634 It is funny to see ClosedAI start sweating because they're a victim of their own hype, but I think the real kicker here is that we may have a framework to solve AI hallucinations in social AIs, and arguably make them more human.

>>41636 >but I think the real kicker here is that we may have a framework to solve AI hallucinations in social AIs, and arguably make them more human. Yes, that would be excellent. If we could make it work on an 8GB RPi (or equiv.) that would be truly exceptional! Please pardon my fun, and thanks for the information GreerTech. Cheers. :^)

>>41637 Lol you don't need to apologize. I'll definitely test that in the near future

I have an project idea, but I need some general advise. The goal is to have a local LLM running on a Linux server, which it could control via CLI. The user could talk to it and give it some tasks to do, but the main part would be that when left with nothing to do, it could decide it's own goals to pursue. (coding, writing a personal website, browsing the web...) One could say that the goal of the experiment is to achieve an AI that could operate mostly by itself and achieve something meaningful while doing so. This would give the waifu the ability to have her own life outside of user interaction, making her both more useful and more realistic. I am planning on writing some LLM wrapper, which would implement API for interacting with the system, as well as handling multiple levels of memory and personality, both controlled by the model itself. More complex features like MCP API support or connection to some external hardware are being considered as possible extensions, but the current plan is a simple, Linux-controlling, AI companion. My knowledge of the current local LLM landscape is severely limited, however, so I come here to ask for help. Are local LLMs there yet? What is the least demanding model that you would consider capable of operating a Linux system and not freaking out when left alone? How powerful hardware would I need to run it? What backend should I use? (ollama, llamacpp, something else) Anything else I should know/consider?

>>41860 Hi Anon, welcome! I wish I could help you, Anon. If some other Anons here don't step up, then I'd suggest you check out /g/ 's threads on these topics in general. In particular, focus on the Local Models General ( /LMG/ ) threads. Good luck with your project, Anon! Please keep us here informed as to your progress. Cheers. :^)

>>41860 I can offer some insights. - having powerful hardware will improve quality immensely, but you can get a lower form of this up now with smaller models. - It will have context limit challenges which will need to be overcome with smart uses of tokens and how you store memories, but you can have a functioning thing as you describe, but detailed memories will eat into this heavily. having different ranked memory tiers and clearing lower ones and a mix of other clever ideas can improve all of these issues, at this stage it's just a matter of having a "lower resolution" AI companion, you could probably later update your methods as tech advances, and greater hardware means greater resolution. - I would suggest using multiple models to parse and make your AI companion smarter and not stuck into the patterns and paths that each models holds close to and deviates towards. to keep it simple, load one model, think for a bit, switch models, think a bit, especially with current limitations I would let a large chunk of time when the companion is by itself computing, to go over it's past and it's token database and refine it and have discussions and debates with itself to self-improve and refine itself. - I would have these personally as modes which you switch from her active life and talking to you, and her maintenance/sleep mode where she improves her database. - I will use sillytavern just as the idea vessel here, as I am used to it but I have not done this work myself properly since I'm busy with life. . . . use something like the memory and world lore books to store memories and events, they can each have a comprehensive recording into essentially "quants" of the lore books, where your chat history is fully recorded as non-quanted, then various levels of condensing and sorting out less important info/tokens, then your companion will think for each task and memory which quant of each lore book is suitable keeping in mind limitations hardware and current state of LLMs. - basically it will have a large database which it will parse and read through similar to how you will have it load models one at a time to use then switching to another, it will load different sets of the full database at a time to parse it. - You do the usual things you would expect, writing the personalities, creating the framework of the database, then you just have it use auto-mode similar to how sillytavern has group chats with different personality card as the other character, which is sort of her maintenance/ recursive improvement mode, again, multiple different reasoning/COT models being swapped through will be ideal here specifically. - Then you just let that be during periods of free time, and you also have her alone/ continues her life mode, which will develop memories and her character and your relationship, and to keep it coherent and immersive, I would make it follow natural time and not try to have it just create events that happened while you were gone as a task itself. The maintenance mode will be crucial here to sort and make her life events both logical and coherent as well as cutting away waste tokens, building those lore books, etc. the model changing is crucial also as even with settings and repetition penalties models slowly converge into themselves without manual maintenance that can be automated with multiple models and the mode I discussed earlier. Most of my knowledge I have is more on how you can do it abstractly, I would just vibecode and let LLMs walk you through setting it up when you get stuck. You can do all of this with smaller models and everything, but the "resolution" of your companion will be smaller, and will take longer for maintenance, and in active conversation mode I would pick your favorite model(s) and just use one as you speak to her, all of the data she needs for the conversation will be prepped as her "mental state" before from maintenance and from lore books which will fetch relevant data as you speak. The specific models I am still a newfag on, no recommendations that are any better than on this forum or halfchans /g/. Ai and LLMs run well on linux though, best of luck.

The good news here is it's possible on local, and the bulk of the work will be in maintenance mode when you are not interacting with her, but for it to be good it will be very slow, but this will be when you are sleeping or away. When you are chatting, you will simply have to accept lower quality speech to you to make it run at a reasonable tokens/second generation, but pulled from the refined database which she created during her past maintenance phase where she runs a stronger model at an abysmally slow tokens per second/minute rate for precision.

>>42608 >>42609 POTD