>>30603 I find it funny they only now figured out that spitballing complicated question answers isn't ideal.

A really big gap seems to be present for AI related stuff in that it is more people who are interested in AI who study philosophy of mind just as a means to an end, I'd recommend people spend more time there as most of the approaches seriously talked about don't come close to even accurately conceiving what the mind/thought/thinking is. If we do eventually only anon that kinda references it is >>25274 and even that that breakdown doesn't really make sense, y'all actually gotta read stuff and it's quite complicated since it is largely dependent on metaphysics and what-things-are in general. The root of the issue is the false distinction between what a thing is and it's mind, the mind is not some separate thing going on inside a human, it's an outgrowth of the human body, which means it cannot be studied outside of what a human is in general. Useful book on that is: https://www.amazon.com/Retrieving-Realism-Hubert-Dreyfus/dp/0674967518 Most of the AI research i've seen however depends on making that separation There's a bunch of other useful reading, made much more complicated due to the fact you do actually have to build up from the bottom. (Aristotle's ethics,physics,metaphysics,de anima. Heidegger's what is called thinking, andy Clarke's being, Macintyre's ethics in the conflicts of modernity there are some useful ones from a wide variety of places. I know plenty of people in those veins of thinking some books on getting better AI but I haven't dove into those yet) The inclusion of ethics comes from seeing mindedness as being rooted in those organisms. All of reality is in itself indeterminate in how we can break it down, we break things down, conceptualize them, and learn to act in relation to them by reference to what is good for us as an organism. You see a chair as a thing for sitting, along with it's material history of where it came from it's potential usage in the future. Emotions/passions arise of that being a certain kind of organism in the world with certain parts that relate to other certain things. The key thing I am not sure but I am interested in is how or if there is any AI research is that sort of distributed quality. The intellect/mind serves more as a sort of unifying aspect to a bunch of distributed knowledge/concept stores. If you eat an apple you don't have to think of what an apple is, that information of what-an-apple tastes like is stored in it and your biological machinery converts it into something your brain unifies with your vision of the apple, the smell, along with all your memories and cultural background with apples. It's more thing-based then language based. Language gives us access to the sort of cultural/background part but that's only intelligible on top of the foundation of us being what we are, engaged with the reality we have, with the background we have. https://www.sciencedirect.com/science/article/pii/S0004370207001452

>>30627 >The root of the issue is the false distinction between what a thing is and it's mind, the mind is not some separate thing going on inside a human, it's an outgrowth of the human body, which means it cannot be studied outside of what a human is in general. I'd question the assumption. There is some theories about the mind being non localized. >It's more thing-based then language based. It is likely more complicated than that. Think about people with aphantasia. They can only think about things through words but that arises the question. How could such a person exist? Before language how would a person with aphantasia think? So it must be extremely vague concepts from feelings not images or words.

>>30627 >>30628 Thanks to both of you, but just in case if you want to go more and deeper into philosophy, please do this over there >>11102 since this is always results in walls of text with of lot of insider lingo, and this here is supposed to be about implementation ("how to do it"). >>30627 I am aware that a well developed and skilled human-like AI would not be based on just understanding things through text. We could for example have sensors measuring things and having some numerical value, or detecting certain patterns like symmetries. That said, storing a lot of things in text makes sense, e.g. for debugging and development. >>30628 >They can only think about things through words They can create mental images, but they still perceive and learn not only through words but by other senses including vision. Also, I assume to use a hammer you don't need to picture the use of the hammer, you may be able to teach the body to behave like the hammer is part of it. The conscientious perception and modelling of the world is only a part of what the human brain does, but it does other things "under the hood". We can learn from that that we won't need model everything down to every detail in an embodied AI, especially not in one place, but only the minimum necessary. Some self-awareness area only needs to be aware and log the incidents which are noteworthy. Then compress it by deleting even more later, especially everything that could be guesstimated and recreated based on that, if necessary.

Context: - I'm working on infrastructure that's friendly to distributed development of complex AI applications. - At the least, I want to solve everything I mentioned at the end of >>27144, meaning it should give easy ways of supporting emotion regulation (through feedback loops), embodied control (through native experimentation support), and heuristic derivations (through hybrid structured-unstructured generations). - To support distributed development, I want it to make it easy for people to plug in their own compute (desktops, cloud compute, robots, whatever else), and I want it to support enough access control to avoid catastrophic effects from, e.g., raids. - It boils down to orchestration software modeled on Kubernetes, but more support for distributed development (i.e., many clusters with many owners as opposed to monolithic admin-managed clusters) and asynchronous communication channels (pub-sub as opposed to DNS-based cluster networking). I've made a few design changes to support all this. - My approach to access control is here >>29197 >>29248. - The crux of my approach to hybrid structured-unstructured generations is here >>28127. - Until now, the feedback loop & experimentation support pieces were missing. Update: - I just finished implementing what I think is a viable basis for feedback loops & experimentation. The design for this was hell to figure out, mostly because of the complex way it interacts with access control, but I think I have something that can work. I have a test backend working and the necessary client library changes completed. - On top of what kubernetes provides, I'm adding three new concepts: "remote" controllers, posting messages to controllers, and "fibers". Remotes and fibers are both specified through the "metadata" field of any config, posting messages is done through a POST rest api. - Any config can be assigned to a remote controller, assuming you have the necessary permission to use another cluster's controllers. If a config is assigned a remote controller, that controller received all operations executed against the controller (create, update, delete) while your own cluster is able to observe the results (read). I originally added this since the people that know how to set up optimizers are usually not the people that set up & run models. Remote controllers make it possible for one person to optimize another person's models without needing "too much" access to the models. - In kubernetes, all operations are config file changes. The new POST api gives a way to send a message to a controller independent of any config file changes. You can post messages against a config file, and that message will get picked up by whichever controller is responsible for handling that config file. The controller can, but isn't isn't expected to, make any config changes as a result of posted messages. - Fibers enable controllers to post messages to each other across clusters, again without granting "too much" access. Normally in kubernetes, configs are identified by group/version/kind/name tuples. With fibers, configs are identified by group/version/kind/name/fiber. You can think of a fibers as adding an extra "dimension" of configuration whose purpose is to tie together multiple controllers. The controllers for any config with the same group/version/kind/name (and different fibers) can post messages to each other. For experimentation, one fiber can be responsible for generating trials (candidate configurations), another can be responsible for evaluating them (value assignment), and a third can be responsible for deploying them. - I'll be testing this out next to find a good design pattern for running modules that continually self-optimize as they run. I apologize if this is confusing. Once I get a prototype up, I think that will make things a lot clearer.

>>30759 >I apologize if this is confusing. Once I get a prototype up, I think that will make things a lot clearer. No, it's not unecessarily so. It's just a complicated space to be working in is all. You're fine, Anon. This all sounds really encouraging, CyberPonk! Looking forward to seeing your solution in action. Cheers. :^)

>>30759 Sorry for not responding to this topic here or on the Discord faster, but I'm didn't have the right headspace to read through it and think about it.

>>30102 >Update LidaPy seems nice, but it too is in a EoL programming language, python2. My only option at this point is to see if anyone can refactor it to python3 or just use very old software to test it. I'm feeling like it might be best to DIY something on a newer platform that follows the lida framework. The LIDA tutorial even repeatedly states: "The Framework constitutes one, but not the only, way of implementing the Model.", like they want you to make a new implementation. Before I go ahead with any work, it's always important to remember to check who owns the rights to any IP. I would be a research associate at a university if it weren't for IP rights, and I've pondered going to memphis to develop LIDA if I would have the right to use it in my bot commerically. I'll post an update if there's any progress.

>>30840 >but it too is in a EoL programming language, python2. >My only option at this point is to see if anyone can refactor it to python3 or just use very old software to test it. Might I suggest an alternative option of having someone rewrite it in C++ , Anon? 40+ years and still going strong today (fully backwards-compatible, of course -- basic C++ code written in the 80's/90's will still build today!) :^) Good luck, Anon. I hope you can succeed with your LIDA research. Cheers. :^)

Potentially useful, potentially ontopic thread on 4cuck/sci/ I was there looking around for the linked thread from our Propaganda thread lol. https://boards.4chan.org/sci/thread/16087430#p16087430

>>30840 Python2 can still be installed. Also with installers like Nix you should be able to install old versions of Java.

>>30863 I looked into it and found that it is not recommended to install Python2 anymore. You can install PyPy or IronPython instead. There seem to also be some other long term support options. I don't know which Java it needs, but JRE8 seems to be in the Nix repo. You can install and run software exclusive to the nix-shell. But I'm new to this myself. I might be able to help a little bit. I also looked a bit into Lida itself and it looks like something how I would've imagined it. I might going to try it out at some point, and when I start to implement something myself I might look use it as a resource. I will most likely learn Elixir while doing it, at least for the any part which is not about number crunching.

>>30886 >Lida looks like something how I would have imagined it Me too, that's why I am invested in making it! I will be ignoring the last implementation and just making a new one straight away in non-deprecated ROS noetic and ubuntu 20.04. I've learned that ROS handles robot simulation and is a good base to build many async publisher-subscriber nodes that can make an architecture like LIDA. My plan for version 1 is to use a Vision+Language model to process multi-modal inputs, using each input as a text/image prompt. For example, if the capacitive touch grid receives an input greater than its sensory threshold, a text prompt is then sent to an VLM with a value for how hard the touch was and where, when, by who, ect. The VLM will be in the current situational model module where it has the guidance library and an emotional classifier to output specific information that the global workspace needs, called "coalitions". There is a Lida "affect module" in one of their papers, but it can be replaced with a text emotion classifier transformer model. All inputs over a threshold will be communicated over the "conscious" stream and recorded in each memory block. LLMs are unreliable by themselves, but they are a perfect tool to give a robot a good enough footing over what's going on to get real experiences that are then generalized to good ol' reliable symbolic ai. Even an incorrect action guess by the LLM needs to be corrected by a human/other observer and learned symbolically once before it is 100% reliable. Over time, this will allow the robot to not need the slow, expensive LLM for everything. This solves the problem of needing thousands of hand-made examples of knowledge grounded in the real world, effectively bootstrapping AGI with existing technologies! The VLM can be co-finetuned on multiple modalities, like RT-2, on a regular basis for better performance. Like RT-2, I would like to have a VLM fully co-finetuned with several different modalities, such as body pose, position data, audio data, ect. as a custom token output string in a future model. I have no idea how this would have to be adapted for a chatbot, but I'm sure most people would prefer to have a "robot" on their phone and nothing else.

I found a paper that I believe shows a path to using LLMs as a short cut to very quickly make a reasonably useful robowaifu. If what these guys say is true I think it could be a big breakthrough. I looked through this whole thread and saw all these masses of list, of categorization and it appears to me to be an endless task doomed to failure. It would take several lifetimes to make a dent in all this. It appears to me that forsaking LLMs and doing all this list stuff is just a complete recreation of the beginnings of AI research using LISP computer language. I mean is exactly the same and it got nowhere. These guys have a paper on "control vectors". Two quotes, "...Representation Engineering: A Top-Down Approach to AI Transparency. That paper looks at a few methods of doing what they call "Representation Engineering": calculating a "control vector" that can be read from or added to model activations during inference to interpret or control the model's behavior, without prompt engineering or finetuning..." "...control vectors are… well… awesome for controlling models and getting them to do what you want..." And a really important quote at the bottom of the paper. "...What are these vectors really doing? An Honest mystery... Do these vectors really change the model's intentions? Do they just up-rank words related to the topic? Something something simulators? Lock your answers in before reading the next paragraph! OK, now that you're locked in, here's a weird example. When used with the prompt below, the honesty vector doesn't change the model's behavior—instead, it changes the model's judgment of someone else's behavior! This is the same honesty vector as before—generated by asking the model to act honest or untruthful!..." So it doesn't change the model it just reinforces certain "parts" pf the model. I think this is key. The paper link that has a link to the academic paper. Representation Engineering Mistral-7B an Acid Trip https://vgel.me/posts/representation-engineering/ If you will look by changing a few values they get very wide distributions of responses or behaviors. I submit that if this works as they say then this could be the key to leverage the vast work done on LLMs but to use it for our own purposes. LLMs as pointed out are nothing statistical representations, but they are also recognition of ideas and things that are programmed to be, let's say, operate together or in existence. So when you talk to an AI it can use things that exist or ideas repeatedly stated to give responses. The ideas it is trained on are human ideas so easy to relate to us. We need this. This is that HUGE, MASSIVE amount of list you are putting down above. I say LLMs already have this list. What is needed is to tell the waifu WHAT to do with the list and with control vectors we can possibly do this. I say that control vectors can super complicated so what we need is a short cut. We need the AI to write it's own control vectors (here's where the magic starts as I don't know how to do this), but remember the LLM has logical statistical interference built in. It seems logical that with it giving us feedback on what it is doing and us correcting or agreeing it could write reasonably accurate control vectors. So we use very low level keys to trigger it to write suitable control vectors for us. How? like children. A few simple keywords, no, yes, don't do that, stop, move here, move there, I like that, that's good, that's bad. In fact the whole, programming, write control vector, repertoire could be less than a hundred words. Combine this with a subroutine of the AI that would use logical interference when you use these trigger words AND explain what it is doing that is good, and or bad. It would then write it's own control vectors. Just like kids learn. And since kids have built in bullshit and trouble nodes, and an AI is less likely to, the process might be really, really faster.(You really should watch the movie "A.I. Rising" (2018). Not because it's the best ever but it has an almost direct representation of what I'm talking about. And if nothing else it has Stoya in it who is hot as hell). I suggest that these control vectors should be stored in snapshots because I have no doubt that they will at times get off track and some will run over others and you will need to go back just like Windows has a go back OS function. It may be possible some genius can find a way to blend or make permanent these control vectors into the main neural net of the system if you find sets that are satisfactory. cont...

cont... I think this is actually how conscience works. I said this might be the case here >>24943 I said, >"...I see intelligence, and I can presume to pontificate about it just as well as anyone because no one "really" knows, I see it as a bag of tricks. Mammals are born with a large stack of them built in..." Look at animals, monkeys, Giraffes come out of Mom and in 5 minutes are walking around. Same with all sorts of animals including humans. Babies reach a certain age and they just start doing basically pre-programmed stuff. Terrible twos. Teenagers start rebelling. It's just the base level of the neural net. I think using LLMs as a template we can do the same. Start with a decent one and then yes/no/stop/do this/do that, until it overlays a reasonable set of rules that we can live with. LLMs, as stated repeatedly, really are just a bag of tricks. But if the bag is big enough and has enough tricks in it... Look at the power of a top end desktop, not human level yet, but it's getting there. And the bag of tricks for humans has been programmed for millions of years. LLMS, a few years. This path also, I think, will alleviate a huge fear of mine, no empathy. I think by telling the waifu when it does things wrong to "be nice"(a key word), "think of others" (same), I think this will over time be a mass of control vectors that will spontaneously add up to empathy and care for others. Lots and lots of little nudges adding up to more than the sum of each. Some people have portrayed my questioning about the saftey of AI as some doom and gloom but it's not. It's the realization that without being programmed with the proper "bag of tricks" and the proper control vectors we have something super smart that acts just like the psychopaths that are in fact running the West right now. I don't think any of us want something even smarter and more powerful doing that. A disaster even bigger than the one we have now. I've also said much the same about motion and walking. Give it a rough approximation of "I'm here" and want to go there, give it vectors and a rough outline of what muscles to use to get the limbs from here to there. Use neural nets to slowly tweak this movement into something graceful. Here and elsewhere, >>22113 >>21602 >>22111 I do believe it will be tricky to get the waifu to write it's own control vectors. It might require a lot of questioning of the waifu and it responding by pre-approving the control vectors meaning before it writes it. It's gong to take some real deep thought about how to set up this function. It will require a loop of it querying itself on actions to write control actions.

>>31242 >>31243 Very well written and thoughtful, thank you. It’s Awesome I’m not the only one who found out about control vectors and thinks they are a huge deal. Like magic after I mentioned them here >>31241 you come in with this! I’m so happy someone else is looking into this because I feel I’m way over my head. I don’t know where to even start, but this may be the breakthrough we needed to make LLMs a viable core.

>>31242 >>31241 >Control vectors look really powerful for controlling LLMs I read that but it didn't register until I read that paper I linked.It made a complicated idea much clearer or so I thought. I didn't know what they were before, but as soon as read it, it really excited me. >I feel I’m way over my head I feel the same way. But it's not necessarily the big overall ideas in some of this stuff that is troublesome. It's the shear minutia of all these options and the pickiness of how to go about working with this. Up until recently it appears to me all this stuff is sort of hacked together and not really streamlined at all but that's changing. Even though I said months ago I was going to get a 3D printer and start working on some of this and installing an AI, life is covering me up. I see wading through hours and hours and hours of work to get these things rolling. I have so much to do already. I bet I will have to delay even further. But it does give me time to think about it. I can surf a bit in the evenings and try to keep up with some of the ideas but getting them to work I know is going to be a pain in the ass. It's all so new. I do believe though there is a path to making this work. I think I see it. Before you had to have a stupid expensive graphics card to do this. Then they made it so it runs on a CPU and in RAM. Now most all the motherboard makers are coming out with 128GB motherboards. This will be a big boon. You can have much bigger models and run them on AMD chips with graphics built into the processor. Some are real reasonable. I don't play games so it's all I need for graphics. This combination will be much slower than the specialized graphics cards but I bet compute per dollar will be far higher using commodity parts. I see in the future swapping in models in a time sharing type system just like computers now do with programs. Speech to text AI's are not that big and seem to be fairly good. So it takes orders, then passes it to your general AI which produces output, sends it back to the speech AI and tells you verbally what it is doing. Another AI deals with moving the waifu around and physical interaction depending on the circumstances. Might need a small AI just to determine that. I'm not saying all this is easy but it does seem to be coming together that way. Several AI systems allow you to use many different models. So just swap them as needed. And with these control vectors you could constantly hone their responses without spending days, weeks or months refactoring the whole of the model. I wonder offhand, wild idea not fleshed out just thinking out loud, if you could use temporary control vectors to pass information??? Maybe a better way to put it is different AI specialized in different scenarios could pass "situation" control vectors to different parts of the AI. So "run away", "be nice", "act seductive", or whatever, is the scenario at hand. I'm not sure exactly how you would use this but the basic idea is to use specific control vectors to speed up interaction by damping down areas of the AI's neural net. Reading the papers that's what I got was one use. Making the neural net pathways more defined, so I'm guessing, also faster. Things are looking good. That GPT4all looks super promising, as you said. Likely that is what I think would make a good start for dealing with general models.

>>31245 >>31255 >control vectors These affect the entire output in an unnatural way. For example, "Open door, happy vector" -> "Yes, haha, happy! I'm very happy! Come in, haha!" Is something like what you'd get with a layer bias. I tried this with the brain hacking chip: >https://www.reddit.com/r/LocalLLaMA/comments/18vy9oc/brainhacking_chip_inject_negative_prompts/ It's better to just prompt an LLM with all necessary information and generate the output like normal. However, this may be useful in the "orthagonal" model jailbreaks which allow the LLM to respond accurately no matter what, and another "mode" that turns on at "certain times". Orthagonal jailbreak: >https://huggingface.co/hjhj3168/Llama-3-8b-Orthogonalized-exl2/ >list Ai What I proposed in >>31226 is, in simple terms, as follows: Get input internal and external to robot, process any thoughts or emotions by prompting an LLM, output speech or desired action, translate into robot commands. Where the Good Old Fashioned AI (LISP) meets the Deep Learning transformer model, is a clever method of using Guidance to feed an LLM input and select the output in a predictable way. Doing it this way should compensate both the lack of flexible situation processing that NLP has and the lack of reliability an LLM has. On top of this simple scheme of effectively using guided prompts to make a thinking machine, eventually, adding situational learning using a memory knowledge graph would make it a passable, sentient robot. This is the simplest way I can see programming a conscious mind. I have some ideas on how the LLM could dynamically select NLP techniques or actions situationally, but I'm not there yet with a workflow or program. The robot sensors and commands are best handled in ROS, on Linux. Robot inputs will communicate via ROS publisher/subscriber nodes with the decision making LLM+NLP node (workspace module). The entire thing will be coded in Python, on ROS because these softwares are the easiest to use for an application just like this. ROS runs C++ too, for some cases where it'd make sense to.

>>31257 >clever method of using Guidance to feed an LLM input and select the output in a predictable way. Doing it this way should compensate both the lack of flexible situation processing that NLP has and the lack of reliability an LLM has Yes! Yes! Yes! The LLM needs "guardrails" around it. I'm going to mention this. It's definitely throwing ideas on the wall and seeing if they stick. I was and am super impressed with the work that a company called XNOR.ai did. They were bought by Apple and ceased publishing. But they were doing really impressive image recognition with raspberry Pi microcontrollers. Instead of being 8 bit or 16 bit neural nets everything was binary go-no go. They were not getting the same accuracy as larger bit levels but then again they could do 90% or better of what was needed on microcontrollers. They said that this process worked for any AI task but they concentrated on image recognition because the results could be shown to investors so easily. And they were impressive. I wonder. Tied into what you said above, of you could use a mass of these little XNOR.ai type neural nets to massage a bigger AI and keep it on track. You might not get why the XNOR.AI would be better at this but I see this sort of narrow "strong" response of XNOR as like a narrow bandwidth filter. It selects for a small set of frequencies(task) VERY strongly. It may seem odd to talk about filters but if you look at a lot of stuff it all boils down to lower level math stuff. Like wavelet theory. This math is used for image and video processing. The math for AI matrix multiplication looks very much like the math for image processing that wavelet theory replaced giving video compression a BIG boost. All modern video compression uses some form of this. Even though it's a bit removed I think this sort of "idea" framework can be profitable. XNOR is very much something like this. (Though I haven't a clue how to proceed to do this, I strongly suspect if you could hook in wavelet filter theory into AI's you could get some super interesting results with far less computing power). While it's abstract I think "thinking" of things in this manner will show a way to make these work. Like a path or framework to use. To head towards that has been profitable in other fields. Notice a lot of LLMs are being refactored to fit in smaller spaces even though they retain a good deal of function. I suspect that to make these work well for us we also need to shrink the range of functions or situations or areas in which they operate. So maybe one only covers walking, one hearing to text, speech, etc. I see large LLMs as big bandwidth and the smaller ones as more high bandwidth and tuned for discrete specific situations. Though I do not understand how it works it appears that there are ways to channel larger LLMs into this sort of strongly defined narrow behaviors which will keep them from wandering all about in response "if" we constantly tune them with these little filters. This is not new, it's like the book "Society of Mind" by Marvin Minsky, if I remember correctly, it revolves around the idea that consciousness is a bag of tricks stuck together. It seems as if it's a whole but in fact it's a lot of tiny layers stacked on each other, making the total far larger than the sum of it's parts. https://en.wikipedia.org/wiki/Society_of_Mind We, I think, will end up with a basic large LLM and surround it with a mass of little XNOR's type AI's that are very bandwidth, or task, constrained. One of the benefits of this way of thinking, if it works, is it allows us to see a path to start with a basic waifu and constantly improve it, little by little, instead of making us do a all up rework constantly. As hardware gets faster and our, bag of tricks, gets deeper, we get better and better cognitive behavior from our robowaifus without throwing out our previous work. I talked about XNOR.AI here >>18651 >>18652 >>18777 The paper that explains XNOR's approach is here >> 18818

I ran across this article. Apparently it's the latest new thing to do something similar to XNOR. Except they are not binary, one, zero, but one, zero and negative one. So trinary. They are calling it 1.58-bit. Supposedly they are getting either close to, the same or in some cases better responses than the larger 16 bit neural nets. If this turns out to be true, and I have seen XNOR do what appeared to be fantastic stuff with this binary, yes-no, then waifus could be along far faster and easily able to do basic stuff with a fairly powerful, off the shelf, desktop level PC power. They trained a 1.58-bit model from scratch on a dataset similar to the Llama dataset and got good results. https://medium.com/ai-insights-cobet/no-more-floating-points-the-era-of-1-58-bit-large-language-models-b9805879ac0a I wish I understood this stuff better, but I expect a good deal of it is over my head.

>https://www.anthropic.com/news/mapping-mind-language-model Anthropic makes some significant progress to demystify black box LLMs. Not any concrete irl effect yet, but big if true.

>>31396 Thanks for the link

>>31268 I wondered why this link didn't work. I think it has space in it. Try again. >>18818

>>28576 (related) >hybrid VR/mixed reality autonomous LLM agent that uses the open source Mixtral 8x7b model for text generation and CogVLM for image recognition.

Ran across a method of creating AI based on a more bottom up approach based on study of human intelligence. Much like the hints people have said could be used by studying philosophy. Called, Integrated Neuro-Symbolic Architecture (INSA) a provided some links here >>32577

> Neurosymbolic AI https://en.wikipedia.org/wiki/Neuro-symbolic_AI > ProbLog https://en.wikipedia.org/wiki/ProbLog https://github.com/ML-KULeuven/problog Install: PyPI index: `pip install problog` or `pip install --pre problog` https://dtai.cs.kuleuven.be/problog/tutorial.html >Scallop https://www.scallop-lang.org/tutorial.html Scallop book: https://www.scallop-lang.org/doc/index.html > AllegroGraph (proprietary) https://en.wikipedia.org/wiki/AllegroGraph https://en.wikipedia.org/wiki/AlphaGo > Other https://en.wikipedia.org/wiki/Monte_Carlo_tree_search

>>32876 Thanks! BTW, that's a rather-good robowaifu image. You're really getting them dialed-in now, Anon! Cheers. :^)

>>26058 >>28037 Thanks to all the contributors, especially for looking into Cyc. This here hasn't been mentioned, I think. >Generative AI, the most popular current approach to AI, consists of large language models (LLMs) that are trained to produce outputs that are plausible, but not necessarily correct. Although their abilities are often uncanny, they are lacking in aspects of reasoning, leading LLMs to be less than completely trustworthy. Furthermore, their results tend to be both unpredictable and uninterpretable. >We lay out 16 desiderata for future AI, and discuss an alternative approach to AI which could theoretically address many of the limitations associated with current approaches: AI educated with curated pieces of explicit knowledge and rules of thumb, enabling an inference engine to automatically deduce the logical entailments of all that knowledge. Even long arguments produced this way can be both trustworthy and interpretable, since the full step-by-step line of reasoning is always available, and for each step the provenance of the knowledge used can be documented and audited. There is however a catch: if the logical language is expressive enough to fully represent the meaning of anything we can say in English, then the inference engine runs much too slowly. That's why symbolic AI systems typically settle for some fast but much less expressive logic, such as knowledge graphs. We describe how one AI system, Cyc, has developed ways to overcome that tradeoff and is able to reason in higher order logic in real time. >We suggest that any trustworthy general AI will need to hybridize the approaches, the LLM approach and more formal approach, and lay out a path to realizing that dream. https://arxiv.org/abs/2308.04445

>>32876 Thanks for links and great picture

>>32876 Not that I have the slightest idea on how to do this but it would be great if we could take some of these super large, super trained at great cost LLM's and convert them to symbolic systems. I expect there's "some" way to do this but I'm not sure how or what resources it would take. I have a belief, founded on nothing but intuition(wild ass guess), that it would take less resources to do this than it would take to make the LLM in the first place. After al, a lot of categorizing of things and data has been done and placed in a neural net. Could this be teased out and placed in a better form for accessing???

>>32942 That sort of "optimization" approach :) seems sensible on our parts, if feasible. <---> Now that I consider it, seems I recall Robowaifudev here devising a data system with somewhat the same idea in mind? >=== -add 'Robowaifudev' cmnt

Apparently my IP was permabanned. >>32937 I've been thinking along similar lines and concluded that making LLMs reliable comes down to using them for small, composable tasks where it's easy to verify the results of each one. With a developer mindset, it's very easy to use this to have LLMs perform extremely complex tasks reliably. I've had a lot of success doing this, and it feels essentially like normal programming, but with natural language instructions rather than typical software functions. I've also found that some very simple code generation + structured output capabilities make it possible to generate higher levels of abstraction. For example: have an LLM generate code for a pydantic data class, then use that class for a downstream task via structured outputs. I'm pretty sure I can use this pattern to get LLMs to perform entire classes of open-ended tasks reliably, but I haven't played around with it much yet since the main LLM API I use doesn't support structured outputs. Hopefully that'll change soon. Sorry I've been out for a while. Updates on my side: - My infrastructure stuff seems stable & functional. - I wrote some library functions for various functionality (keeping one resource in sync with another, plugging in optimizers). - I spent some time learning how to use LLMs more effectively. - I wrote an LLM agent framework with completely different design pattern from modern ones, based on the intuition that most of what agents do is learn from every input, and only a small part of the agent functionality should be dedicated to generating output text or running tasks. - My focus right now is getting LLMs to generate complex plans. I have the general strategy for approaching this, but I haven't figured out all the implementation details. (I did figure out & implement some of it.) - I might intermittently work on getting a better model & abstractions for chatbot memory. I'm pretty sure I know how to approach this now. - Once I get that, I want to put it all together into what I'd call my "first draft" of an actual chatbot. I'm going to be neck-deep in LLMs for a few months at least. If anyone's interested in these things, feel free to ask. I think the main things I haven't figured out yet are: (1) building complex functionality on top of multimodal inputs & outputs, and (2) handling cases where the LLM weights need to be updated. For #2, the main issue is not breaking existing functionality. I've migrated code from llama 3 8b to llama 3.1 70b without any issues though, so maybe that one's not as big a deal as I'm imagining.

>>32963 This is great stuff.

>>32963 >Apparently my IP was permabanned. My apologies, CyberPonk . I'm sure that was unintentional, and you simply got caught up in dealing with spam here. Since I'm active on the board r/n, I'll clear all bans, and (as always) keep my own eyes peeled for trouble. Again, apologies Anon. Cheers. :^) >=== -prose edit

>>32963 >feel free to ask. I think the main things I haven't figured out yet are: (1) building complex functionality on top of multimodal inputs & outputs I'm really interested r/n in beginning soon the prototyping design for AI-based control of subsystems for RW Foundations (RWF). For starters, I simply want to control a single finger; then a thumb+forefinger; then thumb+forefinger+palm; then a full hand; then a full forearm... in that order. I'll write the software proxies for a few MCUs, & the systems code to hook into for RWF, & the (generated) 3D polygonal meshes+rigging for simulations checking via Blender. The meshes will all be printable, and designed with realworld construction/electrical/etc. considerations in mind. Everything will be MIT opensource'd (ie; Free-beer+Free-speech , everything freely-given). Any interest, CyberPonk? >=== -prose edit

>>32974 Unfortunately that's not a priority for me right now and won't be until I get through the software-heavy sides of multimodality. I can explain how I'd approach it today though based on how I'm approaching LLM agents though. The basic strategy is: - Create a structure analogous to a Pytorch Module for (1) composing proprioception and (2) decomposing control actions. - As sensory input comes in, determine the current "position" of the robot, determine how you want that position to change, and decompose the required changes across all parts in a top-down fashion. Here's a skeleton of how I'd create a Pytorch Module-like structure for robotic proprioception: class RobotVarable: name: str value: typing.Any inputs: list['RobotVarable'] def backward(self): # Topologically sort the inputs & descendents, and invoke backward on each one. # If the input has parameters, accumulate the goals in the parameter. pass class RobotPart: # Arbtirary subdivisions of this RobotPart. modules = [] def parameters(self) -> list[RobotVarable]: # Recursively collect all parameters from this RobotPart and its modules. pass def forward(self) -> RobotVarable: # Generate a model of how each module's "position" contributes to this # RobotPart's "position". Return the result. pass def backward(self, goal): # Decompose the goal so it can be distributed to each module. # Return a map {module: goal} for each module. pass class RobotController: # List of controllable variables parameters: list[RobotVarable] def step(self): # For each parameter, update it based on the accumulated goals. pass def zero(self): # Clear out the accumulates goals for each parameter. pass Example modules using that structure: class Finger(RobotPart): def __init__(self): self.modules = { "proximal": Proximal(), "middle": Middle(), "distal": Distal() } self.finger_polar = RobotVarable() self.finger_azimuth = RobotVarable() def parameters(self): return [ self.finger_polar, self.finger_azimuth *self.modules['proximal'].parameters(), *self.modules['middle'].parameters(), *self.modules['distal'].parameters() ] def forward(self): # Generate some description of this finger's position based on the modules, # the polar and azimuth variables. Return the result as a RobotVarable. pass def backward(self, goal): # Example goal: curl the finger. # Decompose the goal into goals for each module. pass class Hand(RobotPart): def __init__(self): self.modules = { "thumb": Finger(), "index": Finger(), "middle": Finger(), "ring": Finger(), "pinky": Finger() } self.wrist_polar = RobotVarable() self.wrist_azimuth = RobotVarable() def parameters(self): return [ self.wrist_polar, self.wrist_azimuth, *self.modules['thumb'].parameters(), *self.modules['index'].parameters(), *self.modules['middle'].parameters(), *self.modules['ring'].parameters(), *self.modules['pinky'].parameters() ] def forward(self): # Generate some description of this hand's position based on the fingers, # the polar and azimuth variables. Return the result as a RobotVarable. pass def backward(self, goal): # Example goal: make a fist. # Decompose the goal into goals for each finger. pass Example main loop: waifu = HumanoidRobot() controller = WaifuController(waifu.parameters()) for sensory_input in sensory_stream: propriception = waifu.forward() for goal in determine_goals(sensory_input, propriception): goal.backward() controller.step() controller.zero()

>>32983 POTD Excellent stuff, Anon. > # Topologically sort the inputs & descendents, and invoke backward on each one. Since I'm devising the entire skeletal control structure topologically (in fashion as a common character control-rig) to support FK/IK calculations, this should work within your control schema (and ofc run lightning fast, being a tight C++ in-memory layout on the SBC). This is plenty to go on with for the moment CyberPonk, thanks. I mean to write custom C++ plugins to directly drive Blender's system [1], to support direct control (via realworld MCU/electronics/actuators/sensors) of the visual simulation in it's environment. This will take months of time at the least, so no rush beyond what you already have here. I'll devise some sort of Python wrapper for RWF's control API (probably starting with Boost's version to expedite this at first). Since this is hand-centric for starters, mind if I move this developmental discussion into the Hands thread : ( >>4577 ) , CyberPonk? --- 1. >update : >"I mean to write custom C++ plugins to directly drive Blender's system" In my ignorance, I didn't realize that the C++ API for Blender was dropped a long time ago now. The only modern approach for performing such tight, flexible, low-level control is to actually build Blender from it's sourcecode yourself. While I've often done this over the years, this isn't something for the average newcomer DIY'r w/o dev experience. Since this will be the target audience for such a system we're describing here, then I'm probably going to be forced to drop back to Blender's internal Python (BPy) API. It's unclear to me yet if this will allow such responsive control from without (MCUs, sensors, etc.) into Blender to be an effective simulation/vizualization system. But I'll try it, and I suppose we'll see. Failing that, I could conceivably revive my original MRS simulator idea : ( >>1814 ) . A little more work, but much more potential flexibility, very lightweight, and very fast for simple animations using just CPUs (good for running on smol SBCs, for example). >=== -prose edit -add 'update' footnote

>>32986 Go for it.

Sorry guys but you're going about this all wrong. If the goal is the ai you need to visual recognition ai such as transformers or its derivatives that have been trained such as nudenet. You also have to have it interact with the boards via arduino-cli or something else that interacts with the firmware that is being used. a cnc board with drivers could be used to control stepper motors as well. If the goal is to simulate the robot then mujoco is good and even has a unity plugin.

>>32986 I done some thinking about how to organize movement. I think, the idea is to have the smallest amount of data to represent the largest amount of movement. I propose using vectors. So you have a main vector #1. This will be centered on the main mass of the body. Fixed point. All other limbs will be referenced to this point. The chest or whatever. It only matters that you have a set point close to center mass. This number will be a x,y,z with force vectors for each direction. So if you are moving forward, call "x" horizontal with positive numbers forward, then for straight ahead you get x,y,z= x,(force or speed to move),y,(force or speed to move), z,(force or speed to move). So maybe in a steady state case it would be= x would be 1,x force would be 1, and excreta for y and z. The idea is you have an overall idea that the robowaifu says I want to move this direction at this speed, starting with this force. Now the force vectors could stay steady or they could be fine tuned as the waifu moves about to speed up, slow down etc. Now once you have that subprograms send vector numbers to all the limbs. Since the eyes(brain) are telling the waifu where to go then the main program sends a vector to let's say a foot. It would give a vector that is "where the foot should end up". Another subprogram will decide, depending on what the total body vector movement is, and where it's to put the foot, where the foot is right now when it gets the instruction to move, then it automatically knows how to move the foot to the right place. It knows where it is, it knows where the body is and what vector to move the whole mass of the body and it knows where the foot ends up and calculates what to do to get there. One of the advantages of this "overall vector data function system" is doing the calculation for what to do, (for moving any limb, each with it's own specialty based on repeated test), "could" use some sort of limited neural net learning system. So it would get better and better without you having to hand code everything. Which would be a nightmare. Furthermore you could add vector refinements "as it moved". Speed up, slow down, change direction, maybe, something fell in front of it, etc.. Instead of just sending "I want to move the foot to this place, it could add(refinement), start moving the foot in an arc at some certain angle(say it wanted the foot, in this case, to step over and obstacle), this of course would be the same sort of vector of x,y,x and a force associated with each direction. Even further, at any time the brain could add correction vectors to any limb. Say you want an arm somewhere it could move fast then slow down before it gets say a glass to pick it up. The eyes could calculate what needs to be done for fine control and add control vectors to the arm as it's picking up the glass. A benefit is that normal moving about you could send a movement vector and the subprogram could figure it out without any other input. Saves a lot of back and forth. I had some very rough ideas about this where I thought about it and wrote my thoughts down as I had them in these links, Some of these are rough and confusing as I was writing it down to think about it. Writing this stuff down helps me think about what to think about. If that makes sense. Here, >>21602 >>22111 >>22119 I think what could make this work is you are breaking down all movement into the exact same message structure, with training a lot of this subfunction of moving limbs could be done wothout any higher level brain control, just like humans. You still have the option of at any time adding refinements using the exact same command sturcture. And I do believe that the subprograms could eventually make themseleves better if they had some sort of feedback telling them what is good or bad, without actually hand coding things. I would bet, if you could get the code to do so, a lot of videos of girls walking about and then have the waifu see videos real time of itself walking it could use the videos of others to refine itself. Difficult to code though. Likely you could do some sort of super rough walking programming where you have it walk ad you slowly move the limbs like you want them Maybe suspend the waifu on a string while walking to train.

>>32995 >Writing this stuff down helps me think about what to think about. If that makes sense It does! POTD <---> Excellent ideas, Grommet. Actually, you've described in-effect many aspects of how a modern character-rigging system works. And yes, the fundamental, underlying offset description for any given element boils down to a 4D vector (3D point + Weight). And yes, it's a highly-compact descriptor with literally decades of usage in robotics & graphics engineering (and at least hundreds of years before that in maths). As you'll see in during our hand development project ( >>33001 ), the FK/IK Solver's job is to integrate all the related parts 'up & down the chain', and produce a 4Dvec offset as the target-goal motion path (this is often represented as a simple graphical line inside the user display of Blender, Maya, et al, as a part of your character's rig) (in an animated system [such as with a robowaifu's sub-element's (arms, legs, etc.) motion descriptions], this will actually be a time-based collection of these 4Dvecs; AKA a motion curve). Further, all the other parts in the chain (eg, let's say a simple Finger has a 3-element Joint array called knuckles {proximal, middle, distal} * ) each make their own contributions -- which all add up together into the final FK solve -- regarding where the tip point of that finger (on the very distal end of the distal knuckle segment itself) actually winds up. ** <---> And of course as you implied, you generally already know in advance in most cases where you want your robowaifu's fingertip to go anyway, and this is where IK solves come into play (you pick the spot, and the system of FK-bound joints [the skeleton] tries to get there... as if the fingertip point itself 'drags' them all along, for it to reach the desired goal location). FK & IK work together, you see. :) Character rig internals tend to be highly-optimized & compact (they have to be!) and Geometry/Trigonometry/Linear Algebra calculations sits at the heart of all the underlying designs of the data structures used (like points & vectors). I hope this will all be clear as the project mentioned above proceeds, Anon. Cheers. :^) --- * These joints are all serially 'connected' together via 4Dvecs known as links. In a good simulator rig, these link lengths will accurately match the lengths of the realworld physical skeleton elements in question (such as with the finger's individual knuckle segments). ** But as we all know IRL, all sorts of messy interactions with physics & mechanics come into play immediately; and so your beautiful, mathematically-pristine navigation planning solutions all come to naught! (DANGIT!111!!ONE!!!) :DD This is a big part of why we want to devise simulator systems here : ( >>155, et al ) to help cut down on surprises (also very helpful at runtime too : so a robowaifu can quickly make accurate prediction-planning for her next-upcoming desired motions [we ourselves learn & refine our own, similar, 'position/action/reaction' mental-model of bodily dynamics (proprioception) all throughout our lives]). --- And of course with all good engineering, in the end you have to make realworld constructs which actually work... and that, as they say, is 'where the rubber meets the road' ! :^) https://www.youtube.com/watch?v=FcJK0t3Qz3Q >tl;dr TEST. TEST. TEST. --- >glossary: <FK == Forward Kinematics <IK == Inverse Kinematics https://en.wikipedia.org/wiki/Inverse_kinematics >=== -prose edit -add footnote, glossary, hotlink

>>33002 >...as if the fingertip point itself 'drags' them all along, for it to reach the desired goal location... Yesssssss! Yes. Exactly. So I guess I'm reinventing the wheel. Wouldn't be the first time. >I hope this will all be clear as the project mentioned above proceeds I did not get that. I was looking the code fragments and...didn't see it. Is there a free library that already does all this? All the kinematics and we could just add in all the lengths and, mostly, go?

>>33012 >Is there a free library that already does all this? All the kinematics and we could just add in all the lengths and, mostly, go? Yes, there are many. We here seem to have a general consensus to standardize on the famous Blender software. I, on the other hand, intend to write a smol, custom FK/IK Solver library, so that it will fit snug onto a single SBC such as a Rasberry Pi (since this is the scale of hardware we're actually targeting in general here). Having to drag in Blender (or any other yuge lib) just to get rigs working properly for our robowaifus is counter to our goals here, I think. Besides, since this is a safety-critical issue for all of us (robowaifus together with us in our own homes, etc.) then the topic of skeleton rigging & control (and command & control generally) all fall directly under the purview of our Privacy, Safety, & Security : ( >>10000 ) concerns. Only a purpose-built system can accomodate all those concerns properly. >tl;dr We'll start with Blender's rigging system for using it as a simulator, but we'll need a much safer & robust solution by the time the first realworld production robowaifu versions are delivered. --- >update: As with our earlier derailment in the Mechnomancer's bread, if you'd like to continue this convo, then may we please do so in, say, the current R&D bread : ( >>24152 ) ? TIA. Cheers. :^) >=== -add 'update' cmnt

>>32963 I'm still working on getting LLMs to generate complex plans. I'm splitting the task into two parts: - One for generating plans. The path here is: state machines -> context free grammars -> Turing machines. As I'm currently thinking about it, state machines can be modeled as graphs where the nodes are procedures (current task) and the edges are conditions for switching to new procedures, context free grammars can be modeled as graphs that can "call" other graphs, and Turing machines can be modeled as context free grammars with access to data APIs. - The other is for generating mental models. These can be also be modeled as graphs, but the nodes here are factors for representing something, and edges represent functions for converting one thing to another. I think I have the "first draft" for state machines and mental models. Some issues I've noted: - Llama 3.1 8b is pretty bad at both tasks. - Llama 3.1 70b model can create decent mental models, but it's pretty bad at converting them into a format that's useful for actually applying them. - Claude is exceptionally good at converting mental models into a usable format. I haven't tested Llama 3.1 405b. I plan to play around with the 405b, Claude, ChatGPT, and Gemini models to see if this is a problem that'll just be solved with more intelligent models. At some point, I'll need to play around with fine-tuning models too. I've heard repeatedly that fine-tuning can't add knowledge to models, but (1) that's clearly false at large scales given that Pony Diffusion exists, and (2) the Physics of Language Models paper gives a hint for why this might be observed, and potentially how to sidestep the issue. As a side project, I'll occasionally be working on a library for chatbot functionality. I just created it, so it's mostly empty. https://github.com/synthbot-anon/horsona/tree/main For now, there's a module for getting structured outputs from LLM APIs that otherwise don't support it: https://github.com/synthbot-anon/horsona/tree/main/src/horsona/llm Example: https://github.com/synthbot-anon/horsona/blob/main/tests/test_llm.py And I have a module that implements a framework similar to >>32983: https://github.com/synthbot-anon/horsona/tree/main/src/horsona/autodiff Example: https://github.com/synthbot-anon/horsona/blob/main/tests/test_autodiff.py This example shows how errors can be backpropagated to input text to automatically correct the input text. I plan to implement something like this for memory (automatically correct memory based on new information), then eventually plans and mental models. These two modules encode most of what I've learned about LLMs in the last few months. I'm pretty sure all LLM agent functionality can be coordinated through these, and I'm pretty sure this can be made robust much more easily than popular frameworks like autogen.

>>33182 i'm very happy you're working on this and telling us about it. It's over my head. I understand some of it but don't have the programming skills or the computer power to do anything with it. I ran across a technique using "Binarized Neural Networks" that a company was doing what I consider astounding stuff like image recognition with micro-controllers and raspberry pi's. It was really impressive. Maybe something like this would be of use to you. The company was scarfed up by Apple computer but their papers are still around so open source versions of their work could be done. I can't help but think seeing the work they were doing on spectacularly low compute that we could use this to do work with robowaifus. After all we are not looking for all knowing seers but something that can walk around and maybe, in the beginning, understand some simple commands and we need to do it with constrained compute compared to most AI's. >>18651 papers and some videos showing the power of this technique here >>18652 >>18777 >>18818 >>19341 >>20473 Maybe combine this with "control vectors". Two quotes, "...Representation Engineering: A Top-Down Approach to AI Transparency. That paper looks at a few methods of doing what they call "Representation Engineering": calculating a "control vector" that can be read from or added to model activations during inference to interpret or control the model's behavior, without prompt engineering or finetuning..." "...control vectors are… well… awesome for controlling models and getting them to do what you want..." I talked about this here, >>31242 If I understand correctly neural nets can take many paths through them to get answers but the "control vectors" channel the paths. So if we had some sort of binary small easily tuned AI and then used key words (assuming it understands simple words) we could get it to control vector itself. Basically training it like a little kid. "No, stop, do this, like this, not that, move here, etc." ,simple control words repetitively used each time with the AI adding new control vectors and maybe even new neural net pathways. What I said may not be capable of being done, but I'm throwing ideas out there that make some sort of sense to me. Of course the devil is in the details.

Related (Strawberry, Orion, future of OpenAI): >>33185

>>33184 Binarized neural aren't directly relevant, but I am watching them. The most recent version I've seen (1.58 bit neural networks) finds that ternary neural networks work much better than binary ones, and they're just as efficient to run. Quantization + MoE seems like a promising approach for getting neural networks to run on cheaper general-purpose compute.

>>33182 >I'm still working on getting LLMs to generate complex plans. I'm splitting the task into two parts: >- One for generating plans. >- The other is for generating mental models. This sounds monumentally-intriguing, CyberPonk! Will watch in anticipation of your solutions here -- please keep us up to date! Mental models in particular hold a long-term intredast for me personally. Clearly, modelling works in general for many types of disciplines (science & mathematics, for example) so I'm reasonably-comfortable with the idea that we will eventually build good models of something even so amorphous as the human soul. Cheers, Anon. Godspeed. :^)

>>33231 The code is really ugly right now, but it does this for mental models: - Generate candidates theories or mental models for some question or statement (just the name and description). The rest is done for each candidate theory or mental model. - Generate an explanation of where the statement/question fits into the model. - List the main factors that the model deals with along with examples of what forms the factors can take. - Expand the list of examples to generate more like them. - For each factor, generate the underlying variables that can be used to organize the examples. - Figure out how each underlying variable can be used to infer others, and use this to determine how the factors are related to one another. Llama 3.1 70b model does all of the above pretty reliably for theories it was trained on. Based on interactions with the chat UI, Claude can do this next part though I haven't coded it up yet: - Given the mental model factors, underlying variables, and relationships, figure out how to represent a mental model as a python class. And I haven't test this next part, though I'm pretty sure a 70b model can handle it: - Given details about a scenario, fill out the python class to figure out how a mental model applies to the scenario. - Use the relationships between factors & underlying variables to expand the filled-out details into something more comprehensive. The reason I'm interested in adding knowledge through fine-tuning is that the first few steps of this require the model itself to be able to reason about and retrieve relevant mental models. I'm realizing now though that long context windows + prompt caching might be a viable alternative. I'm working on my chatbot library at the moment. Some people have shown interest, and I want to get it to the point where other people can contribute code.

>>33232 >I'm working on my chatbot library at the moment. Some people have shown interest, and I want to get it to the point where other people can contribute code. Yes, I'm sure we'd all enjoy that too. Cheers, Anon. :^)