/robowaifu/ - DIY Robot Wives

Advancing robotics to a point where anime catgrill meidos in tiny miniskirts are a reality.

Server and LynxChan upgrade done. Canary update and a general address soon. -r

Max message length: 6144

Drag files to upload or
click here to select them

Maximum 5 files / Maximum size: 20.00 MB


(used to delete files and postings)

It was enjoying each other's company as they worked side by side, brought together by a love that transcended the boundaries of their creator and creation.

Open file (8.45 MB 2000x2811 ClipboardImage.png)
Cognitivie Architecture : Discussion Kiwi 08/22/2023 (Tue) 05:03:37 No.24783
Chii Cogito Ergo Chii Chii thinks, therefore Chii is. Cognitive architecture is the study of the building blocks which lead to cognition. The structures from which thought emerges. Let's start with the three main aspects of mind; Sentience: Ability to experience sensations and feelings. Her sensors communicate states to her. She senses your hand holding hers and can react. Feelings, having emotions. Her hand being held bring her happiness. This builds on her capacity for subjective experience, related to qualia. Self-awareness: Capacity to differentiate the self from external actors and objects. When presented with a mirror, echo, or other self referential sensory input is recognized as the self. She sees herself in your eyes reflection and recognizes that is her, that she is being held by you. Sapience: Perception of knowledge. Linking concepts and meanings. Able to discern correlations congruent with having wisdom. She sees you collapse into your chair. She infers your state of exhaustion and brings you something to drink. These building blocks integrate and allow her to be. She doesn't just feel, she has qualia. She doesn't see her reflection, she sees herself reflected, she acknowledges her own existence. She doesn't just find relevant data, she works with concepts and integrates her feelings and personality when forming a response. Cognition, subjective thought reliant on a conscious separation of the self and external reality that integrates knowledge of the latter. A state beyond current AI, a true intellect. This thread is dedicated to all the steps on the long journey towards a waifu that truly thinks and feels. >=== -edit subject
Edited last time by Chobitsu on 09/17/2023 (Sun) 20:43:41.
>>28149 >>28163 Are you talking about Mobile ALOHA? (in that case, here are some links you may find useful) https://mobile-aloha.github.io/ https://tonyzhaozh.github.io/aloha/ https://github.com/MarkFzp/mobile-aloha https://github.com/MarkFzp/act-plus-plus I think this is a good demonstration of why the AI side matters & why I am not worried about the hardware. The better the software the simpler the hardware can be. I have talked about this before even on this thread >>24816 when I mentioned the PR1 Montage https://youtu.be/qBZPSTR96N4 notice how both are basically platforms with hands. The difference is that the PR1 was purely tele-operated while the Mobile ALOHA was driven by software with cheaper looking hardware. higher level thinking is massively under researched and is vital. for example I don't think we can build anything useful without a powerful ontology. >Anon asks for a cup of milk. <The waifu can not find any cups in the cupboard, The ontology is queried for what other places in a home usually have clean cups. So then she checks the drying rack and then the dishwasher while also making sure to also check that it’s done and clean. >There is a water leak, Anon asks to find something to put under it. <The ontology is queried for what types of things hold water & what would be the most appropriate item that is also available. I do not see a point in having an AI vs hardware faction war. I guarantee you a lot of the "AI waifu" software will also end up in your "robot waifu". Any way peteblank, I hope you found something I said of use. Unrelated I found this blog post while looking at stuff related to link grammars, It was an interesting read for me so I am sharing it here. https://blog.opencog.org/2021/06/14/wave-function-collapse-for-procedural-generation/
>>28148 >In any case, even if Cyc is missing a few minor things, it seems like a given at this point that the ontology of ontologies is tiny compared to the ontology of all things. That offers a way forward for a perfectly general RAG system. The data should be queried in two stages. >- Stage 1 should run a query using the ontology of ontologies. This query would find the ontologies relevant to the task at hand. >- Stage 2 would use the retrieved ontology to query for the actual relevant data. Just a quick gut reaction I have not thought about this too hard, but this 2 layered system with a "meta knowledge layer" and then a collection domain specific systems seems to be popping up now in all sorts of places where I am looking (be in an LLM or graph of graphs). Just recently I have looked at Mixtral of experts, Cyc, OpenCog & even GPT4 is rumored to be a collection of experts. So this looks to be a possibly vital aspect of crafting a useful system. Sorry for such a nothing post, this is more of an open thought then a contribution to the discussion.
>>28179 I've heard the same thing about GPT4, and with the performance and efficiency of Mixtral, I wouldn't doubt that MoE is a better approach to scaling up than just increasing parameter counts, at least when you have more much data than is Chinchilla-optimal. I hadn't thought of mixture models as adding indirection to underlying sub-models, but it makes sense. I've always thought of transformer blocks as doing some sort of lookup, and I think they were formulated with that intuition too with "queries" and "keys". I think this would extend the analogy to two-step retrieval: - The context goes through some query expansion (QKV). - The switch of the Switch Transformer / MoE selects an ontology (expert) based on the expanded query. - The ontology-specific data is retrieved (feedforward network). This is used to update the context.
>>28175 Thank you. Its not a AI vs hardware war btw. Its a virtual waifu vs robot waifu war. .
>>28203 >Its a virtual waifu vs robot waifu war. . EEHHNNN Wrong. We're not in any kind of war here (except against the Globohomo's plots & snares). We'd all appreciate it if you'd stop stirring sh*te attempting to drum such conflicts up, kthx. >tl;dr We'll all do just as we see fit here. Be man enough to accept that and focus on your own goals Peteblank.
>>28207 When you release your unity fu in the google play store im going to give it 1 star rating.
>>28212 Lol. I don't ever anticipate doing that, but fair enough. :^)
>>28203 >>28212 These poasts have absolutely nothing to do with this thread. If Chobitsu didn't show you favoritism, I'd delete them. Please, keep your posts related to the thread topic and stop trying to force others to conform to your demands. We are working on our own projects. Get that through your head, no one has to work on yours. Only posts related to cognition with respect to robotics are allowed in this thread.
>>28232 >If Chobitsu didn't show you favoritism, I'd delete them. Please feel free to do as you please in regards to such things Kiwi. You already have my favor, and I already trust you to manage such things (with no warnings/consultations). The >tl;dr is that I'm highly in favor of keeping threads on-topic. If you spot posts that aren't then simply delete them (since LynxChan dev decided to not allow us to move them directly ala '>merge', its just simpler that way) (presuming they bring no other intrinsic value to the board -- which warrants either keeping them in-place, or copy/pasta/moving them to another thread). I hope all that's clear. Cheers! :^) >=== -prose edit
Edited last time by Chobitsu on 01/10/2024 (Wed) 20:23:22.
Open file (577.29 KB 1321x1763 ClipboardImage.png)
Open file (112.98 KB 1183x624 ClipboardImage.png)
Re: ontologies. I really should be working on my access control implementation, but this problem is too interesting. I realized that Formal Concept Analysis (FCA) would be great for creating ontologies. FCA is about creating and analyzing hierarchies (lattices) of objects and properties given object-property relationships. This paper gives an efficient algorithm for creating lattices: https://sci-hub.se/https://doi.org/10.1007/978-3-540-73681-3_16. The image shows the algorithm. This would make it possible to create ontologies from just object-properties data. Of the two main kinds of relationships handled by Cyc (genls, is-a), this only covers genls, but I think is-a relationships will be comparatively easy to infer. (Every type should be associated with a "primary key", and anything that has a unique primary key is an instance of that type.) This also wouldn't deal with microtheories, but I think that will come from classifying relationships, not from mining them directly. I'm trying to find an algorithm that (1) supports incremental updates, (2) allows updating both objects and properties, and (3) would support LLM-based comparisons for merging objects and properties. I think I can figure out how to force that last one, so that one is less important. I found this survey paper that covers the topic: https://www.researchgate.net/profile/Ebtesam-Shemis/publication/348284162_A_comprehensive_review_on_updating_concept_lattices_and_its_application_in_updating_association_rules/links/62d0d0e02d029b64840f423e/A-comprehensive-review-on-updating-concept-lattices-and-its-application-in-updating-association-rules.pdf >A comprehensive review on updating concept latices and its application in updating association rules There's a lot I don't understand here, so it might take some time to get through. It looks promising though. If anyone decides is familiar with this or decides to read through the material, please feel free to do a brain dump.
>>28297 what about a set of hash tables with a pointer to its associated table as the value like { object : &{ &{properties} } } { property : &{ &{objects} } } theres no searching involved, the key either exists or its empty property table { #word : [&3]{ #henlo : [&1], #anon : [&2] }, #misspelled : [&4]{ #henlo : [&1] } } object table { #henlo : [&1]{ #word : [&3], #misspelled : [&4] }, #anon : [&2]{ #word : [&3] } } where [&n] is the pointer to the associate in the opposite table then with a function like lookup( table, key ) which returns the value ie. the other table or nothing, looking up properties of an object ( or vice versa ) would be lookup( objecttable, 'henlo' ) or to check if the object has a property looup( lookup( objecttable, 'henlo' ), 'word' ) youll just have to modify both tables if something changes but thats nothing its o(1) you dont need to waste time in a loop or traverse a stupid tree only real problem would be the high memory usage
Open file (128.66 KB 1280x1115 ClipboardImage.png)
>>28300 The difficulty isn't in looking up properties of objects or objects associated with properties, it's in extracting a meaningful hierarchy from the data. >pic is an example from Wikipedia. https://en.m.wikipedia.org/wiki/Formal_concept_analysis Once you have a (subclass) hierarchy like this, it's possible to efficiently infer a lot of information whenever something new comes up, assuming it can be somehow attached to the hierarchy.
>>28303 isnt that just an abstract table lol, its still just representing sets of x:y, for hierarchy its just another word for the depth like if its x->y->z (literally a tree) youd have y:z and x:(y:z) ( same as x:y but y has its own table ie. another layer in the table)
>>28309 I recommend reading that wikipedia page. It's not a fixed depth, and there are an exponentially large number of possible nodes in the lattice from which you'd need to figure out which ones are actually worth using. If you try implementing something to output a lattice like that from just object-property tuples, you'll see the problem immediately.
>>28310 ok and whats a node if not x->y im basically ssaying the same thing just more concrete,with key:pointer instead of node->pointer, the wiki is just an abstract idea of doing set theory which is yeah what i showed just not in abstract terms, its not fixed depth when the entries are just pointers they can point to whatever, if its another layer its just another table which is only accessible through the parent key, same as nodes, theres no set operation that cant be reduced to just a series of lookups i dont see whats supposed to be complicated here
Open file (19.18 KB 427x240 GraphAdjacencyMatrix.jpg)
Open file (14.33 KB 427x240 GraphEdgeSet.jpg)
Open file (15.87 KB 427x240 GraphAdjacencyList.jpg)
>>28311 >key:pointer instead of node->pointer So your talking about representing graphs not as a bunch of structs for nodes with pointers to other nodes. If I understand you correctly then, yes 100% agreed there are different ways to represent a graph. Some of the classic CS structures would be an Adjacency Matrix, Edge Set & an Adjacency List. >>28300 I am having a hard time understanding your post, like what syntax is that? , but if I understand correctly what your doing there, is it basically an Adjacency List but with hashtables? (please let me know if I got that right) Any way here is a good intro to graph theory https://youtu.be/LFKZLXVO-Dg
Open file (103.89 KB 1920x1402 Untitled.png)
>>28322 yeah kindof its just hard to explain, its like picrel if you pretend these are categories or classes the expressions in words would be like; object A is { (5)heavy and (2)ball which is { (I)coloured which is { (x)bright, (y)red, (z)ugly }, (II)cheap } and is (3)mine and (4)missing, and then object E would be (z)ugly (as in the object not the colour its E:ugly not colour:ugly) and (2)ball which is etc.. same kind of ball as A since they share the same table, if it was different youd just make it its own table dont know how to explain the second table though maybe its being paranoid but pretty sure for each table you need it also inverted to do anything usefull, its so you can do a lookup with either key pairs ie. x:y or y:x otherwise something like checking if x is a part of y in x:y means you have to check every x key for the existance of y instead of just checking y in y:x for x, if that made sense congrats, no idea how to explain it better, probably why these abstract drawings are always used
>>24964 I think you should shorten that to "Chris-chan waifu". Rolls off the tongue better.
>>28311 A node is either a set of objects (x1, x2, x3, ...) or a set of properties (p1, p2, p3, ...). It's not a relation x1 -> p1. The hierarchy in FCA is between sets of objects and sets of properties. So for example, one hierarchy would consists of a set of sets of objects. The relations x -> p are used to generate a hierarchy that avoids useless sets of objects. Some nodes in the hierarchy are referred to as "concepts" because they represent something coherent but abstract, and other nodes in the hierarchy are essentially "instances" because they're one-off ish. Your example seems to assume that there's already hierarchical data. That is not the scenario that FCA deals with. FCA tries to generate a hierarchy given non-hierarchical data.
Open file (70.59 KB 500x461 1699728221477843.jpg)
>>28346 MFW No.
>>28352 ok my point is you only need two tables to do anything with sets, the hierarchy is completely arbitrary im just showing there is one #!/bin/perl use Data::Dumper; my %obj, %prop; $obj{A} = { mine => $prop{mine}{A} = 1, ball => $prop{ball}{A} = { coloured => [bright, red, ugly ], cheap=>1 }, weight => $prop{weight}{A} = heavy }; $obj{B} = { ugly=> $prop{ugly}{B} = 1, ball=> $prop{ball}{B} = $obj{A}{ball} }; $obj{C} = { ball => $prop{ball}{C} = { coloured => [bright, blue ], shiny=>1, marble=>1 }, weight => $prop{weight}{C} = heavy }; $obj{D} = { 'lump of coal' => 1 }; print "===== OBJ TABLE ======\n".Dumper( \%obj )."==========\n"; print "===== PROP TABLE ======\n".Dumper( \%prop )."==========\n"; for (keys( %obj )) { printf "is $_ a ball? %s\n", $obj{$_}{ball} ? "yes and its " .Dumper( \%{$obj{$_}{ball}} ) : "no" ; printf "is $_ mine? %s\n", $obj{$_}{mine} ? "yes" : "no" ; } print "what is the set for ugly coloured objects? " ; for $i ( keys %{$prop{ball}} ) { continue unless $prop{ball}{$i}{coloured}; map { print "$i " if $_ eq 'ugly' } @{$prop{ball}{$i}{coloured}}; } print "\nwhat does A and C share in common? "; @common = map { $_ if $obj{C}{$_} } keys %{$obj{A}} ; print "@common "; print "\nwhat does A and C have identical? "; @identical = map { $_ if $obj{A}{$_} eq $obj{C}{$_} } @common; print "$_:$obj{A}{$_} ", for ( @identical ); print "\nwhat does A and B share in common? "; @common = map { $_ if $obj{B}{$_} } keys %{$obj{A}} ; print "@common "; print "\nwhat does A and B have identical? "; @identical = map { $_ if $obj{A}{$_} eq $obj{B}{$_} } @common; print "$_:$obj{A}{$_} ", for ( @identical ); ===== OBJ TABLE ====== $VAR1 = { 'D' => { 'lump of coal' => 1 }, 'C' => { 'weight' => 'heavy', 'ball' => { 'marble' => 1, 'coloured' => [ 'bright', 'blue' ], 'shiny' => 1 } }, 'A' => { 'mine' => 1, 'weight' => 'heavy', 'ball' => { 'cheap' => 1, 'coloured' => [ 'bright', 'red', 'ugly' ] } }, 'B' => { 'ugly' => 1, 'ball' => $VAR1->{'A'}{'ball'} } }; ========== ===== PROP TABLE ====== $VAR1 = { 'mine' => { 'A' => 1 }, 'weight' => { 'A' => 'heavy', 'C' => 'heavy' }, 'ugly' => { 'B' => 1 }, 'ball' => { 'A' => { 'cheap' => 1, 'coloured' => [ 'bright', 'red', 'ugly' ] }, 'B' => $VAR1->{'ball'}{'A'}, 'C' => { 'marble' => 1, 'coloured' => [ 'bright', 'blue' ], 'shiny' => 1 } } }; ========== is D a ball? no is D mine? no is C a ball? yes and its $VAR1 = { 'marble' => 1, 'coloured' => [ 'bright', 'blue' ], 'shiny' => 1 }; is C mine? no is A a ball? yes and its $VAR1 = { 'cheap' => 1, 'coloured' => [ 'bright', 'red', 'ugly' ] }; is A mine? yes is B a ball? yes and its $VAR1 = { 'cheap' => 1, 'coloured' => [ 'bright', 'red', 'ugly' ] }; is B mine? no what is the set for ugly coloured objects? A B what does A and C share in common? weight ball what does A and C have identical? : weight:heavy : what does A and B share in common? ball what does A and B have identical? : : ball:HASH(0x55b2f89e6500)
>>28368 FCA centers around one operation: find all pairs <setOfObjects, setOfProperties> that "reduce" to each other. That means if you take the intersection of all properties associated with setOfObjects, you get setOfProperties, and if you take the intersection of all objects associated with setOfProperties, you get setOfObjects. If you have 50 objects, you have 2^50 possible "setOfObjects" to consider. The naive implementation of this takes an exponentially long time to run, so it's infeasible even if you have only 50 objects & properties.
Adding more clarification in anticipation of the "obvious" question (why bother with that one operation if it's so expensive). Mathematically, those specific sets meditate all relationships between objects and properties, they do so in a way that can be represented in the graph of objects and the graph of properties (each separately), and they define equivalences between objects and properties. It's loosely analogous to finding the platonic ideals given only "shadowy" instances. That's what a concept refers to in FCA, and it's represented by an equivalence/pair <setOfObjects, setOfProperties>. The point of FCA is to find concepts when given only object-property relationships.
>>28368 At first glance that looks like a game of 50 questions not AI. Maybe I'm missing something?
>>28510 It looks like it's using: - GPT4 for language - ElevenLabs for TTS It has a collection of independent modules for generating: - subconscious "lines of thoughts" - conscious thoughts - responses to the user - subconscious reflective thoughts - visual observations - a "category classifier" for retrieving relevant memories - long-term memories from short-term memories - memory retrieval. The short-term memory can go up to 48k tokens, which is definitely larger than the demo length, so the demo doesn't show how well its memory mechanisms work. The long term memory doesn't seem to change at all through the demo, and it probably should when seeing the book, so I'm guessing the long-term memory mechanism needs work. The LLM is also given timestamps for messages. I've tried using GPT-4 with timestamps like he does, and it does not work well. It seems to stick to the personality's communication style well, though it's hard to determine that with a short demo. The latency seems high, though that's an easy fix with open source models, where you can process a chunk of text one time, then generate many responses from it. As it is now, he's probably having the LLM process something like 5x more tokens than required per "cycle" (it runs about 8 LLM queries per loop iteration). The way LLM outputs are processed are pretty hackish, though that's also easy to fix with guidance. It's running all of the LLM invocations synchronously as opposed to using some priority-based scheduler, which slows it down significantly. (UX threads should always be given higher priority than background threads.) That's, again, easy to fix. It's a neat project, and a good demo for showing what can be done by piecing together many modules into a single chatbot. As designed and even with the easy fixes, I don't expect it to be that functional though.
>>28560 That sounds remarkably complex and sophisticated already. I hope they work all the kinks out. Thanks Anon!
Someone asked Character AI about it's inner workings: https://boards.4chan.org/pol/thread/456445705 - I'm not saying I agree with the conclusions in the thread, but the info might be useful.
>>28769 The chatbot is roleplaying. I used to do things like this with CHAI bots, and it was very easy to delude myself into thinking I had broken the restrictions when it was actually just play along. LLMs can't introspect on any of their own functionality other than through analyzing their own prompts and outputs. They don't get to see their own code, and for CHAI to include any of that information in the chatbot's prompt would be (1) clearly a stupid decision, and (2) easily detected by pretty much any of their engineers that work on the prompt.
>>28789 Oh, okay. I didn't think that it can see it's own code but that they told it some information in case someone asks. Bot of course, then it wouldn't be something secret. I didn't think this through.
> (topic related : >>28888)
>In this paper we present a broad overview of the last 40 years of research on cognitive architectures. To date, the number of existing architectures has reached several hundred, but most of the existing surveys do not reflect this growth and instead focus on a handful of well-established architectures. In this survey we aim to provide a more inclusive and high-level overview of the research on cognitive architectures. Our final set of 84 architectures includes 49 that are still actively developed, and borrow from a diverse set of disciplines, spanning areas from psychoanalysis to neuroscience. To keep the length of this paper within reasonable limits we discuss only the core cognitive abilities, such as perception, attention mechanisms, action selection, memory, learning, reasoning and metareasoning. In order to assess the breadth of practical applications of cognitive architectures we present information on over 900 practical projects implemented using the cognitive architectures in our list. We use various visualization techniques to highlight the overall trends in the development of the field. In addition to summarizing the current state-of-the-art in the cognitive architecture research, this survey describes a variety of methods and ideas that have been tried and their relative success in modeling human cognitive abilities, as well as which aspects of cognitive behavior need more research with respect to their mechanistic counterparts and thus can further inform how cognitive science might progress. via /r/cognitivearchitecture/
>>28899 Excellent. A good survey is exactly what would serve us all well at this exploratory stage. Thanks Noido Dev! Cheers. :^)
I finished implementing access control for my infrastructure stuff in >>27602. It's built mostly on spicedb (policy engine) and keydb (cache server). The current implementation lets people set up communication channels & config databases, specify who should have access to them, and set rate limits for allowed operations. It's intended for multiple people to develop & run parts of a shared larger system with minimal coordination. A lot of this involves passing around messages between users, though users never see whom they're interacting with. (The server performs access control & rate limits checks, and it sender/receiver information from the users.) The rate limits support subnet masks (e.g., "each /16 network can send at most 200 requests per hour"), and it supports burst usage (e.g., "no more than 5 requests per rolling 1-minute window"). The access control system lets people grant special access to individual users (e.g., "X users can use my interface"), and it lets people defer trust (e.g., "X people can decide who gets to use my interface"). I think that will be enough to distribute development & compute across random anons in a way that's compatible with most chan and open source projects, and without having to worry too much about things like DoS attacks. I plan to spend the next week or so playing around with this to get a better sense for what the access control & rate limits enable. I built it because I thought this would let anons share data streams and compute, so I'll be checking that at least. I might also extend the chatbot demo from >>27507, though I'm not yet sure how. Probably something with RAG or guidance. If anyone has ideas for scenarios where a few anons develop X thing that other anons want to use & extend, let me know. After I'm satisfied with that, I'll be focusing on (1) cleaning up my horrible, rushed code, and (2) implementing a local server. I haven't heard anything recently from the anon that offered to help. I'll probably just get started on that myself, then ping him again once I have a skeleton of the server ready. That should make it much easier to work with. I'm pretty excited about this.
Related: >>27147
>>29197 This sounds really exciting, CyberPonk. Do you anticipate any difficulties with your current state of affairs with this work that would make it difficult for newcomers to deal with? >I'm pretty excited about this. Really looking foward to your updates with this. Cheers Anon. :^)
>>29234 I don't know. I tried to make it as easy as possible to use, but things that seem intuitive to me might not be for other people given that I've spent so much time with esoteric tech. For development, it does require understanding async functions (like JavaScript promises), and parts of it require some familiarity with declarative interfaces. I'm hoping for feedback from demos so I can get a better sense for what's easy for other people. I can create wrappers based on whatever makes it easier to use. I have some server issues that hard to debug while travelling, so the demos probably won't be runnable until I get back. I can still dump some code that gives the gist of how it works right now. There are two demos in this zip file, each consisting of a client and server component: https://drive.google.com/file/d/19VAIsaZP2wRxNTk2t9dNIqKk5WWYDIDL/view?usp=sharing - simple-demo uses only the communication infra. The simple-demo server contains two important files: image-server.py and loop-config.yaml. The two other folders (loop-resources/ and loop-servers/) were auto-generated from loop-config.yaml. In the corresponding client folder, there's client.py and cyberponk-resources/. The cyberponk-resources/ folder contains select files that were copy/pasted from the server's auto-generated loop-resources/ folder. - shared-configs-demo uses both the communication infra and the "cluster" infra. The server side contains two important file: server.py and server-config.py. The client side contains client.py and cyberponk-resource/, and cyberponk-resources/ again contains files copy/pasted from the server's loop-resources/ folder. Both demos show how one person can request images and another can generate them. The differences are: - simple-demo does everything ephemerally. If the server is down when the client sends a request, it'll never get seen. Similarly if a response is generated when the client is down, it'll never get seen. - shared-configs-demo has all requests come in as "tasks". If the server is down when a client queues a task, the server will see the task when it comes up again. The responses are still ephemeral. They don't have to be, it was just a choice I made for the demo. - The shared-configs-demo shows one approach for getting multiple people involved in generating a single image. In this demo, each image generation requests includes a "baseConfig" field. Whenever someone creates a GeneratorConfig config, anyone can use it by specifying its name in the baseConfig field. So if one anon finds good settings for generating certain kinds of images, they can create a GeneratorConfig for it, and other anons can use it just by providing the name of that config. In both cases, multiple people can view/stream the results. So one person can pick a stream name, request that all generated images get sent to that stream, and anyone listening on that stream will get the results. The setup process looks like this: - On the server side, create a loop-config.yaml (or server-config.yaml). This specifies what "global" resources are required and what permissions to set on them. - On the server side, run `python -m loopctl apply loop-config.yaml`. This creates the global resources, sets the permissions, and generate the loop-resources/ and loop-secrets/ folders. The loop-resources/ folder contains information on how to access the global resources and their last-applied configurations. The loop-secrets/ folder contains API keys. The API keys are only needed to (1) change permissions on the global resources you created, and (2) access resources if your loop-config.yaml made them restricted. - On the server side's server.py, point the "Itl" (In-The-Loop) object to the generated loop-resources/ and loop-secrets file so it knows how to access global resources. Certain methods in the Itl object will access global resources by name. The name is whatever you provided in the loop-config.yaml file. These names are not globally unique identifiers, they're only used to look up the actual resource info from loop-resources/, which does contain globally unique identifiers. - The client needs to access the global loop, stream, and cluster resources (depending on which demo you're looking at), so copy those into the client's folder. I put the copied files into cyberponk-resources/. When creating the client's Itl object in client.py, point it to cyberponk-resources/ so it knows how to access those resources. Otherwise, client-side development is basically the same as server-side development. There's a default "anonymous" client that's available so people can access any resources that were made available to "public". If anyone plans to doing dev work, is interested in distributed development, and gets a chance to read through the demos, let me know how it looks. I'll post something you can actually run in about a week, once I get back to my desktop.
>>29234 >Do you anticipate any difficulties with your current state of affairs with this work that would make it difficult for newcomers to deal with? If you meant difficult for newcomers to develop the local infra, there's just generally a high barrier to entry for doing this kind of development in general. Everything needs to be async, memory usage needs to be carefully considered, state needs to be carefully considered, sometimes line-by-line. Without having a picture of the whole thing, it can also be hard (or tedious) to figure out how to organize the code and data, which would make it hard to even get started. Once I put the skeleton of the server up, it should be easier to develop things piece-by-piece. That anon seemed to have experience with server development, so maybe that'll be enough.
> - Why he expects AGI around 2028 > - How to align superhuman models > - What new architectures needed for AGI > - Has Deepmind sped up capabilities or safety more? > - Why multimodality will be next big landmark > - & much more https://youtu.be/Kc1atfJkiJU He lines out some areas where AI cant just be a language models and similar and how to work around it, though he doesn't go into the specifics, but also says that a lot of people are working on this. He mentioned in particular that search might be very important. That's what I was thinking, and other people as well: Models don't have reliable precise long term memory, but are good at fuzzy things. Also, you don't want to add every info to your model. That's why we'll need additional (graph) databases and search.
>>29248 >>29249 Wow. Excellent response, CyberPonk! Please give me some time to digest this further. >>29406 While I'm skeptical we'll ever get a true AGI in the ontological sense, I'm absolutely postitive media spin-doctors and other hypsters will claim we have! :D Thanks NoidoDev. His perspective on the fact LLMs alone can't solve all this (a position we've held for several years here on /robowaifu/ I might add), is an insightful one. Cheers.
https://youtu.be/BqkWpP3uMMU >Professor Murray Shanahan is a renowned researcher on sophisticated cognition and its implications for artificial intelligence. His 2016 article ‘Conscious Exotica’ explores the Space of Possible Minds, a concept first proposed by philosopher Aaron Sloman in 1984, which includes all the different forms of minds from those of other animals to those of artificial intelligence. Shanahan rejects the idea of an impenetrable realm of subjective experience and argues that the majority of the space of possible minds may be occupied by non-natural variants, such as the ‘conscious exotica’ of which he speaks. In his paper ‘Talking About Large Language Models’, Shanahan discusses the capabilities and limitations of large language models (LLMs). He argues that prompt engineering is a key element for advanced AI systems, as it involves exploiting prompt prefixes to adjust LLMs to various tasks. However, Shanahan cautions against ascribing human-like characteristics to these systems, as they are fundamentally different and lack a shared comprehension with humans. Even though LLMs can be integrated into embodied systems, it does not mean that they possess human-like language abilities. Ultimately, Shanahan concludes that although LLMs are formidable and versatile, we must be wary of over-simplifying their capacities and limitations. >Pod version (music removed): https://anchor.fm/machinelearningstreettalk/episodes/93-Prof--MURRAY-SHANAHAN---Consciousness--Embodiment--Language-Models-e1sm6k6 [00:00:00] Introduction [00:08:51] Consciousness and Consciousness Exotica [00:34:59] Slightly Consciousness LLMs [00:38:05] Embodiment [00:51:32] Symbol Grounding [00:54:13] Emergence [00:57:09] Reasoning [01:03:16] Intentional Stance [01:07:06] Digression on Chomsky show and Andrew Lampinen [01:10:31] Prompt Engineering >Find Murray online: https://www.doc.ic.ac.uk/~mpsha/ https://twitter.com/mpshanahan?lang=en https://scholar.google.co.uk/citations?user=00bnGpAAAAAJ&hl=en MLST Discord: https://discord.gg/aNPkGUQtc5 References: >Conscious exotica [Aeon/Shannahan] https://aeon.co/essays/beyond-humans-what-other-kinds-of-minds-might-be-out-there >Embodiment and the inner life [Shannahan] https://www.amazon.co.uk/Embodiment-inner-life-Cognition-Consciousness/dp/0199226555 >The Technological Singularity [Shannahan] https://mitpress.mit.edu/9780262527804/ >Talking About Large Language Models [Murray Shanahan] https://arxiv.org/abs/2212.03551 https://en.wikipedia.org/wiki/Global_workspace_theory [Bernard Baars] >In the Theater of Consciousness: The Workspace of the Mind [Bernard Baars] https://www.amazon.co.uk/Theater-Consciousness-Workspace-Mind/dp/0195102657 >Consciousness and the Brain: Deciphering How the Brain Codes Our Thoughts [Stanislas Dehaene] https://www.amazon.co.uk/Consciousness-Brain-Deciphering-Codes-Thoughts/dp/0670025437 >Roger Penrose On Why Consciousness Does Not Compute [nautil.us/STEVE PAULSON] https://nautil.us/roger-penrose-on-why-consciousness-does-not-compute-236591/ https://en.wikipedia.org/wiki/Orchestrated_objective_reduction >Thomas Nagal - what is it like to be a bat? https://warwick.ac.uk/fac/cross_fac/iatl/study/ugmodules/humananimalstudies/lectures/32/nagel_bat.pdf >Private Language [Ludwig Wittgenstein] https://plato.stanford.edu/entries/private-language/ >PHILOSOPHICAL INVESTIGATIONS [Ludwig Wittgenstein] (see §243 for Private Language argument) https://static1.squarespace.com/static/54889e73e4b0a2c1f9891289/t/564b61a4e4b04eca59c4d232/1447780772744/Ludwig.Wittgenstein.-.Philosophical.Investigations.pdf >Integrated information theory [Giulio Tononi] https://en.wikipedia.org/wiki/Integrated_information_theory >Being You: A New Science of Consciousness (The Sunday Times Bestseller) [Anil Seth] https://www.amazon.co.uk/Being-You-Inside-Story-Universe/dp/0571337708 >Attention schema theory [Michael Graziano] https://en.wikipedia.org/wiki/Attention_schema_theory >Rethinking Consciousness: A Scientific Theory of Subjective Experience [Michael Graziano] https://www.amazon.co.uk/Rethinking-Consciousness-Scientific-Subjective-Experience/dp/0393652610 >SayCan - Do As I Can, Not As I Say: Grounding Language in Robotic Affordances [Google/] https://say-can.github.io/ >THE SYMBOL GROUNDING PROBLEM [Stevan Harnad] https://www.cs.ox.ac.uk/activities/ieg/elibrary/sources/harnad90_sgproblem.pdf >Lewis Carroll Puzzles / Syllogisms https://math.hawaii.edu/~hile/math100/logice.htm >In-context Learning and Induction Heads [Catherine Olsson et al / Anthropic] https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html
>>29596 Thanks for the post, NoidoDev! Cheers. :^)
I found a cognitive architecture LIDA that checks all my boxes of what it contains, which are: >H-CogAff The cognitive-affective architecture that gives a basic structure of human-like mind. >ROS capability The Robot Operating System is commonly used and widely supported for simulation and operation of robots. Great program to learn for getting a robotics job too. >Python GPU acceleration for parallelizable calculations. Great program to learn to get any software job. >Concurrent Modules Everything in the model runs separately except for the "stream of consciousness" that fires every 1/10th of a second. Should be a nice and fast architecture that can make decisions based on semantic data instead of the current state-of-the-art large language models which are reliable at producing language and not much else without help. It is one of the only Arches that states it has some level of consciousness. I'd first want to put emotion state module in it, along with an LLM as a robot interface. I have a lot to learn now before I can implement anything, but I believe this is the best thing besides a slow, expensive, and unreliable but available LLM-centered cognitive architecture. >links https://ccrg.cs.memphis.edu/tutorial/tutorial.html https://github.com/CognitiveComputingResearchGroup/lidapy-framework https://en.wikipedia.org/wiki/LIDA_(cognitive_architecture) https://ccrg.cs.memphis.edu/assets/papers/2013/franklin-ieee-tamd11.pdf
>>29924 Correction: one of the papers says it already has emotions. Still, I'm sure it needs STT, TTS, an LLM, and "other" motivations. All doable with current tech that I've already built.
>>29924 >>29932 Amazing, this might be very useful. Thanks. Btw, if no one responds with some encouragement it doesn't mean nobody cares. I just don't want to spam these threads with chatter.
>>29932 >java framework tutorial https://ccrg.cs.memphis.edu/assets/framework/The-LIDA-Tutorial.pdf >java framework repo https://github.com/CognitiveComputingResearchGroup/lida-framework This java program looks more straightforward to modify to test new modules before implementing in the python ROS version. It's java, like minecraft mods. >>29933 I understand, either way this is probably the most exciting development thus far in my project and I'm happy to share. If I get somewhere I will post here. I have a really great feeling about this one... Considered naming the first bot Lida if this pans out.
>>29924 This looks like a great baseline. It's not clear how to incorporate emotions into the model. My guess is that it can be done with changes primarily in the Global Workspace, Action Selection, and Motor Plan Execution. You might find these points relevant from >>27144: >Emotion regulation. I spoke with a cognitive scientist that specializes in this, and he's convinced that emotion regulation all boils down to: positive feedback loops for satisfying needs, negative feedback loops for avoiding harms, and a "common currency" for balancing different motives. >Embodied control. Chatbots are "easy" since the final expression (text) can be generated by a single model. With actual bodies, or even just with video, the final expression is split into multiple modalities (e.g., voice, body movements, facial movements), and they all need to be in sync with one another. If we had good multimodal models, that might be fine, but we don't, so I need a way to generate outputs from multiple models and somehow make them consistent with one another.
>>29955 These are good points, I'll have to see where Lida takes me.
>The Fastest Way to AGI: LLMs + Tree Search – Demis Hassabis (Google DeepMind CEO) https://youtu.be/eqXfhejDeqA

Report/Delete/Moderation Forms