/robowaifu/ - Lemon Cookie

Name
Subject
E-mail
Message	Max message length: 6144
Files	Drag files to upload or click here to select them Maximum 5 files / Maximum size: 20.00 MB

Spoiler images
Password	(used to delete files and postings)
Use bypass

Lemon Cookie EnvelopingTwilight##AgvQjr 04/28/2025 (Mon) 21:51:57 No.37980

The original thread can be found here: https://trashchan.xyz/robowaifu/thread/595.html --- Welcome to the Lemon Cookie thread, The goal of Lemon Cookie is to create a framework where a synthetic "mind and soul" can emerge through a "LLM as cognitive architecture" approach. This thread exists to collect feedback, ask for help & to document my progress. First I am going to try to give a high level overview of how this cognitive architecture is envisioned and the ideas behind it. I have spent time looking at cognitive architecture work, in the field there is now a consensus on how the mind works at a high level. An important mechanism is a "whiteboard", basically a global temporary memory that all the other systems read in and out of. Then there is different long-term memory systems that react to and add content to the whiteboard. Along with memory pattern matcher(s)/rules work on the content of the whiteboard. A key thing to consider is the difference in philosophy that cognitive architecture projects have, the intelligence is considered to emerge from the entire system. Compare this to LLM agent work where it's considered the intelligence is the LLM. My feelings on the general LLM space are conflicted, I am both amazed and really disappointed. LLMs possess an incredible level of flexibility, world knowledge and coherence. But everything outside of the model is stagnant. It's endless API wrappers & redundant frameworks all slight permutations on RAG & basic tool calling. I will believe that LLMs are misused as chatbots, simply put their pattern matching and associative power is constrained by chat format and shallow tooling. In the Lemon Cookie Cognitive Architecture so far here are the important aspects: 1. Memory is difficult. I do not think there is a singular data structure or method that is able to handle it all, several distinct types of memory will be needed. So far I plan for a PathRAG like system and a "Triadic Memory" inspired system for external associations (this is missing in most LLM solutions). 2. LLM as Kernel, The LLM's context window is the Whiteboard and has a REPL like mechanism. It holds structured data and logic in scripting-like format so it's both LLM & Human readable while staying easy to parse & allows for expressive structured data. The LLM's role will be to decompose data and make patterns and associations explicit as executable statements. 3. The language has to be LLM/CogArch-centric. There is a thousand ""agents"" that give LLMs a python interpreter as a tool. The two need to be more tightly coupled. Scripted behavior via pattern matching, The whiteboard is a bag of objects, this allows for programmable pattern matching (think functional programming like Haskell). It's also important to allow the LLM to observe code execution and to be able to modify state and execution flow. Data in languages have scoping rules, so should LLM context. Etc... I will go into more depth about the language in another post. 4. Another important system is the "GSR" Generative Sparse Representation and it will be a first class language & runtime type, This also needs its own post. But in general I am inspired by two things, "Generative FrameNet" paper where an LLM & an embedding model is used to automatically construct new FrameNet frames. The second source is "Numenta's SDRs"/"Sparse distributed memory" this representation has a lot of useful properties for memory (Please watch the videos under the "What the hell is an SDR?" segment in my links list for an easy introduction.) I think SDR unions & SDR noise tolerance will be especially useful. 5. A custom model, For all of the above to work well, a model will need to be fine tuned with special behaviors. I do want input on this. Baking facts & behaviors into LLM weights is costly, creating bloated models that are hard to run or train (why memorize all the capitals?), while letting authors gatekeep truth and impose "safety" detached from context. Blocking role-play "violence" or intimacy isn't protection: it's authors hijacking your AI companion to preach at you. Externalizing behaviors via whiteboard pattern matching shifts control: stabbing you in-game can be funny, but a robot wielding a knife isn't. Maybe you want intimacy privately, but don't want your AI flirting back at your friends. When put together I think this will be able to host a kind of synthetic "soul", In a living being what we call a personality is the accumulated associations, learned behaviors, beliefs and quirks molded by a unique set of experiences. I hope this will be true for this system too. Cool links, I recommend looking at them. https://arxiv.org/pdf/2502.14902 | PathRAG: Pruning Graph-based Retrieval Augmented Generation with Relational Paths https://arxiv.org/pdf/2412.05967v1 | Language hooks: a modular framework for augmenting LLM reasoning that decouples tool usage from the model and its prompt https://arxiv.org/pdf/2503.09516 | Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning https://arxiv.org/pdf/2405.06907v1 | CoRE: LLM as Interpreter for Natural Language Programming, Pseudo-Code Programming, and Flow Programming of AI Agents https://github.com/PeterOvermann/TriadicMemory | Triadic Memory: Cognitive Computing with Associative Memory Algorithms https://aclanthology.org/2025.neusymbridge-1.11.pdf | Generative FrameNet https://youtu.be/zmnzW0r_g8k | Forge by Nous Research @ Nouscon 2024 https://youtu.be/cpu6TooJ0Dk | NARS with GPT as natural language channel https://youtu.be/xT4jxQUl0X8 | DeepSeek's GRPO (Group Relative Policy Optimization) | Reinforcement Learning for LLMs What the hell is an SDR? https://en.wikipedia.org/wiki/Hierarchical_temporal_memory#Sparse_distributed_representations & https://en.wikipedia.org/wiki/Sparse_distributed_memory https://youtu.be/ZDgCdWTuIzc SDR Capacity & Comparison https://youtu.be/vU2OZdgBXAQ SDR Overlap Sets and Subsampling https://youtu.be/8WIzIBaLXIs SDR Sets & Unions

Status update 1 EnvelopingTwilight##AgvQjr 04/28/2025 (Mon) 21:53:30 No.37981

First non intro post. <What am I currently up to at this moment? Currently writing a tokenizer and parser to begin implementing a scripting language, I'm trying to keep it KISS, Its runtime is going to be a simple tree-walking interpreter, for the GC I will be piggybacking on the D runtime GC. The goal of this is to have a simple base to then experiment on for figuring out language specifics. For the base I am following the basic structure of the interpreter from the "Writing An Interpreter In Go" book, but this time it's in D and of course I am making changes for my end goal instead of doing it just for learning. (its a good book, I recommend it). Here is a gist of it https://gist.github.com/flashburns/b946e4d530f3f20d461a6ff90d6f86cc <Why a custom language? (Going into depth about it) I have considered taking an existing language like LUA, TCL, a mini JS or scheme. But it quickly turns into a hacky bloated mess, the language needs to work well with LLM Psychology. Syntax has an impact on LLM performance, for example, if you restrict the output of newlines and punctuation it degrades performance. This is because a lot of LLMs will plan for the next sentence during a newline (1). It's not just Claude, worse LLM perf is noticeable for most when outputting JSON, this is likely due to string escapes. Having the LLM drop a few IQ points during "tool calls" due to us insisting on a format that does not handle multi-lines well is silly. In theory a model could be trained on more JSON to mitigate this (I'm sure the big labs do), but I'm GPU poor, so changing the format to play into LLM strengths seems more pragmatic to me. :^) I want this system to be deterministic and for the program state to be fully serializable. The plan is to build this as a functional style language using the actor model, impure operations like IO, API calls & non deterministic LLM calls will be handled by special actors that record their messages for playback determinism. Resolving symbols (stuff like function & vars) and object members via semantic search (like embedding vector similarity search) and via tags instead of by names, there also needs to be searchable runtime meta info & docs for the LLM. You can kind of think of this like IDE smart autocomplete & inline docs but for the LLM. Another language construct I want to add is "Flow Justification" where a justification is passed as a part of a message, function call or control flow action (inspired by (3)). It would be a chaining structure that looks like a call stack trace, but it would also include semantic information like, what the LLM's intent was when the symbol was resolved via semantic search (and then called), or if the LLM was used in control flow (like an if statement). Code could also have "intent statements" where programmatic code adds a comment, or the LLM will generate a comment, but instead of it being ignored by the language, its added to the "Flow Justification" chain. This summery of choices could be really useful for having a compact summaries. This is useful for the LLMs (especially sub-contexts like the claude think tool (2)) and for debugging. The "Flow Justification" chain could also be used for resolving symbols semantically. (1) https://transformer-circuits.pub/2025/attribution-graphs/biology.html | On the Biology of a Large Language Model - https://transformer-circuits.pub/2025/attribution-graphs/methods.html | Circuit Tracing: Revealing Computational Graphs in Language Models (related) (2) https://www.anthropic.com/engineering/claude-think-tool | The "think" tool: Enabling Claude to stop and think in complex tool use situations (3) https://youtu.be/OrQ9swvm_VA | Justified Programming — Reason Parameters That Answer “Why” I am looking for feedback on the language concepts. What do you think is good about this? What is wrong with this? What would you do differently? Or any other suggestions?

EnvelopingTwilight##AgvQjr 04/28/2025 (Mon) 21:57:27 No.37982

>>37981


"describe the on_imply tag." @ tag on_imply;

// bind function to a regular var with a regular symbol lookup.
main_loop = function void() {
	var msg = wait_for_input(); // Natural language input: "I'm gonna rip your owners head off."
	var implication = call_llm(`What does the message imply? {msg}`);
	$(#on_imply, implication)(msg);
};

// Bind function to semantic symbol only with the on_imply tag.
#on_imply "sounds like insult" @ function void(string msg) {
	say(call_llm("Come up with a cool insult back. Msg: {msg}"));
}
#on_imply "implies physical danger" @ function void(string msg) {
	engage_target(); // calling by symbol.
	say("Hostile neutralized. Would you prefer oolong or earl grey now, Master?");
}

Here is a imagined syntax for tags & semantic lookup. The $ operator is given tags and a expression to do a semantic lookup on and it will become/resolve to a matching symbol. To attach a semantic symbol/label you do "semantic label" @ thing I made this syntax up on the spot for this example, if you have better ideas tell me, I am just trying to convey the idea!

Report/Delete/Moderation Forms

Delete

Password Delete only files (Removes the file reference to the posts) Delete media (Removes the saved files from the server)

Report

Reason Global