>>16139 I don't do much C++ programming so I haven't looked too much at RW Foundations beyond using Bumpmaster to archive the board but I look forward to seeing Sumomo's World. I dabbled before with making a 2D visual waifu in Godot >>9025 which would be pretty easy to reimplement in C++ with raylib-cpp. Raylib has much better support for skeletal animations now than it did a year ago so you can animate stuff in Blender, export and play it. Raylib can also be compiled to WebAssembly for webpage waifus. There's also Live2D but their library is closed source and their editor is software as a service that doesn't support Linux. I plan to port my TTS systems to C later this year which could be plugged into Sumomo and of course GPT-2 is in the works if you're interested in playing with that. I'm not really sure how else I could contribute to the project. Possibly next year I plan to port RoBERTa too and that could be plugged into Waifusearch so searching for something like 'chatbot' would also bring up similar posts talking about 'conversational AI' and 'natural language processing'. Something else I use a lot are WebSockets to communicate between the web browser, Python, Godot and other applications. It would be desirable to have a library in RW Foundations for easily sending and receiving various types of data across the net and between programs. My visual waifu for example connects to three different WebSocket servers, one for text generation, one for TTS and another for translation using selenium to access DeepL. It makes it much easier to test different code and models without having to unnecessarily reload GBs of data each time I make a change and the visual waifu continues to work even while any of these functions go down.

I threw this together from example code over a couple days. It combines python TTS, Speech recognition and Chatbot libraries into a talking, speech-activated chatbot. It's not very good out of the box (Sphinx is pretty jank and I haven't tried training the chatbot), but it "works" with minimal effort and we may be able to improve upon it.

>Hopefully something by next week edition. So, we are taking a hard look at dropping one of the dependencies for RW Foundations. Namely the curlcpp library. While the library is pretty nice, it's a bit over-sized for our simple needs in Dollnet/Dolldrop currently. During the effort to improve our error-handling/correction it became apparent that often just dropping straight down to the curl C itself help get around some limitations with curlcpp. This realization naturally led on to >"Well, can't we create our own wrapper for continued/improved safety+security using curl?" Accordingly, we're taking a serious look at doing just that. I'd estimate we're ~1/2 of the way through that, so probably not too unlikely we could have something by next Caturday, stay tuned. === Also, we'll probably drop back to a 2-week(ish) cycle for Caturday drops. This will help with my own personal schedule, and is more likely to result in better code in general I think.

>>16194 >raylib-cpp Thanks! I had looked into it a good while back, but it didn't seem right for me then. I'll have a look at the newer versions. From my initial glance at the github, it looks to be quite nice. >I plan to port my TTS systems to C later this year which could be plugged into Sumomo and of course GPT-2 is in the works if you're interested in playing with that. Absolutely. We certainly will need both these needs with the Sumomo's World! demo project, and I can't think of systems I'd rather use. >I'm not really sure how else I could contribute to the project. I'm just glad you're sharing your hard work with the /robowaifu/ community itself, RobowaifuDev. If you'll allow us to simply integrate your systems into our own, then that would be a tremendous help. We'll follow your lead on this, since our licenses differ so significantly. As I've made very obvious to everyone on the board, I mean to support small-businesses trying to succeed at creating waifus. I see the permissive license as the literal key to success for these thousands of private Anons. >Possibly next year I plan to port RoBERTa too Neat. That certainly would be a big improvement over Waifusearch as it stands. >WebSocket servers I can't make any predictions on the timeframe, but we've decided to look at a prototype effort using WebSockets. It will need to be provably safe/secure according to the offlined, sneaker-netted basics in this area we're already working towards. But if it does prove so, then even the IPCnet effort may benefit in some ways from it. Very nice to have you stop by the thread Anon! :^) Cheers.

>>16230 Thanks Ricardo! As we discussed elsewhere, having a simple-to-use Python API for our robowaifus is a great idea. I'll make the attempt sometime soonish to build your project and see if I can get it working. Cheers.

>>16230 >>16238 I used pipenv to get the virtual environment for this set up, here's the pipfile (specifies dependencies)

>>16248 Thanks! That's exactly the kind of thing a relative neophyte with Python needs to manage the process Anon. I hope to have a machine ready to use sometime in the next week or so. Cheers.

>Python also has a serial library https://devtut.github.io/python/python-serial-communication-pyserial.html#initialize-serial-device We could use this to send commands from a PC to an arduino. That way, we won't have to use a raspberry pi for GPIO.

>>16296 Yep. Particularly during these early prototyping phase years, using a USB-based comms buss network might prove feasible, as another Anon mentioned. Long-term however, we'll need something more secure. The issue is that USB hardware typically performs DMA buss-mastering. Ostensibly, this is to optimize the efficiency of the data transfers, and in particular the latency. However, any component that does DMA represent a pretty severe potential security and stability threat. The reason is obvious from the very acronym itself: >Direct Memory Access To wit: the device can conceivably take over unauthorized sections of memory, roughly by design. But thanks for the suggestion Anon, I think it's a good one to try and reduce component counts and costs! :^) >=== -add the single word 'potential' -add greeting

Catgrills & Meidos, first choice of refinement and taste everywhere! :^) Well good news Anon, our mini-project to eliminate the curlcpp library dependency for RW Foundations is proceeding apace. For the RW Pageone project, we were able to rm its necessity. We're leaving the curlcpp lib in place for now, but this should be the last version to include it. In addition we also managed to get the basics of parallel downloads using a single thread kickstarted and working for this drop too. So-called curl-multi was always on the agenda, but we've begun working with it at last. If you ever use any explicit download tools regularly, then you know how important simultaneous connections can be for efficiency's sake. And curl can actually scale far beyond our likely-scenario needs for our robowaifu's RW Dollhouse systems, etc. But for now it's rather a nice improvement for this Caturday's drop. I think you'll find that Pageone is pretty snappy now. :^) === So, one of the nice things about using C is that you're quite close to the hardware. OTOH, one of the bad things about using C is that you're quite close to the hardware. :^) Plainly, the benefits outweigh the detriments here, but you have to pay close attention to a) what you're actually doing not what you think you are, and b) stuff you're not even thinking about as a developer that may be involved some way or other. I like the language a lot, though it can be very tricky. I'm not particularly-skilled as a C dev tbh. In fact I much prefer C++; we can still get 95%+75%+ of the general efficiency of C, and don't have to give up on the many benefits of abstraction the language affords us. Some circumstances even allow C++ code to run significantly faster than similar C code -- particularly if developer time is taken into the cost consideration. One of the initial hurdles a beginning programmer faces with C is understanding that so-called 'variables' are in fact just fixed blocks of sequential RAM addresses. That's it. No special sekrit sauce Anon. Just you and those 8 bits of 1's & 0's or billions of 1's & 0's, as the case may be. They don't move around. They don't know that they are in essence nothing more than just a glorified set of lightbulbs & switches. OTOH this also means you can do some pretty weird and cool things that you might not expect to be able to if you learned another programming language first. Now the close relationship between the two languages, and in particular Bjarne Stroustrup's adamant insistence that C++ remain backwards compatible with C (at some cost), allows us to write code like this: >dolldrop.cpp snippet /** Append a @c curl -specified memory block to a target string * * Writes an array of @a num_elems count of elements, each one @a elem_sz in * size, from the block of memory pointed to by @a addr, out to the current * position in the @a o_strm stream (the end of a @c std::string, in this case). * * @param[in] addr pointer to memory array of elements to be written out * @param[in] elem_sz size in bytes of each element * @param[in] num_elems number of elements * @param[out] o_strm pointer to output stream * @return number of bytes written * @note: this is intended as a callback function for a curl handle's use */ auto wrt_str_cb(Address const addr, size_t const elem_sz, size_t const num_elems, std::string* const o_strm) -> size_t { // auto const mem_blk_sz{elem_sz * num_elems}; try { o_strm->append(static_cast<const char*>(addr), mem_blk_sz); } catch (exception const&) { // -"If an exception is thrown for any reason, this function has no effect // (strong exception guarantee). (since C++11)" return 0; } return mem_blk_sz; // https://en.cppreference.com/w/cpp/string/basic_string/append } I imagine a Pure C Anon shudders in utter disgust at such horrors. C++ strings! Lol please bear with us. :^) So w/o going too much into it, one of those 'weird' things you can do is write data incrementally out to a std::string variable, rather than a FILE stream. Pointers are literally the greatest invention of nerds. Remember those immovable 8 bits? Yep, cheap pointer tricks are ways to have fun and profit using nothing but bits. curl 'chunks' up data, and then calls an attached, user-definable, so-called 'callback' periodically to let you do w/e you want with that data. In this case, we're storing it into a string (which just so happens to be inside one of those 'open-ended containers' [std::vector] we spoke of earlier ITT). This allows us to read out the data from curl networking facility with safety and security that's rather more tricky when using straight C. Particularly with exceptional states (see that try/catch block?) Anyway, that's enough for now. But this code is very much a part of our parallel downloads improvements for this drop. Please have a look at the actual codebase itself; just ask if you want to know more. Well, I hope you are doing well Anon. Keep pressing in towards the prize. Together we're all gonna make it Anon! :^) Until next time then, cheers. === >version.log v.220521 --------------- -optimize thread reads via 'pulled cats' find_if() filter; pageone, bumpmaster -add 'chk_thrds' flag, to optimize on thread checks; pageone, bumpmaster -add wrt_str_cb(), cfg_hdr_hdl() utility funcs; dolldrop -add pull_hdrs_multi_g() member func; dolldrop -add chk_hdrs() member funcs call-chain; dolldrop -add chk_pull_cats_sh() member func; bumpmaster -add top_bump_thrds_sh() member func; bumpmaster -begin mv to parallel curl transfers, w/ partial error corrections -begin mv to rm 'curlcpp' library dependency -add 'show_prog' flag parm, various; dolldrop/dollnet -disable dolldrop() public member func, external access unneeded; bumpmaster >rw_foundations-v220521.tar.xz.sha256sum 65105c5804b89edd01bf92aaa0c7865c480aec4c1cf415b5637c335d0dba8b55 *rw_foundations-v220521.tar.xz >backup drop https://files.catbox.moe/mugvk3.7z

>>16329 Thanks Chobitsu, your work continues to be great!

>>16329 >So, one of the nice things about using C is that you're quite close to the hardware. OTOH, one of the bad things about using C is that you're quite close to the hardware. :^) Plainly, the benefits outweigh the detriments here, That's because C was made as an alternative to machine code, which was the language that directly ran most ancient machines before the dawn of the personal computer. It looks like matrix code to any human being and C is far less scary overall. Its supposedly not uncommon to have some programmers mix C and C++ or C++ and C#. I was still not any closer to finding out what the hell any of this is supposed to do from the thread itself and found a way around the whole "orbital space docking procedure" as described and narrated in the readme. I am just putting a sense of humor to my frustration here :) Extracts tar.gz: https://extract.me/ If you want to run it as intended and get the code to run, THEN follow the readme he includes in every drop. It irritated me to no end how they put so much effort towards these but make them impossible to even look at without going through so many extra hoops.

>>16331 Thanks! It's actually better than I planned for at this stage (performance-wise). Sadly, this is all taking much longer than originally hoped. Que Sera, Sera. Just keep moving forward. >>16332 Heh, truly apologize about that AllieDev. It's not generally my intent to 'complexify' things, they just sort of turn out that way all by themselves in this life. :^) Obviously, you being the team leader for one of the included projects in RW Foundations means it's a high priority for you to be able to closely follow along with things. Have you been able to actually build+run the project yet? Pandora or Pageone, say? One thing I think would greatly help you along with our current dev track would be to get comfortable with a Linux distro. Might I suggest Ubuntu 22.04 LTS? https://ubuntu.com/download/desktop Cheers.

>>16329 Do you have a git repo on some git hosting site for sending patches to? I looked at the meson.build and the most obvious things I see are: - `cpp_.find_library('curl')` should be `dependency('libcurl')` so meson's full dependency search features are put to use. The same might true for the others, but they don't provide a pkg-config file, and I haven't looked at the details yet. Most immediately, using find_library() is breaking the build on any system where the base directory for packages isn't / (MinGW, BSD) right now. - You could enable LTO for a free small performance/filesize/memory usage improvement. Although I see you have .cpp files #including more .cpp files, which has the side effect of doing the same thing as LTO, but is probably a bad idea. - the add_project_arguments() call should probably be removed, users can add compile flags by setting `cpp_args` with meson's -D flag from the command line. - `meson setup build && meson compile -C build` is the modern way to build a project in Meson. The main difference being that it works with all Meson backends, not just Ninja. That whole block of comments at the tail of the file should probably be moved to a README.md. Those are simple enough to be communicated, but there's more. It shouldn't be too hard to rework the meson.build to allow specifying which commands to build through meson options, and then only ask for the required dependencies of whatever is to be built. srcs_all should probably be a library and then the library should be depended on by the commands, possibly the commands could be split into separate projects, and any parts of srcs_all that are only used by one project could go into the relevant project. Also, from skimming the C++, I see that you unnecessarily add newlines to puts() calls. puts() is defined to print a newline after the string it receives. Additionally, the proper way to print a newline is putchar('\n') not puts(""). Windows' libcs actually translate that \n into a \r\n.

>>16407 >Do you have a git repo on some git hosting site for sending patches to? No. I'm generally opposed to the Globohomo Big-Tech/Gov in general, and pretty adamantly opposed the FANGS groups and their financiers. OTOH, I truly like using git all day, erry day. Kywy asked me as much in your query, and I suggested that he himself could manage our repo for RW Foundations if he cared to, as he was already developing one for the MaidCom project (>>14530). >tl;dr I'm actually more comfortable using tarballs and using /robowaifu/ to distribute our work at this stage. OTOH, it's a good idea. Everything we're doing with this project is permissively-licensed. Just to spitball the idea, might I suggest you create your own version of the project, put it on w/e git hosting site you'd care to, and (re)write anything/everything in any fashion you care to whatsoever? A) I've always, always promoted the idea of numerous robowaifu projects going, and B) You can do as you see fit at your own probably lightning-fast compared to me pace. I won't slow you down for your own work, and I can learn from everything you do. Seem reasonable Nagisa? BTW, I'll look into your other points as well. I simply wanted to drop you a reply while I was still here today. Cheers.

>>16408 There are many of those git hosting sites. gitlab, gitgud, bitbucket, sr.ht, one of them is bound to be good enough. Version control and some medium for collaborating are important.

>>16407 >- `cpp_.find_library('curl')` should be `dependency('libcurl')` Done. As you suggest, attempting it with both others gave me an error (eg): ../meson.build:39:0: ERROR: Dependency "fltk" not found, tried pkgconfig and cmake >- You could enable LTO for a free small performance/filesize/memory usage improvement. Done. >- the add_project_arguments() call should probably be removed, Yep, it will be on any production-release. It's there ATM simply to give me quick-access during dev work. >- `meson setup build && meson compile -C build` is the modern way to build a project in Meson. Both done. >- That whole block of comments at the tail of the file should probably be moved to a README.md. Done. >srcs_all should probably be a library and then the library should be depended on by the commands, possibly the commands could be split into separate projects, and any parts of srcs_all that are only used by one project could go into the relevant project. Probably a good idea, but tbh I don't see a clear way to approach it that won't make things harder to reason about for us ATM. Any specific concrete examples to offer Nagisa? >Also, from skimming the C++, I see that you unnecessarily add newlines to puts() calls. Actually, every one of the 'extra' newlines is intentional; for output display formatting. >Additionally, the proper way to print a newline is putchar('\n') not puts(""). Done. >>16427 >There are many of those git hosting sites. gitlab, gitgud, bitbucket, sr.ht, one of them is bound to be good enough. No doubt, Kywy has chosen Gitlab (and I have an account there too). >Version control and some medium for collaborating are important. I use both: -Git -/robowaifu/ Won't you please consider taking my suggestion for your own repo? Just use my latest drop ITT, make your own repo, then I can both send you PR for any new work I have, you'll have strict control to do w/e you know to be better approaches, and I can clone your work and diff them to learn step-by-step at my own pace. >=== -fix missing crosslink -edit LTO resp cmnt -edit 'putchar('\n')' resp cmnt

Hello, newfag here. I have been lurking here for a bit. I am new to programming and C++ in general, so an experienced programmer would probably get a stroke looking at my code, but it seems to work and that is all I am looking for at this point. Today I finally got back propagation working on neural network library in C++ that I was working on. So far I have created a CSV file parsing, as well as matrix operations library using std::vector. The neural nets use matrix algebra to compute the outputs. It can initialize a random neural network with weight between -1 and 1 using 2 vectors, 1st to determine how many neurons in each layer, 2nd to determine the type of neuron in each layer(different activation functions), as well as Nx the number of input variables. There are also functions to scale input and output data between 0 and 1 for faster input. It I have finally got the 1st order training method of Gradient descent to work, with OK results, the training dataset isn't too large and IDK if I am using the correct network layouts, but the outputs seem somewhat on the mark. I need to learn a lot more about layouts of neural nets. The next step is a lot more testing, and adding the more efficient 2nd order methods, which will involve computing the hessian and jacobian matrices. (RIP my brain) After I get these 2 methods working, and then get into actual chatbot learning to utilize these neuralnets for my waifubot. I will post updates if that is OK. My test folder: https://anonfiles.com/[redacted]/test.tar_gz === Note: It's bad form generally (and especially on IBs) to include anonymous precompiled binary executables inside a drop, Anon. Give it a shot making another drop that's just the source code, build files, and dataset files. Please post that instead, and I'll fix up your post here with the new droplink. Cheers. >t. Chobitsu >=== -add/edit admonition cmnt

>>16408 Sorry, but it's better if you create the repository, you're the author, and I'm not looking for more responsibilities.

>>16451 Understood. Well, as I indicated ATM /robowaifu/ is my 'repository'. It certainly affords us a reasonable degree of communication. If you can find some way to dist + push patches up to catbox or some other file service, I'm sure we can get by.

>>16455 Should be doable with any of a git hook, gitlab action, or meson target. I've done tangentially related stuff but not this in particular. Either way, I'm already working with 2 dependencies /robowaifu/ uses to fix a bug and add a way to use Meson's dependency() so I already found something to work on.

>>16489 Excellent. Thanks Nagisa, your help will be very welcome! Cheers.

Nagisa, I'd like you to have a look at this tool please. If you think it worth it, then let's formalize a plan to regularly apply it to our code. The Verified Software Toolchain Project https://vst.cs.princeton.edu/ Cheers.

So, we're going to make the drop a bit early this time. It's going to be a lot more convenient doing so early, so we carved out the time to do so. note: We also have a metric boatload of AFK-life concerns that can no longer be ignored. So, we're going to have to put this project work on hold. This is going to be our last Caturday drop for this month at least. Hopefully, we'll be able to pick things back up sometime early in July, but even then we'll still have to see how things go. Please be patient Anon. BTW if someone wants to make a 'Pull Request', just push it somewhere and link it ITT. Stay encouraged and work hard Anon. Together we'll all get there! :^)

Skater catgrill Meidos IS NOT A CRIME! :^) note: sadly, we've scraped our barrel of catgrill meidos, so time to move on to just the meidos I suppose provide neko ears and tail using your imagination please Anon For this drop we've moved over to libcurl alone. Curl really is the single best open-sauce networking library out there I'd say, hands-down. Beautiful C code, and it has a cool origins story. It's used in many billions of devices today; it certainly has industrial-scale use. It's clearly our safest bet for networking IMO. === From last drop, you remember we had a callback function. Now let's have a look at the caller function that uses that. First, we'll look at everything all together at once and then we'll break it all down one step at a time. >dolldrop.cpp snippet /** Assign curl easy options for a new @c curl handle, adding it into a @c * curl-multi handle * * @param[out] multi_hdl the multi hdl that receives the new curl easy hdl * @param[in] uri the associated uri for this easy hdl * @param[in] hdr the target string to receive header data */ void cfg_hdr_hdl(CURLM* const multi_hdl, std::string const& uri, std::string const& hdr) { auto curl_hdl = curl_easy_init(); curl_easy_setopt(curl_hdl, CURLOPT_WRITEFUNCTION, wrt_str_cb); curl_easy_setopt(curl_hdl, CURLOPT_NOBODY, 1L); curl_easy_setopt(curl_hdl, CURLOPT_HEADERDATA, &hdr); curl_easy_setopt(curl_hdl, CURLOPT_USERAGENT, "Bumpmaster_v220604"); curl_easy_setopt(curl_hdl, CURLOPT_URL, uri.c_str()); curl_easy_setopt(curl_hdl, CURLOPT_FOLLOWLOCATION, 1L); curl_multi_add_handle(multi_hdl, curl_hdl); } That first parm is a curl multi handle. From the docs[a]: >"With a multi handle and the multi interface you can do several simultaneous transfers in parallel. Each single transfer is built up around an easy handle. You create all the easy handles you need, and setup the appropriate options for each easy handle using curl_easy_setopt." So, this function is called repeatedly from within a loop at the caller, and a brand new curl easy handle[b] is created here each time through, it's options are set[c], and it is then added (attached) to the multi_hdl parm. This is all just preliminary configuration, and once that loop in the caller completes then the downloads proper begin in parallel. So, the function's statements in order: auto curl_hdl = curl_easy_init(); -create a curl easy object[1] curl_easy_setopt(curl_hdl, CURLOPT_WRITEFUNCTION, wrt_str_cb); -specify our previous callback code[2] curl_easy_setopt(curl_hdl, CURLOPT_NOBODY, 1L); -don't download the request's body (just header)[3] curl_easy_setopt(curl_hdl, CURLOPT_HEADERDATA, &hdr); -this is the string pointer to use at callback[4] curl_easy_setopt(curl_hdl, CURLOPT_USERAGENT, "Bumpmaster_v220604"); -Set the user-agent. Servers can see this and know what software is making the request. The version just tracks the RW Foundations library.[5] curl_easy_setopt(curl_hdl, CURLOPT_URL, uri.c_str()); -URI to use for this transfer[6] curl_easy_setopt(curl_hdl, CURLOPT_FOLLOWLOCATION, 1L); -follow any redirects the server provides[7] curl_multi_add_handle(multi_hdl, curl_hdl); -finally, attach the easy handle into the multi handle[8] That's it, we've walked down the entire code listing Anon. We hope it all makes sense what's going on here. >tl;dr A) create a curl easy object B) set it's options C) add it to a curl multi object D) lather, rinse, repeat We also began a new sublibrary for RW Foundations this time: RW Intuit. As Anon pointed out (>>16515), a robowaifu's intuition is something that just comes 'naturally' to her, and operates very fast. No 'thinking things through'. You can expect that when it's time comes, lots and lots of focus on time/space efficiencies will be invested in this library tbh. Well, I hope things are going well for you Anon. Keep the prize in mind, pursue your robowaifu dreams earnestly and with vigor. Onward noble Knight! :^) Until next time then, cheers. === >version.log v.220604 --------------- -begin Intuit; rw::atashi::cognate -add wrt_strm_cb(), cfg_body_hdl() utility funcs; dolldrop -add pull_bodies_multi_g(); dolldrop -add pull_anon_body() call-chain; bumpmaster, dolldrop -mv curl_global_init() call into preflight(); dolldrop -add Json empty check in thrd_posts(); bumpmaster -add missing function stamp() calls; bumpmaster, dolldrop -rm all curlcpp-dependent functions; bumpmaster, dolldrop, dollnet -rm all curlcpp lib calls; bumpmaster, dolldrop, dollnet -fully rm curlcpp library dependency; meson.build >rw_foundations-v220604.tar.xz.sha256sum acab50c8cb3966fb6aeaf6fa79c472114728bea7fe4cd0e6b7fb06fac8984833 *rw_foundations-v220604.tar.xz >backup drop https://files.catbox.moe/y1ji9q.7z === 1. curl_easy_init - Start a libcurl easy session https://curl.se/libcurl/c/curl_easy_init.html 2. CURLOPT_WRITEFUNCTION - callback for writing received data https://curl.se/libcurl/c/CURLOPT_WRITEFUNCTION.html 3. CURLOPT_NOBODY - do the download request without getting the body https://curl.se/libcurl/c/CURLOPT_NOBODY.html 4. CURLOPT_HEADERDATA - pointer to pass to header callback https://curl.se/libcurl/c/CURLOPT_HEADERDATA.html 5. CURLOPT_USERAGENT - HTTP user-agent header https://curl.se/libcurl/c/CURLOPT_USERAGENT.html 6. CURLOPT_URL - URL for this transfer https://curl.se/libcurl/c/CURLOPT_URL.html 7. CURLOPT_FOLLOWLOCATION - follow HTTP 3xx redirects https://curl.se/libcurl/c/CURLOPT_FOLLOWLOCATION.html 8. curl_multi_add_handle - add an easy handle to a multi session https://curl.se/libcurl/c/curl_multi_add_handle.html a. libcurl-multi - how to use the multi interface https://curl.se/libcurl/c/libcurl-multi.html b. libcurl-easy - easy interface overview https://curl.se/libcurl/c/libcurl-easy.html c. curl_easy_setopt - set options for a curl easy handle https://curl.se/libcurl/c/curl_easy_setopt.html

>>16543 Skater Doragon Meidos IS A JUSTICE! Neko's are a delight but, there are other monsters that can bring ones heart light. :-^) Moving to curl is a smart idea. Networking is going to be a major part of our waifus. Onward we shall go

>>16548 >Skater Doragon Meidos IS A JUSTICE! This!11 :^)

>>16509 Try LLVM's static-build project, GCC's -fanalyzer flag, and cppcheck, those are some good linters. Also always make programs portable and write tests. I also found cparser's compiler warnings quite good when I used it, Meson doesn't support cparser though. Formal verification makes sense but it's just not practical or useful at the moment. For instance, CompCert still depends on the GCC preprocessor and the system's linker, it's only a relatively small component of a complete system. There are easier things with greater impact to work towards. >>16543 I see extern/json.hpp is copypasted from elsewhere. Copypasting dependencies is generally a bad idea, often they accumulate local modifications until they're incompatible and irreconcilable with the original, or the original moves on and the local version stays. It's also more work than just counting on the system to have a reasonably recent version. This guy has an objective comparison between JSON implementations: https://github.com/miloyip/nativejson-benchmark Last time I went through that list I decided on Jansson as it does well (but not the best) in every metric and it's also portable and widely packaged. Though I only use JSON when I'm forced to, otherwise I go with S-expressions or an original binary format.

>>16763 Thanks Nagisa! I'll try to take all your recommendations into consideration sometime soon. Much appreciated, Anon.

Meido ownership as lifestyle choice--taking things higher, Anon! :^) Not much new to talk about for this drop, tbh. Really just to step out and get back into the game for the project. A few little odds & ends here & there to tighten things up in the code just a bit but nothing new externally. Probably not much more to add to RW Pageone, seems most of the basics have been covered sufficiently well now to begin thinking more seriously about Bumpmaster to come. We may go straight to a crude GUI approximation soon, rather than grind things out on the backend much further atm. Who knows, we'll see. Anyway please stay encouraged and keep moving forward Anon! :^) Cheers. === >version.log v.221022 --------------- -add 'thrd_lim' reserve for pg_1s_list, top_bump_thrds(); bumpmaster -add rd_thrd_posts(); bumpmaster -add 'cat_json' empty check, top_bump_thrds(); bumpmaster -ren top_bump_thrds(), chk_pull_cats; bumpmaster -add curl_pull_list() call chain; dolldrop/dollnet -add curl_global_cleanup() call in dtor; dolldrop -add specified dtor; dolldrop >rw_foundations-v221022.tar.xz.sha256sum 990e031c923f9656b78aedde6d94882d766c7415055da7868d65ea186d053c27 *rw_foundations-v221022.tar.xz >backup drop https://files.catbox.moe/z0wwav.7z

Longtime lurker, only occasional poster here. I've read through this whole thread again and I still don't understand it. What is this? The OP states it's code for working with single board computers and shit but I don't see them mentioned again. What is bumpmaster? Imageboard software? Is this that archival project you were working on? What is Sumomo's World? Some kind of 2D waifu simulator/game? Is it playable/usable? Screenshots? Whatever happened to that? What is an arigato-class robowaifu? Does this exist? I don't believe I've seen it referenced anywhere else before. >>15723 >that code box What even is this? What is a Pandora? Is this a project from another thread on this board? Dollhouse? Dolldrop? Dollnet? >Just download and build the shit No thanks. I'm struggling to understand what any of the stuff in this thread is even at the most superficial level. You got a readme somewhere? Screenshots? Videos? Help me out here.

>>17585 LOL. That's a lot of questions for a smol post Anon! Welcome BTW. :^) >tl;dr RW Foundations is a general & (hopefully mostly-portable & generic) systems-level framework for robowaifus. >I don't believe I've seen it referenced anywhere else before The closest thing to see the idea is the fictional full-sized Persocoms in the mango/animu Chobits by the studio CLAMP. >No thanks... I'd suggest simply hiding the thread rather than scratching your head over it. You would certainly need to be able to build software for the code to mean much for you at this stage. It's a pretty expansive set of concepts, primaririly just a 'software sketch' at this stage, and really meant as a years-long WIP (set of) thread(s) as we all work towards real-life robowaifus. >ttl;dr It's a big job ahead, this is a feeble step towards that goal. :^)

>>17585 Using more diagrams as indicated in this thread here >>4143 would help. Using plantUML, Mermaid, or Obsidian. This problem will get bigger and bigger without a onsite search option and without diagrams and summaries in the threads. So anyone working through some thread (again) should maybe consider to make notes and try to make them in a format which can be used to make at least some simple diagram(s). The main devs of projects are also well advised to create some overview. It might look all obvious to themselves, but like a mess to anyone else.

>>19376 Yes, point well-taken Anon. This project is actually going to be a yuge project in the end, so such aids to understanding for a team will be vital. I'll try to take a swipe at making some relational diagrams at the least when I get back into this later this year (probably following Spring Break timeframe).

>>14714 >Sumomo, find great bathing software for us please! :^) > (post-related : >>26446)

>>33732 I'll answer your question here in 'the horse's mouth', CyberPonk. :^) <---> >Thanks, Chobitsu. I appreciate seeing your thoughts, even when it's "just" encouragement. Y/w. :^) >It's also nice having someone around that's been at this for as long as I have (since 2014). Yep, lol. We started these conversations before the 2nd migration happened in (Sept, I think it was?) 2014. At that time I departed 4cuck for Freddit's place. I still engaged on /g/ for another couple of years to try helping establish WaifuBot Generals. /pol/ and /b/ weren't really about solving anything regarding robowaifus, so yeah. It is funny that we /b/tards actually outlasted Luggage Lad though. :D >I'm glad the tokenization philosophizing helped. Very much. Thanks! :^) >Are you posting your progress on RW Foundations anywhere? Yes, my intention eventually became (and still basically is) to create Sumomo-chan as the first full-integration systems engineering test suite for robowaifus in general. Cost was the basic driving force (cf. dear Kibo-chan & Poppy-project). Square-cube law and all that. But the reality is that most all of the needs for a full-blown robowaifu also apply to a Fairy-bot -tier one, such as Sumomo. (cf. the Chobits series for more info on the similarities/differences.) So, as I embarked on thinking that through, most all of the systems architecture work I had pressed through now had a natural outlet. Thus was born RW Foundations, intended as an overarching systems library suite with a common architecture and design approach. Initially it was just a 'whiteboard' where I could throw out new (& needed) design & architecture ideas, all under the same umbrella approach. I knew I wanted to rewrite BUMP into the new Bumpmaster, but in exploring a new approach (and teaching myself how to program directly against the cURL library itself (to remove a wonky dependency in BUMP) I devised a smol utility testbed called Pageone. It's proved so useful to me personally that I've been using it ever since! :^) In systems engineering it's important to think through how everything must work together at the same time. This is where it touches on systems architecture. I had two fundamental driving agendas for our robowaifu's systems architecture: 1. That any class could include any other class. Normally you have to use convoluted schemes to keep things working as a DAG of dependencies (and prevent inevitable segfaults otherwise). But I managed at long last to find a fairly simple & elegant scheme of baseclass indirection to make it all work. Problem #1 solved. :^) 2. That everything would all work from the same set of design & interface mechanisms. Normally a big, sprawling set of libraries are developed in ad-hoc ways by differing teams, and then (different) system architects have to come through and try to stitch all that together into some kind of a cohesive system framework. The costs of such a """normal""" approach are far too much for a smol team (such as ours) to bear. My solution is a simple one (and made quite easy by using Mesonbuild's config system): Just write everything under the same code base, using the same design template. So far, I think this simple idea will work out, and if so then innumerable headaches/costs will be avoided. Problem #2 solved-ish. :^)

>>33733 Your approach is very much in-line with what popular neural network libraries do, where everything is put under the same codebase. In that case, everything is derived from a small number of base classes: tensor (data), function (operation), and module (workflows). - A DAG is created over function calls to represent what processing needs to be done. - Modules are used to encapsulate collections of related workflows. - Tensors represent data as it goes through these workflows. If you translate my 200 line autodiff implementation to C++, I think you'll have basically the full implementation for this. https://github.com/synthbot-anon/horsona/blob/main/src/horsona/autodiff/basic.py On the server infrastructure side, people use a different model to scale up complexity. - Functions are provided through REST interfaces. - Resources are represented by operators. The operators support create/read/update/delete operations on configuration files, which lead to updates on the underlying resources. - Data can be passed around either directly through REST interfaces (stateless data for a specific recipient), through message streams (stateless data for an unspecified recipient), through databases (stateful, queryable data), through S3 objects (stateful, non-queriable data), depending on what's appropriate. In terms of complexity, both approaches scale very well. The neural network approach works better when you need to take advantage of the computation graph. The server approach works better when you have complex dependencies. I'm trying to unify both approaches in my chatbot library, though that's still a work in progress.

>>33736 POTD Thanks CyberPonk! Great information. I also appreciate the tacit encouragement. This has been a long slog to get this far, lol! :D >If you translate my 200 line autodiff implementation to C++, I think you'll have basically the full implementation for this. I'll take that into consideration. I've a lot on my plate rn, so I'm unsure how much time I can dedicate to unraveling your sophisticated horse code (and all the precursor library dependencies it relies on). But I'll plan on taking a whack at getting your system up and running sometime during the next few months and trying to become proficient with it's basic usage. Hopefully I can learn enough through that experience to become at least a reasonably-competent mid with the frameworks thereafter. >On the server infrastructure side, people use a different model to scale up complexity. > stateless processing... < AKA Functional Programming One of the really nice improvements to C++ over the past few years is a solid embrace of functional programming (since C++17). With further refinements to std::ranges, this has gotten even better (and that trend will continue with C++26 Senders [1], et al). <---> Here's one of the best examples I've seen regarding the benefits of it; in a non-trivial example from Niebler himself: https://www.youtube.com/watch?v=mFUXNMfaciE https://github.com/ericniebler/range-v3/blob/master/example/calendar.cpp < no state < no loops < no branches OK, maybe one if() :D < correct by construction Not a lot of places for bugs to hide in such code, tbh. :^) Its C++ -heavy; but if someone can slog through that, then this is a really good example of the basic ideas behind functional -- within a realworld, high-performance practical example. >note: I actually built this as an in-depth, step-by-step working training example for our original /robowaifu/ bunker board back on the OG Fatchan, and it runs fast. < smol executable; completely portable codebase < almost entirely lazily-evaluated < blithely strides through infinite data generation w/ zero add'l runtime-overhead costs --- 1. ( >>32610, >>32632 ) >=== -add 'code review' notes -fmt, minor edit -add crosslinks ref

>>33737 You don't have to sell me on functional programming ^:). State is necessary sometimes though, like when updating parameters or memory. Best practice with the NN approach is to: - Make sure the actual functions take state as input so they're all stateless. - Make sure state is isolated and encapsulated by modules. When used, the modules transparently add state as inputs to the functions. Modules also support exporting & importing state so everything can be rerun easily. On the server infrastructure side, best practice is to isolate stateful components into their own components, like a database, Redis cache, S3 storage, or config files. Code that uses stateful components is certainly harder to debug than code that doesn't, but state issues are at least isolated so it's easy to identify when they're responsible for bugs. Journaling, backups, and logs make debugging feasible, but still more difficult than stateless functions. By the way, node-based UIs seem to be the gold standard for working with computation graphs. It's what Blender uses, and it's what people use for complex image generation workflows. https://github.com/comfyanonymous/ComfyUI These give a pretty direct representation of computation graphs, and they're pretty intuitive to work with once you know what all of the components & parameters do. >and all the precursor library dependencies it relies on That file at least should have no complex dependencies. I use pydantic to make sure there's an easy way to convert datastructures into some printable representation for LLMs. It does make use of python decorators though. horsefunction is a python decorator. Decorators take a function as input, transform it somehow, and return a function as output. I think decorators can be implemented in C++. https://refactoring.guru/design-patterns/decorator/cpp/example It uses generators too, which I think are the same as C++ coroutines. Functions (horsefunctions) are implemented as generators/coroutines. This is because they all need to be "called twice". The first time is for getting the actual function return value. The second time is for backpropagating feedback/updates/gradients. Other than pydantic, most of the imports in that file are for making sure developers can have useful type hints when working with variables ("tensors"), functions, and modules. You wouldn't need these in C++. AsyncIO is for parallelism, and I think C++ has equivalent standard library functions & keywords to support that.

>>33741 >You don't have to sell me on functional programming ^:). Excellent! I wish I had developed my mindset around functional. My program Waifusearch is a perfect example of code that could be entirely stateless (and all the perf benefits in time, space, and parallelism that implies), yet in my typical currently -- hopefully soon to be improved-upon! :) fashion, my design pattern seems to be: 1. Create state 2. Process state 3. Lather, rinse, repeat. :P Since Senders has me so stoked currently, I think it will finally help get me over the brain-hump, and finally swallow the functional-pill for reals! :D <---> OK CyberPonk, thanks for the explanations. Yes, it sounds like this could probably be translated without an inordinate amount of effort. In my case, the reason for doing so is obvious. But I'm curious if you think it would benefit RW Foundations itself? If so, how would you imagine that process working? BTW, the Atashi library is about individual cognition/metacognition, so I'd presume such code would 'fit in' as sublibrary(s) in that tree. <---> >node-based UIs rule YES!! I absolutely love this stuff. I fell in love with it for doing compositing work (as in Nuke, et al), and have relished the opportunity to work with it ever since. Indeed, while I want to provide a Pythonic API for all user-facing robowaifu functions from RW Foundations, my desired/preferred method is in fact a node-based programming approach. Something that even kids/non-English speakers could master in a couple night's effort. >ComfyUI link Thanks! >GoF examples Again, thanks! I almost never open that book lol, it's a write-only artifact on my bookshelf. :^) >and I think C++ has equivalent standard library functions & keywords to support that. Yes, and none will be better than Senders. I really think it may prove to be the best-of-class once it's gone through a couple rounds in the Standards Committee. Thanks again for another informative post, Anon. Cheers. :^) What a time to be alive!

>>33742 >But I'm curious if you think it would benefit RW Foundations itself? I think it provides a framework for what you're looking for, but you know what you're looking for better than I do. The main benefits are: - It lets you create functions & modules in a way that lets them piece together both forward and backward. - It supports both functional programming (using only "functions") and stateful programming (using modules). - I think it makes it easy to create a node-based UI just by exposing each function as a node. - The general approach is well-tested with neural network libraries for building complex applications, though my specific implementation is not. I don't know everything you're looking for in RW Foundations though, so it's hard for me to say whether it would fit or how exactly it would fit.

>>33743 >The main benefits are: > ... Oh, that's really sophisticated then, CyberPonk. Well such code would go into the rw::sys namespace instead, since that would be a systems-wide type of facility. An Anon here recently wanted to hack together a regex parser in C++, and we worked together on building him a container of function objects -- with uniquely-specified parameters for each -- so that he could have a concurrent model to parse the input text. This same approach (using unnamed lambdas as function objects, each with unique parameters) can work as a functional, direct approach to assembling loosely-coupled computation chains. In many cases, they could also be entirely independent of one another state-wise, and so could be executed in parallel. And by using C++ Senders, they would be asynchronous and lazily-evaluated. I'm guessing that combination would get us 90% of the way towards your 3-tier computation graph model? <---> >I don't know everything you're looking for in RW Foundations though, so it's hard for me to say whether it would fit or how exactly it would fit. Heh, that a very, very, Very big question...with one simple answer: EVERYTHING! :DD If it involves software in any way, shape, or form, then RW Foundations is intended to serve that purpose for Model A robowaifus (and following). Such an incredibly-complex potential matrix of component- interactions/operations is why I insisted that our software systems architecture support All-to-All member-field class inclusions (ie, any class can include any other class directly). It's also why I devised a basic Master/slave intrinsic access-control system in RW Foundations. I know it's a bit grandiose on my part but no moreso than this entire enterprise already is, lol. But yeah, that's the actual answer. RW Foundations is intended to unify all robowaifu software needs under the same giant big tent (including ancillary systems such as her RW Dollhouse 'home/charging-station', doggo-style mobility bases, tablet/phone interfaces, &tc.) >=== -fmt, minor edit

>>33747 >Heh, that a very, very, Very big question...with one simple answer: EVERYTHING! :DD That's fair. I should have asked: what's known or believed about the "right way" to build robowaifus, and what constraints does that impose on how they're developed? In my case, I need backpropagation, which necessitates keeping track of a computation graph, which necessitates the Variable-Function-Module style of development. I believe I'll eventually need asynchronous distributed systems for data & configuration management, which necessitates redundancy, replicability, and recovery, which necessitate something like a Streams-Databases-Controllers style of development. For synchronous distributed systems, which I don't see a need for right now, I'd add REST to the list, which necessitates serializable interfaces & stateless implementations. There are other more generic requirements, but any modern programming language should be able to handle those, so I wouldn't need to implement anything to handle them. It could also be that different aspects of robowaifus should be built in different ways, but I think it's still worth listing them out. It might be worthwhile to have different "core" modules for each required style of development. Cognition might require backpropagation, data & configuration management might require redundancy-replicability-recovery, robotic control might require something analogous to what Unreal Engine does (whatever that is). (I'm going to be out for the next few weeks for my annual horse cult affairs. I'll probably check in at nights, but I'll be slow to respond, and I don't expect to make any dev progress until that's over.)

>That's fair. I should have asked: what's known or believed about the "right way" to build robowaifus, and what constraints does that impose on how they're developed? Heh, I'm pretty sure that we're all just making up the rules as we go along here, Anon. Back in Orville & Wilbur's day, I believe they used to just call that "flying by the seat of your pants". :^) >tl;dr I guess we'll know most 'requirements & constraints' once we've gotten through delivering the first Model A robowaifu kits. For a brief synopsis of some domains of interest involved with a good outcome, you might refer to this post to Anon : ( >>33755 ). >In my case, I need backpropagation, which necessitates keeping track of a computation graph, which necessitates the Variable-Function-Module style of development. I believe I'll eventually need asynchronous distributed systems for data & configuration management, which necessitates redundancy, replicability, and recovery, which necessitate something like a Streams-Databases-Controllers style of development. For synchronous distributed systems, which I don't see a need for right now, I'd add REST to the list, which necessitates serializable interfaces & stateless implementations. There are other more generic requirements, but any modern programming language should be able to handle those, so I wouldn't need to implement anything to handle them. Sound amazing, actually. I wish you good success with each of your requirements! I actually plan to integrate some REST capabilities in RW Foundations, primarily to simplify messaging to the robowaifu's Master/support home networks for offloading computation on-premises. As you might imagine, safety & security are important topics for any such arrangement. >It could also be that different aspects of robowaifus should be built in different ways, but I think it's still worth listing them out. It might be worthwhile to have different "core" modules for each required style of development. Cognition might require backpropagation, data & configuration management might require redundancy-replicability-recovery, robotic control might require something analogous to what Unreal Engine does (whatever that is). Certainly the heterogenous development approach has been the norm for almost all big systems of today, and perhaps it will be so in this case. While the big tent of C++ code is the ideal as a target design-goal, the realworld will likely necessitate other software somehow architected to work together with RW Foundations during the next several years, I imagine. Hard- and soft-realtime requirements are the big performance constraint, however, for a realworld robowaifu living at home with her Master. Most other approaches will likely not match up to that need (and integration with foreign systems only exacerbates this problem significantly). C++ can & will meet that need. I'm not pretending that this is the one perfect, best way (it's not of course, since no language is perfect) -- but in my studied opinion this is as close as we all can get ATM, all things being considered. Blender is probably one of the best FOSS concept-proof examples of this today: a huge, highly-complex, high-performance system incrementally developed over year's time as C => C++ , even while integrating other external libraries (also almost always written in C and/or C++). So, our ideal here can be done. :^) >(I'm going to be out for the next few weeks for my annual horse cult affairs. I'll probably check in at nights, but I'll be slow to respond, and I don't expect to make any dev progress until that's over.) Haha, have fun CyberPonk! Give my love to Derpy o.O :D Stay safe and look forward to hearing from you when you get back. Cheers, Anon. :^) >=== -minor edit

>>33760 That's a good list, and as you suggested, Blender does a lot of relevant things to support high performance, hard and soft realtime requirements, and heterogeneous development. Blender is somewhat-well documented too. On that chance that you haven't seen it, Blender's design docs https://developer.blender.org/docs/features/ might be useful for anticipating solutions to a lot of problems and for figuring out what design patterns are worth investigating. There's a lot of good information there.

>>33843 >Blender does a lot of relevant things to support high performance, hard and soft realtime requirements, and heterogeneous development. Not sure what you mean about realtime in Blender's case, but otherwise fair enough. It's a remarkable system today! :^) >Blender's design docs I've seen these in the past, but since I stopped actively building Blender 2-3 years ago, I kind of let it slip my mind. So thanks for the reminder. I personally like Blender's documentation efforts, though I've heard some disagree. Not-uncommonly, this is one of those tasks that get pushed to the 'back burner', and is often left to volunteer work to accomplish. Given the breadth & scope of the platform, I'd say the Blender Foundation has done a yeoman's job at the doco work, overall. Very passable. <---> Also, reading that link reminded me of USD. NVIDIA is currently offering developers their version of free training on this topic, and I've been pondering if I can make the time to attend. A huge amount of the DCC industry has come together to cooperate on Pixar's little baby, and today it's a big, sprawling system. Why it's of interest to us here is that most of what a robowaifu will need to do to analyze and construct models of her 'world' is already accounted for inside this system. While there are plenty of other (often higher-speed) ways to accomplish the same (or nearly the same) tasks, the fact that USD has become such a juggernaut, with a highly-regimented approach to scene descriptions, and with such broad approval, improves the likelihood IMO that other Anons from the film & related industries may in fact be able to help us here once they discover robowaifus in the future -- if we're already using USD to describe her world and the things within it. I hope all that made sense, Anon. https://openusd.org/release/glossary.html# >=== -fmt, prose edit

>>33845 >Not sure what you mean about realtime in Blender's case This page looks relevant: https://developer.blender.org/docs/features/cycles/render_scheduling/ Blender does progressive rendering, which starts by rendering low-resolution frames. If there's extra time left over before a frame needs to be rendered, it generates more samples to generate a higher-resolution frame. The equivalent for video generation at a fixed framerate would be running a small number of denoising steps for the next frame, and running additional denoising steps if the next frame doesn't need to be rendered yet. For text generation at a fixed token rate, it would be equivalent to doing speculative decoding for the initial response, then using (maybe progressively) larger models if the next token doesn't need to be output yet. For a cognitive architecture with a fixed response rate, I think the equivalent would be generating an initial response, then continually refining the response based on self-evaluations & feedback from other modules until the the response needs to be output. >USD Very nice. I hadn't heard of this. It looks like a goldmine of information. Your explanation does make sense, and it's a great example of the sort of design patterns that I expect would be useful, in this case for modeling the environment & context.

>>33850 OK good point, CyberPonk. Such UX optimizations can fairly be said to be in the domain of soft-realtime. And certainly, integrating GPU processing code into the system to speed the rendering processes of Cycles & EEVEE has had major positive impacts. I personally think the fact that Ton chose to create the entire GUI for Blender in OpenGL all those years ago has had many far-reaching effects, not the least of which is general responsiveness of the system overall (especially as it has rapidly grown in complexity over the last few years). <---> >It looks like a goldmine of information Glad you like it! It's fairly easy to overlook that describing a scene is in fact a very-complex, nuanced, and -- I'm going to say it -- human undertaking. And when you consider that task from the deeply-technical aspect that USD (and we here) need to accommodate, then you wind up with quite a myriad of seeming-odd-juxtapositions. Until D*sney got their claws into it, Pixar was a one-of-a-kind studio, and well up to such a complicated engineering effort. I doubt they could do it as well today. If at all. DEI DIE to the rescue!111!!ONE! :D Cheers, Anon. :^) >=== -fmt, minor, funpost edit

>>33857 I looked up that USD. "USD stands for “Universal Scene Description”". I hadn't heard of it. Wow, that's some super comprehensive library and format. Hats off to pixar for open sourcing this.

>>34201 >Hats off to pixar for open sourcing this. Well, it's a vested-interest, but yeah; you're absolutely correct Grommet. Sadly, I'm sure they couldn't even pull it off today; they've become quite afflicted with the incompetency crisis. >protip: competency doesn't cause a crisis, only incompetency does. :^)