/tech/ - Technology and Computing

Technology, computing, and related topics (like anime)

Server and LynxChan upgrade done. Canary update and a general address soon. -r

Max message length: 6144

Drag files to upload or
click here to select them

Maximum 5 files / Maximum size: 20.00 MB

More

(used to delete files and postings)


Project Thread Anonymous 11/05/2019 (Tue) 07:26:28 No.610
ITT: Post cool your (or other anon's) Projects here
Rules: Drop links to the repos and community chat/forum, and add a description as to what it does

Here, I will start.
https://hydrusnetwork.github.io/hydrus/ and https://github.com/hydrusnetwork/hydrus (repo)
https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts (scripts to scraping websites)
https://gitgud.io/hydrus and https://gitgud.io/groups/hydrus/-/shared (add-ons and such)
https://github.com/mserajnik/hyve and https://github.com/floogulinc/hydrus-web (webUI)
https://github.com/git4unrealnondev/Hydrus-App (mobile app that is an original)
https://github.com/glael/nori-with-hydrus (mobile app adopted from a dead project)
https://github.com/git4unrealnondev/Hydrus-Updater (updater for linux and Mac)

What does it do?
1. Download images and files with tags from imageboads, boorus, art galleries, blogs and other sites
(Everything from ExHentai to YiffParty are covered, if you want more coverage just ask)
2. Sort images and files using tags, with a whole publicly shared tag database (PTR) for booru-tier collabs
3. Using IPFS to share images and files peer-to-peer style (because Torrents sucks and Zeronet a honeypot)

Contacts:
https://discord.gg/vy8CUB4 (yes the dev uses discord due to popular demand)
https://endchan.net/hydrus/ (because some anons don't use Discord, and Odili reached out)
I keep saying one of these days I'm going to build a computer. I have the microarchitechture drawn up and even a list of the chips I need. I just can't really afford all of them. It would be about 100 bucks in chips, and I would still need LEDs, a wire wrapping tool, wire wrapping sockets, pc board, an enclosure, etc. I could do everything with point to point sodering, but that will take forever. I guess at this point I have more time than money anyway.
>>614
Then redirect your interest and write a software suite that satisfies the public needs like what the Hydrus people did, that way your life would be worth something... MORE.
Start asking what the /pol/ book thread people or the /a/co/ people or even what the /v/ people want, they will give you a few pointers as to what software are desired.
>>610
I wonder why Hydrus wasn't of more use after the 8ch deplatforming? If they had all the imageboard contents, couldn't they have helped in creating bunkers rapidly and helping anons find them?
>>618
Probably less than 1% of anons have so many images that they require a tag based distributed cataloging software to deal with it, there goes using it to help advertise the bunkers or webrings. As for setting up the bunkers more quickly that depends entirely on the users, look at /tech/ here. It wouldn't be any more lively if all the posts from 8chan's /tech/ were migrated over an hour after it went down.
>>618
Because (a) it is mainly used for images and webms in the *chan context, there are no text scrapers and (b) people use it as a booru so if you want to bolt in an imageboard you might as well try bolting on https://github.com/fatchan/jschan to it (help wanted).

>>619
> Probably less than 1% of anons have so many images that they require a tag based distributed cataloging software to deal with it
make it 3~5%, since we do have a lot of autism to go by, especially the drawfags, /a/co/animu/ people, the /pol/k/tech/ ebooks/webm people and such.
> As for setting up the bunkers more quickly that depends entirely on the users
It is an evacuation game, the more we develop the webring the more people from different boards can collaborate and concentrate their bunkers to prevent fracturing (Webring-based overboard/metabard would have been really sweet)

BTW if anyone wants to create a whole software suite around Hydrus, with tag dictionary/encylopedia, mobile UI, auto-archive/IPFS imageboard, ExHentai-tier tag moderation, and "social media" (newgrounds/tumblr/pixiv/deviantArt-esque) capibilities, be my guest, the more we expend this idea the better it gets.
>>623
any decently sized image archive, be it a small 1000 images or 4+million (fml) a tag base way to access images is always going to be a very useful thing.
>>623
>Webring-based overboard/metabard would have been really sweet
Do you mean inside Hydrus? The reason I ask is that the current 'The Webring' panel here on Julay, and the ones on other sites seem kinda like just that to me.
>>644
OK, I guess I take that back. The current post stats aren't exactly the same as a good overboard.
>>644
>>645
My idea is that you can use Hydrus+IPFS as the imageboard file cache while having another database to store the text of each post.
Assuming that Vichan-based, Lynxchan-based, JSchan-based (and possibly other engines) posts uses the same APIs we can allow better archives and import/exports.
>>657
>Assuming that Vichan-based, Lynxchan-based, JSchan-based (and possibly other engines) posts uses the same API
They do. It's called text.

Seriously, not just being an alog. Text is literally the universal format, and is easily adapted to multitudes of uses. It's why open text is used for formats like HTML & JSON.
>>662
But what data would be in each post or thread? Surely it can't be pure text as there are predefined formatting and data elements.
Also the bad thing about Hydrus is that you cannot have pure text as a file, everything needs a file signature (magic number) of some sort.
>>665
>surely it can't be pure text
I get your meaning anon, but I'm not too sure you get mine yet. And you don't have to ask me, have a look at this 'pure text' for yourself:
https://julay.world/tech/res/610.json

Use a json prettifier if you want to sort out an easier to read representation of exactly what data is tagged where.
>>667
I can see clearly it is formatted as JSON, and it is beautifully structured (unlike some other APIs), kudos to the devs.
BUT the issue is that I need a translator layer that converts this JSON into another format that has a file signature e.g. XML for use in Hydrus.
>>668
I'd suggest you examine the 'translator layer' already devised in the BUMP software's .sites.json file. It has standardized naming in place for all the essential tags and maps each board to those tags.
>>669
So I can basically reverse-engineer these unified JSON downloads back into XML using a standard schema?
Is that what you are trying to convey? Or did I misunderstood you? Because we all know XML is bloat but has good headers and file signatures.
>>657
I really like this idea as using locally stored images/webms/pdfs prevents the CP spam/DMCA problems IBs have had in the past. Plus it saves on bandwidth for both the IB admins and users while dealing with hosting and other legal issues.

There are probably already a dozen distributed FLOSS social media sharing style protocols and clients out there that could be adapted to this.
>>683
Here is a living example https://0xchan.net/media/0xchan_white_paper.pdf (IPFS+ETH combo)
But the major issue is whether a blockchain and smart contract based system can be rigged by anyone or not.
Another issue with non-blockchain IPFS imageboard is that many of them are "dead"
(look at https://github.com/smugdev/smugboard and https://github.com/yatima1460/Kamina ...)
>>610
I heard that Hydrus is moving from Wx to Qt, what is that all about?
>>729
You don't have to take OPs word on it anon, you can read it straight from the horse's mouth.
https://endchan.net/hydrus/res/271.html#272
>>730
How long do you think it takes for Hydrus Qt to stabilize and not have memory leaks?
>>784
Qt is a warapper for a nunch of C++, so never.
let's try that without the role signature this time.
>>786
Are you honestly suggesting Java is less prone to memory leaks anon?
>>789
Better to throw errors before things break than to have memory leaks at all, but then again Go is the new Java, and I heard Go is popular BUT have broken language features.
>>790
Actually, C is the world's number 1 source of security holes and memory leaks (and has been for 40+ years now). And yes it's certainly better to fail fast than leak resources of any kind, including memory.
>>791
I mean what you just said is a given, which is why we have Go, Java, C#, Ada, and *puke* Rust.
Also scripting languages are generally safer (I think) since the abstractions are done to prevent stupid behaviors.
Does anyone have any good projects on image manipulation?
>>955
Eh, I'm working on doing an OpenGL waifu animation system from scratch. So I guess you could call it 'image manipulation' but I suspect you mean like Krita. As to good, well..ofc it will be the world's greatest thing in the history of forever!
>>956
I was referring to image related research and analysis, but animu waifus are kinda based.
Open file (134.50 KB 997x1005 maidos14.jpg)
>>791 I contest, I submit that PHP is the world's number 1 source of security holes.
>>1190 Nah, it has to be SQL.
>>1190 post more maids riding skateboards please
>>791 > C is the most insecure language Wrap it under Nim or Crystal, maybe just use Go... and for god sake DO NOT use Rust >>1191 > SQL is the most insecure Learn 2 safe string boi https://cheatsheetseries.owasp.org/cheatsheets/SQL_Injection_Prevention_Cheat_Sheet.html >>1190 > PHP is the most insecure Exactly.
>>1192 Sure thing, but let's do it in another thread >>>/pdfs/121
Open file (180.02 KB 1039x796 wr.png)
Just realized that I could do this so I did. I made a local webring server, now I can just bookmark this instead of different imageboards. - automatically adds new servers that it finds - outlines the boards that were updated since last refresh - and allows you to favorite and hide boards - color coding of different imageboards The [c] links to catalog instead of index. I'm not sure how to highlight favorited boards, for now it's just a yellow square on the left.
It took me way too fucking long to make this: http://turned.off.li/exp/cowsaid/ Source code: http://turned.off.li/src/archives/cowsaid-1.0.tar.xz All it is a FCGI kludge that takes an input, formats and appends it to a fortune file that's then reloaded. The output of the fortune given is piped to cowsay, and that's printed into the webpage. I wrote it in C for some misguided reason that probably made a lot more sense a week ago and ended up wasting most of my time fixing seg faults because string manipulation is a pain in the ass. But it works, and I never want to see that fucking ungodly mess ever again.
>>1275 > Did it in C > Hosted in a Linux server > Everybody is bash hacking cowsay Kek
Open file (24.26 KB 480x564 akko.jpg)
>>1222 That's neat, anon. >>1276 The input never actually sees the shell, instead it just gets printed directly to the fortune file, so there's no opportunity to inject anything into the shell. I think I prevented HTML injection properly as well, but I wouldn't be surprised if something is flawed or missing in the implementation.
Open file (32.12 KB 1875x1078 Untitled.png)
>>1275 I think I broke it
Open file (247.39 KB 692x910 panzer kraben.jpg)
>>1278 I knew something was going to fuck up. After some testing it seems like the problem had something to do with malloc() and free(), so I ended up removing all the calls to free() at the end of formatCowStr() and I think that fixed it. There's still a couple other bugs, on occasion the % symbol separating individual fortunes will get clobbered so two fortunes will get fused into one, but at this point I think I'll just learn ruby or something else and re-write the entire thing.
>>1289 It seems I spoke too soon, it still randomly segfaults. I put a cronjob to restart it every so often until I manage to rewrite it.
>>1275 I remember my first FCGI project in C, I imagine it was just as much of a fucking mess as yours is. I've recently switch over to kore (https://kore.io), which has some builtins for safe (ish) string manipulation. If you insist on continuing to writing your web things in C for some reason, you should help me find or write a templating library. The ones I know of are: http://www.mindrot.org/projects/mtemplate/ https://mustache.github.io/ http://libctemplate.sourceforge.net/doc.html Which I find unsatisfactory for different reasons; libctemplate is the one I like most though.
>>1275 Please never write C again. Half the office has taken sick leave after I showed them your code and I can't sleep at night anymore. Anyway, all I could find was some segfaulting stuff. No exploitable buffer overflows, nothing to exploit in strfile or fortune or cowsay because those programs are niggerliciously simple and barely interact with the input, and no XSS possible because you strip <> and insert the text outside tags. So I'll just tell what I got for you. First, you basically never allocate space for the \0 that C strings need at the end. The one time you do it's by accident because FCGX_GetLine reads at most n-1 chars. Note that strlen gives you the length except for the \0 and strcat will helpfully discard the \0 in the middle. Second, don't assume the form data is valid. If you manually post julay instead of fortune=world, strstr(cowIn, "=") returns a null pointer and you get a guaranteed crash further on. Third, you don't filter occurrences of \n%\n so you can add multiple fortunes in one request. Not that it matters for this demo, but just sayan. Fourth, have some god damn decency when replacing characters in a string! No need to convert between pointers and offsets and have quadratic runtime: for (char* c = strchr(cowStr, '>'); c != NULL; c = strchr(c+1, '>')) *c = ')'; >>1289 Keep freeing (exactly once) each address you malloc, unless you want memory leaks. As far as I can tell you're only getting segfaults due to allocating 1 byte too few for C strings.
Suggestion for the system if you want to rewrite: fuck fortune and create a directory for generated fortunes. When someone submits a text, pass it through sanitation and cowsay and save it in the fortunes directory. When someone wants a fortune, pick a random file from the directory and embed it in the page. With old-fashioned CGI it can be as simple as: #!/bin/bash TITLE="Cowsaid | turned.off.li" CSS_PATH="/rsc/css/cow.css" FAV_PATH="/rsc/images/favicon.png" MAX_LENGTH=1024 FORTUNE_DIR=fortunes HASHER=md5sum COWSAY=cowsay cd $FORTUNE_DIR #sanitize input input=$(sed -e 's/^.*=//' -e 's/</(/g' -e 's/>/)/g' | cut -c 1-$MAX_LENGTH) if [ -z $input ] then #pick random fortune if no input fortune=$(ls | shuf -n 1) else #make fortune id for input fortune=($($HASHER <<<$input)) #if the fortune doesn't exist, save cowsaid input [ ! -f $fortune ] && $COWSAY <<<$input >$fortune fi cat <<EOF <!DOCTYPE HTML> <html> <head> <title>$TITLE</title> <link rel="stylesheet" href="$CSS_PATH"> <link rel="shortcut icon" href="$FAV_PATH"> </head> <body> <div id="center"> <pre>$(cat $fortune)</pre> <form method="post" enctype="text/plain"> Add your own:<br> <input type="submit" value="$ cowsay"> <input type="text" name="fortune" maxlength="1024" autocomplete="off"> </form> </div> <footer> As served by <a href="//$SERVER_NAME/">$SERVER_NAME</a>. This program made possible by <a href="https://linux.die.net/man/1/cowsay">Cowsay</a> </footer> </body> </html> EOF Alternate suggestion: fuck fortune, fuck cowsay, fuck all external programs. Keep the fortunes in an array in memory because fuck persistence. Have the cow in its own pre block with a small speech line leading to a textarea where you dump a random fortune and let the browser take care of line wrapping. And write the whole thing in php because fuck the world and also because it's only 3 lines. Also someone may have already written cowsay in php.
Okay can someone help a /hypnoanon/ in working out a Twine alternative? >>>/hypno/102
Open file (188.38 KB 400x400 bean megumin.png)
>>1299 >>1300 >forgot to allocate for the terminator character Ey'm a dumbass. Thanks you, anon, for your advice, and for bothering to read my shitty code in the first place.
Has anyone here tried racial detection through facial landmarks? https://onlinelibrary.wiley.com/doi/pdf/10.1002/widm.1278
>>614 This seems really interesting anon. Implementing a new architecture from scratch surely would bring a lot of knowledge.
>>>/v/14998 >https://megous.com/git/megatools/about/ >You can register mega account using the 'megatools reg' tool I'm amazed he doesn't see this as problematic with DRAGNET surveillance. >compared to using mega.nz web client. Anything done in the web browser uses a code you don't have control over (unless it's your own website), and thus can't be fully trusted with your password. I love how the author keeps proving my statement corrent. >meson b >ninja -C b >sudo ninja -C b install LOL But sweet JSON web client though, this is excellently adaptable for ActivityPub.
>>1555 Take your meds.
>>1568 Take your meds.
>>1571 Winners don't use drugs. Take your meds.
>>1572 kys and go back, brainlet. Having insight isn't the same as being crazy.
>>1574 Take your meds.
>>1584 Take your meds.
>>1586 Take your meds.
>>1590 Take your meds.
>>1593 ...take your meds.
>>>/v/17249 >>>/v/17243 >>>/v/17239 >>>/v/17233 https://en.wikipedia.org/wiki/OpenFL https://en.wikipedia.org/wiki/Extended_Vector_Animation https://opentoonz.github.io/ https://en.wikipedia.org/wiki/Synfig https://en.wikipedia.org/wiki/List_of_2D_animation_software I have a bet worth 18 billion euros none of the bluemaxima idiots will petition Adobe to sell of Flash into the public domain. Ideally, you'd translate from flash bytecode into whatever bytecode you want that public domain. Flash2{Haxe,Dart,Clojure,Scala,Vala,Kotlin,C#,F#,[etc.]} But since I'm winning my bet at this point, I'm going to concentrate on a C# time tweening engine and subplant into into engines.
Open file (256.99 KB 600x847 frr.png)
Anyone know how to browse twatter without having those non free javascript running? web archive giving me 403 server error.
>>1798 Use their API. Plenty open source scrappers online
bump
OC encouraged. Here is my idea: >Flyback transformer to produce high voltage >https://en.wikipedia.org/wiki/Pulsed_inductive_thruster to shoot it out >Plasma gun/sword done depending on plasma velocity
Edited last time by NULL_1 on 04/09/2020 (Thu) 22:20:30.

Report/Delete/Moderation Forms
Delete
Report