Git isn't something that I've used much before, so I never had any strong opinions about it.
Now that I'm using it in bulk however, and coming to learn about how it works, I've developed a newfound hatred for it.
Apparently, whenever you use git to download something (which seems to be the only fucking way to get something from huggingface, APIs aside), you get a .git folder for versioning.
Now, even I already knew this, but what I didn't know is that this folder effectively doubles the disk space footprint of whatever you're downloading right out the gate.
I guess this is good for normal use, but for bulk operations that span multiple terabytes, all it does is halve the number of models you could be storing.
I've tried researching a way to get git to not make the .git folder, but if there's a way then I can't find it.
So, I'm going to solve the issue in post.
This set of scripts is completely identical to the previous one, except each script has been appended with [rm -rf ./*/.git], which will delete every .git folder that was created by the previous commands in the scripts.
Messy, but functional, and it'll save terabytes of space.
Fucking git.