I have some server issues that hard to debug while travelling, so the demos probably won't be runnable until I get back. I can still dump some code that gives the gist of how it works right now. There are two demos in this zip file, each consisting of a client and server component: https://drive.google.com/file/d/19VAIsaZP2wRxNTk2t9dNIqKk5WWYDIDL/view?usp=sharing
- simple-demo uses only the communication infra. The simple-demo server contains two important files: image-server.py and loop-config.yaml. The two other folders (loop-resources/ and loop-servers/) were auto-generated from loop-config.yaml. In the corresponding client folder, there's client.py and cyberponk-resources/. The cyberponk-resources/ folder contains select files that were copy/pasted from the server's auto-generated loop-resources/ folder.
- shared-configs-demo uses both the communication infra and the "cluster" infra. The server side contains two important file: server.py and server-config.py. The client side contains client.py and cyberponk-resource/, and cyberponk-resources/ again contains files copy/pasted from the server's loop-resources/ folder.
Both demos show how one person can request images and another can generate them. The differences are:
- simple-demo does everything ephemerally. If the server is down when the client sends a request, it'll never get seen. Similarly if a response is generated when the client is down, it'll never get seen.
- shared-configs-demo has all requests come in as "tasks". If the server is down when a client queues a task, the server will see the task when it comes up again. The responses are still ephemeral. They don't have to be, it was just a choice I made for the demo.
- The shared-configs-demo shows one approach for getting multiple people involved in generating a single image. In this demo, each image generation requests includes a "baseConfig" field. Whenever someone creates a GeneratorConfig config, anyone can use it by specifying its name in the baseConfig field. So if one anon finds good settings for generating certain kinds of images, they can create a GeneratorConfig for it, and other anons can use it just by providing the name of that config.
In both cases, multiple people can view/stream the results. So one person can pick a stream name, request that all generated images get sent to that stream, and anyone listening on that stream will get the results.
The setup process looks like this:
- On the server side, create a loop-config.yaml (or server-config.yaml). This specifies what "global" resources are required and what permissions to set on them.
- On the server side, run `python -m loopctl apply loop-config.yaml`. This creates the global resources, sets the permissions, and generate the loop-resources/ and loop-secrets/ folders. The loop-resources/ folder contains information on how to access the global resources and their last-applied configurations. The loop-secrets/ folder contains API keys. The API keys are only needed to (1) change permissions on the global resources you created, and (2) access resources if your loop-config.yaml made them restricted.
- On the server side's server.py, point the "Itl" (In-The-Loop) object to the generated loop-resources/ and loop-secrets file so it knows how to access global resources. Certain methods in the Itl object will access global resources by name. The name is whatever you provided in the loop-config.yaml file. These names are not globally unique identifiers, they're only used to look up the actual resource info from loop-resources/, which does contain globally unique identifiers.
- The client needs to access the global loop, stream, and cluster resources (depending on which demo you're looking at), so copy those into the client's folder. I put the copied files into cyberponk-resources/. When creating the client's Itl object in client.py, point it to cyberponk-resources/ so it knows how to access those resources. Otherwise, client-side development is basically the same as server-side development. There's a default "anonymous" client that's available so people can access any resources that were made available to "public".
If anyone plans to doing dev work, is interested in distributed development, and gets a chance to read through the demos, let me know how it looks. I'll post something you can actually run in about a week, once I get back to my desktop.