This is ugly. Use a long string with format! or askama templates like we do in other places.
The file spec is probably the correct one, all that is lacking is a function to serialize it directly to ansible file module json.
I was not convinced that we should use ansible for the file copy but here for the systemd unit I am sure that we want to use the ansible builtin systemd module https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/systemd_module.html
This could very well use a harmony id from harmony_types. I like the id format, it is unique, relatively short and contains a timestamp.
Why do we need that? It is yet another dependency? Can't we pass the cloud init any other way when creating the vm? Do we need that for cloud init as I'm assuming or something else?
This comment does not feeld correct. Ensure_package is distribution agnostic. For now we choose to support only debian as this is our first concrete target, but this may change soon and the encapsulation is correct, choosing the correct tool based on the distribution is this function's burden. We might move that to the topology and separate the topologies more granularly between debian an rhel and others but at this moment I think this would be wrong.
Are there alternatives to systemd unit files to make sure it restarts on reboot? What about running the agent itself as a podman container? It would probably have to be privileged but that would reduce the configuration burden on the host and centralize our logic around podman instead of spreading it to systemd.
I tend to disagree with the comment that we need to be tool specific here.
This is tricky and is actually an architectural problem in harmony. Ensure ready is executed eagerly on a topology, but this topology won't necessarily be running a kvm related workload this run and/or might be doing the kvm setup in an earlier task before calling the kvm dependent scores. I am pretty sure we have a ROADMAP entry on that topic of topology initialization dependency. Add a reference here to the roadmap entry with a TODO so we capture this use case when we get to this work.
Don't insult other projects, kubespray is cool and suits a purpose.
feels like ansible should have a purpose built module here too. Don't be lazy and call shell commands. Calling shell commands through ansible is completely useless as they're not truly idempotent in the way that file and package installations are.