Compare and contrast the number of devices that libvirt adds.
FWIW, I strongly recommend to use libvirt to launch your QEMU-based guests. It is far more effective, and more importantly, provides security infrastructure like running the QEMU process as an unprivileged user, protecting the QEMU process (and its associated disk iamges) via the SELinux-based sVirt mechanism, and so forth.
Check out these[2] slides (recording is online, too) on "Security in QEMU" from one of the QEMU maintainers, Stefan Hajnoczi, at last year's KVM Forum in Edinburgh.
There's a lot to be said for running qemu from the command line, vs the complexity and opaqueness of libvirt. For example you can put the invocation in a script and check it into version control, and it will work on anyone's Linux box. You can easily debug the config and be suire about what options qemu was given.
Yes, there may be many switches in the invocation, but at least they are in one place, the invocation syntax rarely changes and works accross distributions, and they are applied from scratch in each invocation rather than having lots of implicit state hidden in daemons and their configurations / databases.
Your example is nicer on the eyes written in a config file style and leaving out the unnecessary options:
I can see the value in "putting the command-line in a script and launch it anywhere". Many of us use bespoke QEMU command-line scripts for test and development. And I even know a small hosting provider who manages their production guests entirely via QEMU command-line "config file" syntax (-writeconfig and -readconfig options—they're little-known, not least because they don't cover all the options; upstream has some proposals to address it).
I appreciate that libvirt may not be a suitable option for some. That said, you might want to consider the below points on where it adds value (for production VMs; not talking about test and development setups):
• It's not about having all the QEMU "switches available in one place". The dizzying array of switches make it trivial to let you shoot yourself in the shoulder (e.g. getting fiddly details like PCI device addressing right manually).
• Knowing the "best practice" command-line will get you optimal performance out of your (production) guest—libvirt handles that for you. And if the best practices change, libvirt will automatically handle that for you, as it keeps track of "QEMU command-line best practice evolution".
• Launching the command-line is only the first step. If it's a production guest, at some point you might need to live-migrate the guest, or take a live disk backup, or if your guest has a long disk image backing chain, you want to "merge" the disk images into a single, coalesced image—all of this without the guest going offline. These tasks can be done manually (assuming you launched QEMU the 'right away' upfront) via extremely tedious and error-prone ways using QEMU's JSON-based run-time interface, QMP (QEMU Machine Protocol). But that's too risky and a colossal waste of time.
• Speaking of QMP, the run-time interface, it has even more dizzying array of commands—to be more precise, as of 2015: QMP had about 126 commands + 33 events. And more than 700 named arguments and results. Try keeping track of all of that manually. It reminds me of how once a QEMU sub-maintainer memorably compared (in his 2015 talk, "QEMU interface introspection: From hacks to solutions") the volume of QMP schema to old religious books: "QMP schema is larger than "Gospel of Luke", but smaller than "Genesis". :-)
IOW, I'm yet to see anyone who launches multiple guests (on the moderate scale of hundreds of VMs) use direct QEMU in production. There are odd examples, as I noted earlier, but it's certainly not the norm.
But NixOS has a neat feature where any (?) NixOS configuration can trivially instead be built as a "vm", which will assemble an initrd for the described system along with a script which will run a tailored qemu invocation to launch the system, leaving a qcow2 file of any changes you make to the filesystem in your cwd. Makes the whole thing extremely usable, and is done in a very extensible way.
Once you add `-nodefaults`, the list of arguments grows because now you must add a bunch of things that were there by default. If by "minimal command-line" you mean "the simplest list of arguments", then your example is wrong. If by "minimal command-line" you mean "command-line that gives the minimal VM", then you're being misleading, because that's not how people will take it.
Most of the devices added by libvirt would be included by qemu automatically if you didn't say `-nodefaults`. libvirt uses `-nodefaults` and specifies its own defaults, so that it doesn't need to keep track of changes in Qemu's defaults between versions of Qemu (since libvirt cares about the nitty-gritty more than most humans do).
PS: You have too many "t"s in the "htttps://" in one of those URLs.
> Once you add `-nodefaults`, the list of arguments grows because now you must add a bunch of things that were there by default [...]
Partly true (refer below).
> If by "minimal command-line" you mean [...]
I appreciate your corrections, but please don't read too much into the word "minimal". I just quickly pasted it from one of the "subjective scripts" named `min-qemu.sh` lying around on my file system. By the time I realized, it was too late to adjust it.
> Most of the devices added by libvirt would be included by qemu automatically if you didn't say `-nodefaults`
Not quite; I just double-checked with an upstream libvirt dev: when you remove `-nodefaults`, some of the built-in devices get removed, whether those equate to "most" or not depends how many devices you've asked for.
> PS: You have too many "t"s in the "htttps://" in one of those URLs.
> like running the QEMU process as an unprivileged user,
Anyone can run your example "qemu-system-x86_64 -display none -no-user-config -nodefaults -m 2048 -device virtio-scsi-pci,id=scsi -device virtio-serial-pci -serial stdio -drive file=/export/cirros.qcow2,format=qcow2,if=virtio" from the command line/bash, as non-root user, yet libvirt which requires root to install, and is expecting root to configure it, is better? How?
Libvirt is one of those softwares which ads a layer of complexity for the sake of complexity, yet offers nothing of value in return.
its not true that libvirt adds nothing of value, it adds easy networking support, easy snapshots, a nice gui interface, a very nice remote protocol and clients (that seamlessly support both graphical and serial output, aswell as on the fly reconfiguration of VMs without restarting them), template VMs (including one time initial provision setup), easy support for storage pools including remote ones, an easy way to manipulate RAM usage on live VMs, easy PCI and USB passthrough, live migration to other hosts, etc etc.
Sure its possible to do all this without libvirt, but it comes as one integrated, easy to manage package instead of hundreds of disparate parts.
You might not need (all) these features, but to claim libvirt is useless is extremely parochial.
Its also untrue that it needs root, you can virt-manager for instance without a root libvirtd instance at all, and it will run VMs without any permissions needed.
> its not true that libvirt adds nothing of value, it adds easy networking support, easy snapshots, a nice gui interface, a very nice remote protocol and clients (that seamlessly support both graphical and serial output, aswell as on the fly reconfiguration of VMs without restarting them), template VMs (including one time initial provision setup), easy support for storage pools including remote ones, an easy way to manipulate RAM usage on live VMs, easy PCI and USB passthrough, live migration to other hosts, etc etc.
everything libvirt does can be done without it by talking to qemu directly. and yes, Ive done many of the above "by hand", it was easier to read man qemu and other qemu docs than to swallow libvirt.
No, not everything it does can be done by qemu directly, plenty of it lies outside of the scope of QEMU entirely, like managing networks (not just network devices), or storage pools, setting up permissions for device redirection, or doing the legwork for VM provisioning and configuration (including net booting and automatically running appropriate scripts), or managing access to VMs both locally and remotely.
Its fine if your use cases are covered by plain QEMU, but libvirt does a hell of a lot more than plain QEMU alone is capable of.
Again all of that is possible via other means, but when other means involve recreating half of libvirt in shell scripts or other code, id rather let somebody else do the work for me.
QEMU is designed to run virtual machines, libvirtd is designed to manage VM server infrastructure.
For one static network, what about complicated inter VM networking schemes (including firewalling) that need to be dynamically adjusted as VMs go on and offline?
Libvirtd is much better suited to managing that sort of complexity.
I have the luxury of opining that such complicated networking schemes should be reined in, and not served with baroque management software. Complexity is the arch enemy of security, because systems must be kept easy to reason about.
Indeed. Instead of contributing a patch to qemu with better defaults, or even support for config files, someone wrote tens of thousands of lines of complex code.
People forget that software is malleable. Bend it to your needs, instead of adding fragile wrapper layers.
libvirt-the-software is tens of thousands of lines of complex code that doesn't really add much. libvirt-the-project does contribute patches in to qemu, and a lot of the reason that libvirt-the-software doesn't add much of value is because libvirt-the-project worked to get capabilities in to upstream qemu.
The libvirt team definitely does view qemu as malleable, and bends it to their needs.
The problem is that the libvirt authors contributing to qemu look at qemu as a machine-interface (to be consumed by libvirt), not as a human-interface. Many of the more poorly documented or otherwise difficult-for-humans flags were contributed by the libvirt team. And they're useful! They make it so qemu can do things it couldn't do before. But they're also kinda the reason qemu is difficult to use without libvirt.
> Libvirt is one of those softwares which ads a layer of complexity for the sake of complexity, yet offers nothing of value in return.
Isn't that sort of like saying the system startup scripts/subsystem adds complexity for the sake of complexity, and you should just use ifconfig and route manually in some script after startup?
In both cases, that you can fall back to the underlying commands is great. That you should use them instead of the normalized interface provided is debatable.
Unless the default changed one of these last few years, I'm pretty sure if you don't use `-enable-kvm` then qemu emulates the selected architecture. That's probably why it doesn't need root.
EDIT: Nevermind, `-enable-kvm` also doesn't need root. I'm not sure what libvirt uses root for.
Hear, hear. QEMU is excellent software, and I don't find that libvirt adds any value to its use.
Instead, one of the more annoying things I have ever done was foolishly installing libvirt and having it do all kinds of whackiness to a previously excellent QEMU set-up. I still cannot figure out where it managed to change all QEMU invocations to be VNC-based.
That's far more than I typically put in my qemu calls. You can technically be fine with just:
qemu-system-x86_64 foobar.qcow2
but I always add `-enable-kvm` and use `-monitor stdio` or `-nographic` depending if I want a graphic display from the vm or not. `-nographic` is all I need to give me access to a serial console in the guest (besides configuring the guest kernel with the paramater `console=ttyS0`). I also add `-m` because the default amount of memory is just 128 MiB, which is typically not enough for my use. Sometimes I add network related arguments, but we're going beyond minimal then.
There's not really much choice than trying to use qemu command line directly on windows. I didn't find any single useful GUI that was usable or newer than 3 or 4 years ago. It's a pretty sad state of affairs. I spend hours until I discovered that I needed to add some magic flag called machine for acceleration even work.
E.g. here is a minimal QEMU command-line that gives you access to a serial console in the guest:
`qemu-system-x86_64 -display none -no-user-config -nodefaults -m 2048 -device virtio-scsi-pci,id=scsi -device virtio-serial-pci -serial stdio -drive file=/export/cirros.qcow2,format=qcow2,if=virtio`
Simple, yeah? Now here (it's too long to post in this comment) is a real world QEMU command-line (as launched by the libvirt[1]):
htttps://kashyapc.fedorapeople.org/Fedora-28-QEMU-command-line-by-libvirt.txt
Compare and contrast the number of devices that libvirt adds.
FWIW, I strongly recommend to use libvirt to launch your QEMU-based guests. It is far more effective, and more importantly, provides security infrastructure like running the QEMU process as an unprivileged user, protecting the QEMU process (and its associated disk iamges) via the SELinux-based sVirt mechanism, and so forth.
Check out these[2] slides (recording is online, too) on "Security in QEMU" from one of the QEMU maintainers, Stefan Hajnoczi, at last year's KVM Forum in Edinburgh.
[1] http://libvirt.org/
[2] https://vmsplice.net/~stefan/stefanha-kvm-forum-2018.pdf