Thanks Ralph!
I don't think you can imagine how helpful your text explanation was.
Qemu documentation is a world of assumptions about the user,
conflicting docs, anecdotal solutions, ambiguity, and pretty much
anything except how the whole thing's put together. Your documentation
(quoted below) is by far the best in terms of explaining how the whole
thing fits together. THANK YOU!!!!!
Based on your explanation of how the bridge interface works, and how
-netdev user creates an IP level hack, and how id strings are used, I
quickly created the following qemu startup script:
============================================
#!/bin/sh
dvddir=/scratch/linuxinst/devuan/devuan_beowulf/installer-iso
qemu-system-x86_64 --enable-kvm \
-cdrom $dvddir/devuan_beowulf_3.0.0_amd64-desktop.iso \
-hda /scratch/qemu_images/beowulf.disk \
-m 4G \
-boot c \
-netdev bridge,id=mybridge0,br=br0 \
-device virtio-net-pci,netdev=mybridge0 \
============================================
The preceding shellscript produces a Devuan guest VM with the following
capabilities:
* Can ping, ssh, or web browse to 192.168.0.1, 192.168.0.2, 192.168.0.13
* Can ping, ssh or web browse to Troubleshooters.Com
* Can be pinged, ssh'ed or browsed from 192.168.0.2 or 192.168.0.97
* Appears much more performant than -netdev user
* Can ping (probably with help from some other procedures I did before)
* Can be run by user slitt (after making slitt a member of group
unpriv_ping)
* Requires no virt-manager, aqemu, brctl, etc.
In other words, my Devuan VM guest looks a hell of a lot like just
another metal machine on my 192.168.0.0/24 LAN. Which is what I wanted.
I still have tasks to complete:
* Convert it from DHCP to a fixed address without losing its
capabilities.
* Make the screen larger and focus easier to achieve
* Make keystrokes go to the Devuan VM guest instead of the Void metal
host.
* Achieve a remote GUI connection via ssh -Y or VNC.
* Making it run from user slitt inste
* Write documentation, which will require me backing out a lot of the
stuff I did to see which is necessary and which is anecdotal fluff.
I'm serious, once I read your documentation the solution became
obvious, requiring only a quick web lookup to determine exact syntax.
Thanks,
SteveT
On Sun, 21 Feb 2021 22:49:09 +1100
Ralph Ronnquist via Dng <dng@???> wrote:
> Networking for qemu is not that hard though better documentation is a
> praiseworthy aim. But there are an awful lot of variations both in
> what to achieve and in the ways in achieving it. These are some
> thoughts from me:
>
> Like for all qemu device emulations, the network setup arguments for
> qemu come in pairs:
> + one argument that declares the internal emulation device, and
> + another argument that declares to host side attachment.
>
> The connection between the two arguments is done by means of an
> identification label, where (for net emulation) the host attachment is
> "named" by the label using an "id=LABEL" option and the internal
> emulation device argument mentions it using a "netdev=LABEL" option,
> where thus the LABEL bit is your choosen name.
>
> If you leave out identifications then qemu makes guesses about how
> arguments go together; it invents labels "cleverly" and uses them
> "intelligently". Unfortunately most documentation relies on that
> hidden labelling and as a result things appear to work by magic.
>
> Further just for the sake of increased confusion, there is also an
> alternative combination or shortcut argument ("-nic") that lets you
> configure both the emulation and attachment in a single option
> collection.
>
> In your case you have declared a single internal device, the e1000,
> and linked that to one of the two host attachments by the label
> "mynet0". The other host attachment, "mybridge0", is dangling, i.e.
> not linked with any internal interface emulation device, which qemu
> advices about with the "no peer for mybridge0" warning.
>
> HOST SIDE ATTACHMENT
>
> The hands-on for the host side setup depends firstly on which type of
> host attachment it is. "user" and "bridge" are two host attachment
> types. The (currently) full list is
> "(tap|bridge|user|l2tpv3|vde|netmap|vhost-user|socket)".
>
> They are all configured in their own ways although typically they use
> the same or overlapping names of options with typically concordant
> uses of them. The qemu-system-x86 "man" page is rather good at
> describing the attachment types and their options.
>
> In my words:
>
> The "user" type attachment is a "pretend" attachment as it
> facilitates IP level networking for the guest without having
> networking presence as an actual host interface. There is little
> good to say about this attachment type except that it's the one to
> use if you don't want to know anything about anything. Well, not my
> preference at least. The "user" type attachment is especially
> peculiar as it includes a DHCP service for the guest with a client
> visible IP address for the host, even though the attachment is not
> associated with an interface.
>
> The "bridge" type attachment results in a host side tap interface
> that get added to (and remove from) a nominated bridge or bridge
> "br0" by default.
>
> The "tap" type attachment results in a host side tap interface
> seemingly without any bridge juggling. However, its default script
> option nominates the /etc/qemu-ifup script which adds the tap to the
> bridge, if any, nominated by a br option, or by default the bridge,
> if any,that facilitates the default route for the host at the time
> guest is started.
>
> The "vde" type attachment is also a "pretend" attachment but it
> facilitates full Ethernet level networking via VDE virtual
> networking, which typically is present as a tap interface on the
> host. In this case you set up a "VDE switch" with a tap, and then
> make qemu connect up to that via the sock option which is
> "/tmp/vde.ctl" by default (coincidentally the same as the default
> setting for a VDE switch).
>
> and so forth.
>
> CLIENT SIDE
>
> The hands-on for the client side is just normal network configuration.
> which parimarily depends on whether to use DHCP (in part or full) or
> static configuration.
>
> With the "user" type attachment, the client is not a full networking
> entity from the host (or outside) point of view as it only handles IP
> level transport. Other attachment types makes the client a full
> networking entity.
>
> To use DHCP the overall setup must of course include a DHCP server on
> the network of the host side attachment (or the built-in one of the
> "user" type attachment).
>
> etc.
>
> Ralph.
>
> On 21/02 03:08, Steve Litt wrote:
> >
> > On Thu, 18 Feb 2021 07:18:44 -0700
> > Gabe Stanton via Dng <dng@???> wrote:
> >
> > > I'm sorry for the confusion. That was not the guide I used. I did
> > > find the guide I used. It seems pretty straight forward, and I
> > > believe it clears up all the confusion and questions caused by my
> > > previous email.
> > >
> > > https://www.debian.org/doc/manuals/debian-handbook/sect.virtualization.en.html#sect.lxc.network
> > >
> >
> > Thanks Gabe. The preceding link helped, but was not sufficient.
> > Although better than most, it shares the same ambiguities as the
> > others, including not telling whether they're referring to the metal
> > host or the VM guest when discussing TAPs, bridges, devices and the
> > like. Also, like most of the others, they don't specifically
> > identify what should go in the "id=" slots.
> >
> > Based on the preceding link, I deduce that the TAP is created by the
> > guest VM, in such a way that it attaches to the bridge created on
> > the metal host, and therefore I have no need to create a TAP on the
> > metal host.
> >
> > Here's my progress so far, based on the link you supply above and my
> > other readings and experimentation:
> >
> > ***
> >
> > I build the bridge purely with ip commands. Also, I don't mess
> > with the firewall (which perhaps has been my problem all along).
> > I'll investigate this tomorrow.
> >
> > Below are some scripts and stuff I'm using. The following is
> > upnet.sh, which I use to set up networking on the metal host, which
> > happens to run Void Linux, which has no /etc/network/interfaces:
> >
> > =========================================
> > #!/bin/sh
> >
> > use_bridge=1
> > use_tap=0
> >
> > dev="enp40s0"
> > ipaddr_major="192.168.0.2"
> > ipaddr_minor="192.168.0.102"
> > gateway="192.168.0.1"
> >
> > error_tap_without_bridge(){
> > echo -n "ERROR: Can\'t set TAP without "
> > echo -n "BRIDGE! "
> > echo Aborting...
> > exit 1
> > }
> >
> >
> > enable_ip_forwarding(){
> > echo 1 > /proc/sys/net/ipv4/ip_forward
> > }
> >
> > unset_everything(){
> > dev=$1
> > ip_maj=$2
> > ip_min=$3
> > gateway=$4
> > echo "Unsetting everything for $dev, $ip_maj and $ip_min"
> > ip link set dev tap0 down
> > brctl delif br0 tap0
> > ip link del tap0
> > ip link set dev br0 down
> > ip addr del $ip_min/24 dev br0
> > ip addr del $ip_maj/24 dev br0
> > brctl delbr br0
> > ip link set dev $dev down
> > ip addr del $ip_min/24 dev $dev
> > ip addr del $ip_maj/24 dev $dev
> > echo ""
> > }
> >
> > set_hostname_and_localhost(){
> > echo "Setting hostname and localhost"
> > hostname=`grep -v "^\s*#" /etc/hostname | head -n1`
> > ip link set dev lo up
> > echo ""
> > }
> >
> > create_phys_device_link(){
> > dev=$1
> > echo Creating device link for $dev
> > ip link set dev $dev up
> > echo ""
> > }
> >
> > set_phys_device_addr(){
> > dev=$1
> > ip_maj=$2
> > ip_min=$3
> > gateway=$4
> > echo -n "Setting physical device addresses "
> > echo -n "$ip_maj "
> > echo -n "and $ip_min "
> > echo -n "for $physdev "
> > echo "with gateway $gateway"
> > ip link set dev $dev down
> > ip addr add $ip_maj/24 dev $dev
> > ip addr add $ip_min/24 dev $dev
> > ip link set dev $dev up
> > ip route add default via $gateway
> > echo ""
> > }
> >
> > set_bridge(){
> > dev=$1
> > ip_maj=$2
> > ip_min=$3
> > gateway=$4
> > echo Setting bridge for $dev
> > echo -n "Creating and setting bridge addresses "
> > echo -n "$ip_maj "
> > echo -n "and $ip_min "
> > echo -n "for $physdev "
> > echo "with gateway $gateway"
> >
> > ip link add name br0 type bridge
> > ip link set dev $dev master br0
> > ip addr add $ip_maj/24 dev br0
> > ip addr add $ip_min/24 dev br0
> > ip link set dev br0 up
> > ip route add default via $gateway
> > echo ""
> > }
> >
> > set_tap(){
> > echo Setting tap
> > ip tuntap add tap0 mode tap
> > brctl addif br0 tap0
> > #ip addr add 192.168.0.66/24 dev tap0
> > ip link set dev tap0 up
> > echo ""
> > }
> >
> > show_networking(){
> > echo -n "Networking follows in 3 seconds..."
> > sleep 3
> > echo "\n"
> > echo "========================================"
> > echo "========================================"
> > ip -4 link
> > echo "......................"
> > ip -4 addr
> > echo "......................"
> > ip -4 route
> > echo "========================================"
> > echo "========================================"
> > }
> >
> > echo "\nBegin upnet.sh"
> >
> > [ "$use_tap" = "1" ] && [ "$use_bridge" != "1" ] && \
> > error_tap_without_bridge
> >
> > unset_everything $dev $ipaddr_major $ipaddr_minor $gateway
> >
> > enable_ip_forwarding
> >
> > set_hostname_and_localhost
> >
> > create_phys_device_link $dev $ipaddr_major $ipaddr_minor $gateway
> >
> > [ "$use_bridge" = "1" ] || \
> > set_phys_device_addr $dev $ipaddr_major $ipaddr_minor $gateway
> >
> > [ "$use_bridge" = "1" ] && set_bridge $dev \
> > $ipaddr_major $ipaddr_minor $gateway
> >
> > [ "$use_tap" = "1" ] && \
> > set_tap $dev $ipaddr_major $ipaddr_minor $gateway
> >
> > show_networking
> > =========================================
> >
> > The preceding just builds br0 with ip addresses 192.168.0.2 and
> > 192.168.0.102, default route (gateway) 192.168.0.1, for my metal
> > host, and runs every time my metal host is rebooted (or it can be
> > run any time). It has provisions to build a tap, or to not build a
> > bridge and instead assign the IP addresses and default route to
> > enp40s0 itself.
> >
> > The next shellscript runs on my metal host to launch a (Devuan) VM
> > guest:
> >
> > =========================================
> > #!/bin/sh
> >
> > dvddir=/scratch/linuxinst/devuan/devuan_beowulf/installer-iso
> >
> > qemu-system-x86_64 --enable-kvm \
> > -cdrom $dvddir/devuan_beowulf_3.0.0_amd64-desktop.iso \
> > -hda /scratch/qemu_images/beowulf.disk \
> > -m 4G \
> > -boot c \
> > -netdev bridge,id=mybridge0,br=br0 \
> > -netdev user,id=mynet0,restrict=no,net=192.168.0.0/24 -device
> > e1000,netdev=mynet0 \
> >
> >
> > =========================================
> >
> > In the preceding, notice that the blank line at the end is
> > necessary to end the series of backslashed lines. I backslash ALL
> > the lines so experimentally I can move things in and out and
> > rearrange. Like I said, I've tried hundreds of combinations.
> >
> > In the preceding, I didn't declare a TAP. As best I could read the
> > various conflicting documentation, the VM guest *creates* the tap
> > and connects it to the metal host created bridge, so no need to
> > create the TAP on the metal host. But then again, I've tried it
> > both ways.
> >
> > One hint I've gotten, a hint which I cannot bring to fruition
> > because of sparse and contradictory documentation, is that this
> > brings up a "no peer for mybridge0" warning. I've tried
> > substituting both "eth0", enp40s0, and "junk" for "mybridge0", and
> > as I remember (hundreds of experiments, didn't write them all
> > down), doing so didn't affect the symptom (the no peer warning). I
> > chose the preceding script due to the following documentation,
> > which seemed better than most:
> >
> > https://www.debian.org/doc/manuals/debian-handbook/sect.virtualization.en.html#sect.lxc.network
> >
> > On my Devuan *guest* VM, I have the following
> > /etc/network/interfaces, as suggested by a careful reading of
> > https://www.debian.org/doc/manuals/debian-handbook/sect.virtualization.en.html#sect.lxc.network
> >
> > =========================================
> > source /etc/network/interfaces.d
> >
> > auto lo
> > iface lo inet loopback
> >
> > auto eth0
> > allow-hotplug eth0
> >
> > auto tap0
> > iface tap0 ifacem manual
> > vde-switch -t tap0
> >
> > auto br0 inet static
> > bridge-ports tap0
> > address 192.168.0.60
> > netmask 255.255.255.0
> > =========================================
> >
> > When, on the *host*, I run my instantiations script to create a
> > guest VM, the following is output to the terminal:
> >
> > =========================================
> > [root@mydesk qemu_images]# ./runbeowulf.sh
> > WARNING: Image format was not specified for
> > '/scratch/qemu_images/beowulf.disk' and probing guessed raw.
> > Automatically detecting the format is dangerous for raw images,
> > write operations on block 0 will be restricted. Specify the 'raw'
> > format explicitly to remove the restrictions. qemu-system-x86_64:
> > warning: netdev mybridge0 has no peer
> > =========================================
> >
> > After that, my Devuan guest VM appears, to which I log in and run a
> > terminal. IP addresses are 192.168.0.15 for the guest VM itself, and
> > 192.168.0.2 (my metal host) for the default route.
> >
> > The VM guest can lynx to my nginx server on 192.168.0.2, and to any
> > HTML page on the Internet, but cannot lynx to my printer at
> > 192.168.0.13 and my metal cable modem at 192.168.0.1. From my metal
> > desktop (which runs the guest VM) at 192.168.0.2 I cannot ssh to
> > slitt@192.168.0.15:
> >
> > =========================================
> > [slitt@mydesk qemu]$ ssh slitt@192.168.0.15
> > ssh: connect to host 192.168.0.15 port 22: No route to host
> > [slitt@mydesk qemu]$
> > =========================================
> >
> > I'm pretty sure "no route to host" means this isn't caused by a
> > firewall problem, although once I fix the routing thing, that might
> > unmask a further firewall problem.
> >
> > In other words, my VM guest is in no way a peer of the various metal
> > hosts on my 192.168.0.0/24 physical Ethernet network.
> >
> > If anybody has any words of wisdom, and can identify whether each
> > wisdom word applies to the metal host or the guest VM, I'd love to
> > hear them.
> >
> > Thanks,
> >
> > SteveT
> >
> > Steve Litt
> > Autumn 2020 featured book: Thriving in Tough Times
> > http://www.troubleshooters.com/thrive
> > _______________________________________________
> > Dng mailing list
> > Dng@???
> > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
> _______________________________________________
> Dng mailing list
> Dng@???
> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
SteveT
Steve Litt
Autumn 2020 featured book: Thriving in Tough Times
http://www.troubleshooters.com/thrive
::
Re: [DNG] Assigning a specific subn…
