For those of you born… shall we say… more recently than some of us, you might not be familiar with the term “mainframe” or think that it is some ancient server lost to the mists of time. The generic term refers to any large single or multi-user computer that was typically larger than one single cabinet in a datacenter. In the vernacular of today, this almost exclusively refers to multi-user hardware from IBM running either a proprietary operating system such as MVS.
Mainframes are still much in use today, running old legacy applications and serving as large servers that (interestingly enough) might run a variant of the Linux operating system. Interestingly, Ubuntu has been available for this platform for some time and I decided I wanted to add this unique architecture to my collection of machines at my disposal.
Unfortunately I don’t have gigawatts of power at my house nor the necessary external watercooling that some of these beasts require (plus, my wife would have had my head) so I decided to go out on a limb and try to spin a modern Ubuntu LTS up on emulated “zSeries” (that’s what IBM calls it these days) hardware.
I initially tried getting this working using the v4 version of Hercules based on an interesting blog post, but was unsuccessful. If you are interested in the details, jump to the end of this post and maybe you can figure out what I was doing wrong. In the meanwhile, here is what I did to get things working using QEMU.
Ubuntu 20.04 LTS on zSeries using QEMU
First things first, I did a search to see if I could find a simple how-to that walked me through the process. I did find some, but they were a bit out of date and also didn’t go into the networking aspects of QEMU to a level where I could successfully spin things up. Here are the posts that I based most of this blog’s work upon:
- https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/timothy-sipples1/2020/04/28/run-ubuntu-z-linuxone
- https://wiki.ubuntu.com/S390X/InstallationGuide
- https://til.simonwillison.net/docker/emulate-s390x-with-qemu
The obvious initial step is to install QEMU as well as the special features that allow it to emulate a zSeries processor from IBM:
$ sudo apt install qemu-system-s390x qemu
The next step in the process is to create a network bridge on the machine that you are using as the host. Do do this, edit your /etc/netplan/*.yaml file (substitute your NIC name and IP information):
# This is the network config written by 'subiquity'
network:
ethernets:
enx00ec6306fb8:
dhcp4: no
bridges:
br0:
dhcp4: no
addresses:
- 192.168.1.99/24
gateway4: 192.168.1.1
nameservers:
addresses:
- 192.168.1.1
search:
- example.com
interfaces:
- enx000ec6306fb8
version: 2
You have to then activate the changes to your network. Note that running the command below could result in you not having remote access to your machine you are doing this on so you might have to re-ssh into the box from another terminal window.
$ sudo netplan apply --debug
Then, set up the qemu tap device that uses the bridge you created above:
$ sudo apt install qemu-kvm bridge-utils
$ sudo ip tuntap add tap0 mode tap
$ sudo brctl addif br0 tap0
Now you need to create a disk image to use as your virtual storage device to install Ubuntu on:
$ sudo qemu-img create -f qcow2 ubuntu-run.qcow2 10G
You can make the image any size you want. I also tried this with a “raw” formatted image but ended up having better luck using the qcow2 format instead. Now you have a place to install your Ubuntu s390x machine.
Now, you need to download the latest install image. I tried to get this working with 22.04 LTS but the installer kept crashing on me at various points in the process. I suspect it might have a dislike for the virtual serial console over telnet business. Therefore, I proceeded with 20.04 LTS instead:
$ wget https://old-releases.ubuntu.com/releases/20.04/ubuntu-20.04.4-live-server-s390x.iso
Now you will need to mount the ISO and extract the kernel and initrd images because those will be used by QEMU in its command-line:
$ mkdir tmpmnt
$ sudo mount -o loop ./ubuntu-22.04.1-live-server-s390.iso tmpmnt
$ cp tmpmnt/boot/kernel.ubuntu .
$ cp tmpmnt/boot/initrd.ubuntu .
$ sudo umount tmpmnt
To make things easier for myself, I created a script to launch the emulated s390x environment named run-s390x.sh:
#! /usr/bin/bash
qemu-system-s390x -machine s390-ccw-virtio -cpu max,zpci=on,msa5-base=off -serial telnet::4441,server -display none -m 8192 --cdrom ubuntu-22.04.1-live-server-s390x.iso -kernel kernel.ubuntu -initrd initrd.ubuntu -drive file=ubuntu-run.qcow2,format=qcow2 -net nic,model=virtio,macaddr=00:00:00:00:00:01 -net tap,ifname=tap0,script=no,downscript=no
Once I was ready to install things, I ran the script:
$ chmod +x run-s390x.sh
$ sudo ./run-s390x.sh
At this point, the emulator is paused, waiting for you to connect (via telnet port 4441 on the localhost address) to a virtual serial console. Therefore, from another terminal window on this machine, connect to the virtual serial console:
$ telnet localhost 4441
At this point you should see the system boot up. It takes some time in the emulated environment. Eventually you get to the Ubuntu installer. From yet another terminal window on this machine, bring the tap0 interface up (I find that it doesn’t come up on its own until something is actually attached to it from qemu):
$ ip link set up dev tap0
$ ip a # confirm that tap0 shows "up"
Back in the installer, I chose to run it in “rich mode” and chose the following options:
- English
- Ubuntu Server
- On the “ccw screen” – just hit “Continue”
- Take the network defaults (should be DHCP from your network) or configure to match a static IP address
- No proxy
- Take the default mirror address
- Skip the 3rd party drivers
The install took a while but eventually completed successfully. Modify the run-s390x.sh file to be as follows (take out the installer, etc.):
#! /usr/bin/bash
cd /root
ip tuntap add tap0 mode tap
brctl addif br0 tap0
qemu-system-s390x -machine s390-ccw-virtio -cpu max,zpci=on,msa5-base=off -smp 2 -serial telnet::4441,server -display none -m 8192 -drive file=ubuntu-run.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-ccw,devno=fe.0.0001,drive=drive-virtio-disk0,bootindex=1 -net nic,model=virtio,macaddr=00:00:00:00:00:01 -net tap,ifname=tap0,script=no,downscript=no
I then created a systemd service to start the qemu session at boot time after the network is operational by editing /etc/systemd/system/s390x.service:
[Unit]
Description=Launch s390x emulator session
After=getty.target
Before=s390x-network.service
[Service]
Type=simple
RemainAfterExit=yes
ExecStart=/root/run-s390x.sh
TimeoutStartSec=0
[Install]
WantedBy=default.target
Next, create another script to bring up the tun0 interface (I called mine /root/tuntap.sh):
#! /usr/bin/bash
sleep 15
ip link set up dev tap0
Set the execute bit on the script:
$ sudo chmod +x /root/tuntap.sh
Then, create a second systemd service to start networking after qemu has attached to tun0:
[Unit]
Description=Enable s390x networking
After=getty.target
[Service]
Type=simple
RemainAfterExit=no
ExecStart=/root/tuntap.sh
TimeoutStartSec=0
[Install]
WantedBy=default.target
Finally enable and start the service:
$ sudo systemctl enable s390x-network.service
$ sudo systemctl start s390x-network.service
Rebooting then confirmed that the service is started at boot time and reachable from the network. At this point you should be able to ssh into the s390x “machine” as if it were a real mainframe running Ubuntu on your network.
Using Hercules instead of QEMU
This turned out to be a bit of a dead-end, for me at least. The newer installer (22.04) had a kernel fault (I suspect because the virtual processor was too “old” as configured to be supported – probably fixable) so I used an older version. I managed to get it to ping the network, but the DNS wasn’t working and even with a hardcoded manual IP address for the Ubuntu server, it didn’t work.
From:
– https://sdl-hercules-390.github.io/html/hercinst.html#install
– http://www.fargos.net/packages/README_UbuntuOnHercules.html
Started with 22.04 server
Install Hercules v4 (the one that installs with apt is v3)
$ sudo apt install git wget time build-essential cmake flex gawk m4 autoconf automake libtool-bin libltdl-dev libbz2-dev zlib1g-dev libcap2-bin libregina3-dev net-tools
$ git clone https://github.com/SDL-Hercules-390/hyperion.git
$ cd hyperion
$ ./util/bldlvlck # make sure everything is “OK”
$ ./configure
$ make
$ sudo make install
Installing the helper scripts
$ cd ~
$ wget http://www.fargos.net/packages/ubuntuOnHercules.tar.gz
$ mkdir ubuntu-hercules
$ cd ubuntu-hercules
$ tar xvf ../ubuntuOnHercules.tar.gz
Install Ubuntu
$ cd ubuntu-hercules
$ wget https://old-releases.ubuntu.com/releases/22.04/ubuntu-22.04-live-server-s390x.iso
$ LD_LIBRARY_PATH=~/hyperion/.libs ./makeNewUbuntuDisk -c 48000 -v 22 # makes a 32g disk for Ubuntu 22.04
# Modify ./hercules-ubuntu.cfg to have the DASD show up with a -v22.disk filename
$ LD_LIBRARY_PATH=~/hyperion/.libs ./boot_ubuntu.sh –help
# In my case, I need to set the default gateway and DNS server and change the hostname
$ sudo LD_LIBRARY_PATH=~/hyperion/.libs ./boot_ubuntu.sh –iso ubuntu-22.04-live-server-s390x.iso –dns 192.168.x.x –gw 192.168.x.x –host s390x
# When you get asked questions in the install, prefix your answer with a period (‘.’)
# Choose CTC for network and then pick id #1 for read and #2 for write (at least that’s the one that worked for me)
# Use Linux communication protocol when prompted
# Do not autoconfigure the network but do it manually
# Select 10.1.1.2/24 for your IP address and 10.1.1.1 for your gw and 8.8.8.8 for dns server
# The system seems to have a problem resolving DNS so I used the IP address of the Ubuntu archive mirror
Pingback: Fast Follower – Unlimited POWER!!!!! | Functionally Paranoid