Where the rubber meets the road… (part one)

So, if you read my recent post on setting up a virtual machine multi-boot image with OpenBSD and Linux, you’ll be familiar with the background for this post.  Today, I’m going to detail the exact steps I took to set up my new work laptop, a Thinkpad X1 Carbon (4th Generation Skylake) laptop running Arch Linux and OpenBSD.  Since we don’t have solid support yet for Skylake, there are some interesting workarounds that others, far smarter than me, were kind enough to leave on the Internet.  I’ll detail those and give credit to the original authors below.

First off, let’s talk about why I’m doing this.  As some of you may know, I’ve been a bit of an Apple fanboi for quite some time.  So the concept of me moving off of Apple hardware to a <yuck> “PC” </yuck> was something that, quite frankly freaked out some of my friends.  One of them described it as one of the four signs of the apocalypse!

So, given that, why do I want to do it.  There is an old saying that you can throw a frog in a pot of boiling water and it will jump out to save itself, but if you put it in a pot of cool water and slowly raise the temperature, it will let itself be boiled to death because the change was so gradual.  Gross!  However, pretty apt to describe my situation.  Every Apple product that comes out is (read this in a Jony Ive accent) “x percent thinner and lighter than the previous one!”  Well, I’ve often wished that Apple would do certain things to the hardware – make it more standard, faster, more RAM, more battery life, etc.  However, I never asked for “thinner and lighter”.

As a result, when I saw the 12″ Macbook released I thought it would be a nice travel laptop and overlooked the fact that it had a single USB-C port (who doesn’t like dongles), in a stiff breeze it might blow over on a table, the CPU was massively under-powered and the NVMe hard drive in it is so weird it actually reports the wrong PCI device ID when queried by the bus.  But whatever, it was thinner and lighter.  Nevermind that I ended up carrying more dongles and adapters in my backpack to offset the weight “loss” and that USB-C can sometimes be finicky and a pain in the butt.  Oh and that extra thin keyboard.  What a pain to type on.  Oh and don’t get me started on the number of times I had to mash the power button for a hard reboot when it didn’t wake from sleep…

Then came the spate of bugs in iOS.  I can’t tell you how many times I’ve had to hard reboot my phone just to do things like make calls.  Oh, and the recurring one where my phone is ringing and the UI is locked up and won’t swipe to answer – awesome!  Then came macOS Sierra.  With kernel panics weekly (hadn’t seen one of those since Tiger), the 6-10 times per day where my external display blanks for 3-5 seconds and that Mail and Calendar crash daily for me.  I had hoped that the .1 release would fix the problems – it didn’t.

So I finally realized I was loving a glorified historical Apple and that I really wasn’t a happy user.  Then came the new Macbook Pro.  For the first time, I saw a new laptop that I didn’t actually want to buy.  As a VI user, the thought of having a fake escape key sounded awful and the little touch screen was little more than a gimmick.  Plus the under powered hardware and that “blessed” thin keyboard again?  No thanks.  So I made the hard decision to make the switch, did my research and settled on this setup.  Whew.  Some backstory!

OK, so I planned on getting the new laptop out of its box, shrinking the Windows partition, adding in Arch and OpenBSD (both with full disk encryption – remember the name of this blog).  Should be fun.

Shrinking the Windows partition was pretty simple.  I booted up, let it do it’s “first time you boot Windows” business, rebooted just to be sure I was in a clean state.  Rebooted and went into Disk Manager.  Resized the partition down to 200G, created three blank partitions – Arch /boot, Arch encrypted luks volume and OpenBSD “slice” (I chose to leave them unassigned relative to drive letters or paths and didn’t format them, I just wanted place-holders) and rebooted to be sure it all still worked.  It did.

Here’s where it got fun.  Time to boot the Arch install media and get started.  I based my install on this excellent blog post.  Also, based on this post, I took the recommendation and turned off secure boot.  The first thing I noticed was that the NVMe drive showed up as a different type of animal on Arch than I had expected.  It wasn’t a /dev/sd* device, it was /dev/nvme0n1 and my partitions were /dev/nvme0n1p1, etc.  So here is where I give you kids a PRO TIP.  When you are creating your encrypted luks volume, don’t get confused by the fact that there is a number in the base device name and think you are telling it to encrypt that partition and instead create a luks volume on the entire drive.  Yep.  That’s what I did.  Darnit! I could use the rescue partition – oh wait, I managed to whack that too.

Oh well, I don’t like Windows anyhow, so I’ll salve my pride by saying I intended for this to be a dual-boot install not a triple-boot one.  If I ever need to update the BIOS I’ll cross that bridge when I come to it.  So, starting from the top.  I booted the Arch install media, put a new GPT partition table on the disk and created three partitions.  A /boot partition that was type 8300, a luks container for my encrypted volume (type 8300 as well initially), and a type a600 partition for my OpenBSD slice.  I formatted the /boot partition as FAT32 and created a root, home and swapfile in the encrypted luks/lvm partition.  I did the install of Arch into it, rebooted and all was good.

I then set up a gnome desktop with gdm as the display manager (enabled on boot), created my default user (which I added to the wheel group and tweaked sudoers to allow me to run root commands when I need to) and installed my apps.  I rebooted and everything was swell.  The system handled my HiDPI display just fine, recognized all of the devices and even had hibernate / sleep working out of the box.  The only tweak I had to do was to disable Wayland because I intend to use Virtualbox to run my Windows VM for Skype for Business (the only reason I ever have to go to Windows these days).

In total, I ended up adding the following packages from the base Arch repository:

  • xorg-server
  • xorg-server-utils
  • xf86-video-intel (select libinput for the trackpad based on the earlier post I referenced on turning off secure boot)
  • mesa-libgl
  • gnome
  • gnome-shell
  • gnome-extra
  • gnome-initial-setup
  • gnome-software
  • gnome-tweak-tool
  • gnome-shell-extensions
  • evolution
  • evolution-ews (interestingly I had to first set up a non Exchange mail account before I could successfully set up an Exchange account)
  • firefox (follow this hardening guide for privacy optimization)
  • libreoffice (I installed stable because “fresh” seemed to oddly have some HiDPI problems)
  • aspell-en
  • chromium (for the rare times I have a site that doesn’t work on my locked down version of firefox)
  • openconnect and networkmanager-openconnect (we have a Cisco firewall at work)
  • vim (because it’s better than Emacs <grin>)
  • networkmanager
  • nmap
  • git
  • virtualbox (see this wiki entry on how to configure)
  • virtualbox-guest-iso
  • vlc and qt4 (so I can have the GUI)
  • ufw
  • openvpn and networkmanager-openvpn
  • freerdp
  • remmina
  • gstreamer

From the Arch User Repository (AUR), I installed the following:

  • slack-desktop
  • owncloud-client-service
  • skypeforlinux-bin
  • bluejeans
  • font-manager
  • hipchat4
  • tor-browser-en

OK.  Now for the fun part – installing OpenBSD.  I rebooted off of the latest snapshot install media, popped out to the shell and discovered to my dismay that it looks like my NVMe drive is not supported yet.  Dangit!  OK.  I think I know how to fix this, I’ll build a patched kernel and see if I can get around this – darned PCI device identifiers.  That will have to go in a separate blog post.

Making it real

In my last post, I talked about creating a multiboot Virtual Machine that had OpenBSD, FreeBSD and Linux on it.  In that post, I mentioned that my next step would be to actually set up a laptop on real hardware with this configuration and document the process.

In the intervening time, two big events and a lot of little ones occurred that are interesting to take note of here:

  1.  Apple released the new MacbookPro computers.  To say I am underwhelmed would be an understatement.  I recall a time when Apple not only was innovative and aggressive, but also built some of the best hardware I’ve ever had the privilege to use (more on that with #2 below). Unfortunately adding a tiny little OLED touch strip above the keyboard and further reducing the ports you have available to you doesn’t exactly align with that picture in my head.  But hey, it’s a free country (at least it still is, I’m writing this before the US Presidential election ) so YMMV.  You might really dig the new hardware from our friends in Cupertino.
  2. I bent my 12″ Macbook.  Yes, you read that right.  I bent the thing.  Not by dropping it off of a tall building or slamming it in a car door, but by simply carrying it in my backpack and putting it into the overhead bin on my flight.  It’s not really noticeable except for the fact that it now wobbles when I type on it.  And yes, I tested it on multiple flat surfaces.  And no, it isn’t that the little rubber foot fell off of it.
  3. I’m feeling like I’m being pecked to death by the numerous bugs I’m seeing in macOS Sierra.  There was a time a couple of releases ago where WiFi was a real problem.  Well guess what, that’s back.  I also am a big fan of Apple’s built-in mail and calendar programs.  Well, they randomly crash (or minimize when I click on another window) several times a week.  Oh, and did I mention the hibernation and resume from suspend problems I am having?  Oh, and the fact that while I’m typing this, the external monitor that is hooked up to my wobbly laptop via Apple’s own dongle has gone black for 1-3 seconds several times?

It just doesn’t feel like I’m the core demographic that they are targeting any more.  To that end, I’m seriously considering (and I know this will freak out anyone who knows me), stepping away from the Apple ecosystem.  Now the privacy nightmare malware application known as Windows 10 would not be where I land, so I begin to contemplate a world where I live on OpenBSD full time and work as well as at home.

Unfortunately, there are “corporate” things that I do that will require me to periodically use Windows (we have internal web-based tools that absolutely require Internet Explorer) and Microsoft Office is a necessary evil in the corporate world.  I know I could limp along with a remote desktop client to get to IE and possibly use the web tools in Office365, but I expect that I would run into pain that way.  This means that somewhere on this mythical machine, I would need a Windows VM.

So suddenly, my idea of doing a little experiment of dual booting a laptop got serious – as in deadly serious.  This “test” I’m conducting will be an experiment to see if I can walk away from macOS as my daily driver.  Now, ironically enough, the laptop I’m going to test this out on is an older MacbookPro 13 inch (an 11,1 model).  How is that for funny.  If I can live this way for a few weeks, however, I would be moving officially to a Lenovo Thinkpad X1 Carbon as it is the only small laptop I can find currently that has good open source OS support, a 1TB SSD and 16 GB of RAM.

OK, enough talking onto documenting what I did.  First, I started off by taking this poor old MacbookPro and running DBAN on it.  For those of you who aren’t familiar with DBAN, it stands for Darik’s Boot and Nuke.  Its a collection of utilities you can boot off of a thumb drive that will allow you to securely wipe a hard drive.  I erased the drive (including the EFI partition) on the MacbookPro so I could have a fresh start.

The next step was to download Ubuntu 16.10 onto a thumb drive and install it.  While I would have preferred to use Arch (I like rolling distros), I don’t want to ever be in a situation where I whack my work machine because I waited to long between updates, etc.  Ubuntu (or as I like to jokingly call it – Grandma’s Linux) is probably my better choice here.

Installing it was interesting though.  I couldn’t take the default and let it take the entire drive, plus I wanted to encrypt the disk so I did the manual configuration.  Understanding how to set things up took a couple of attempts and I’m still not totally satisfied with my end result.  In the end, I created an EFI system partition at the front of the disk that is 1,024MB in size, a /boot EXT4 partition that is the same size, and a 200GB encrypted volume that has in it a single EXT4 volume that is mounted as / to the filesystem.  What is missing is a swap drive.  I couldn’t create a “normal” swap partition because the Ubuntu installer informed me that Linux doesn’t encrypt its swap (that was a surprise to me as an OpenBSD user) and it wouldn’t even let me say “who cares do it anyhow.”  It just flat out refused to proceed until I did a swap-less install.  I got grumbled at by the installer but it let me proceed.

Once I rebooted (and the Ubuntu installer religiously hangs on me after installation so I did my typical wait 5 minutes and hold down on the power button – I think I am beginning to see why Bryan Cantrill calls Linux a “dumpster fire”) and rebooted.  I was prompted for my volume encryption password and was dropped very quickly at the desktop.  Now I created a swap FILE (not a partition):

$ sudo fallocate -l 20g /mnt/20GB.swap
$ sudo chmod 600 /mnt/20GB.swap
$ sudo mkswap /mnt/20GB.swap
$ sudo echo "/mnt/20GB.swap none swap sw 0 0" >> /etc/fstab

After doing that, I rebooted and confirmed that I had a working swapfile.

Next, I downloaded the latest snapshot of OpenBSD 6.0-current and dd’ed it to a thumb drive.  I rebooted, installed OpenBSD and promptly stepped all over the drive and made it unbootable.  Oops!

At that point, I decided I really wanted to better understand what I was doing and what better way to do that than to install Arch and do all of the LUKS stuff by hand.  I found a handy-dandy article that walked me through it.  I first downloaded the Arch install ISO and dd’ed it to a thumb drive which I booted from in UEFI mode on the Mac.

# wifi-menu

# timedatectl set-ntp true

# lsblk

From there I was able to find out that I needed to use the block device /dev/sda as my target (the internal drive on the laptop).  I then used gdisk to create the partitions.  I first issued the “o” command to wipe and recreate the GPT partition table.  I then created partition 1 as a 1,024M /boot and EFI system partition with a partition type of ef00 so that it was identified properly.  I then created partition 2 as a standard ext4 partition (this will be the LUKS partition) that consumed half the free space and partition 3 as an ext4 that consumed the rest (this will be where I will be installing OpenBSD).

# cryptsetup -c aes-xts-plain64 -s 512 -h sha512 -i 5000 -y luksFormat /dev/sda2
# cryptsetup luksOpen /dev/sda2 crypt
# lvm pvcreate /dev/mapper/crypt
# lvm vgcreate lvmpool /dev/mapper/crypt
# lvcreate -L 35GB -n root lvmpool
# lvcreate -L 16GB -n swap lvmpool
# lvcreate -l 100%FREE -n home lvmpool
# mkfs.fat -F32 /dev/sda1
# mkfs.ext4 /dev/mapper/lvmpool-root
# mkfs.ext4 /dev/mapper/lvmpool-home
# mkswap /dev/mapper/lvmpool-swap
# swapon /dev/mapper/lvmpool-swap
# mount /dev/mapper/lvmpool-root /mnt
# mkdir -p /mnt/boot
# mount /dev/sdX1 /mnt/boot
# mkdir -p /mnt/home
# mount /dev/mapper/lvmpool-home /mnt/home

Now that I have the volumes created and mounted, I installed Arch as I normally would.  By the way, how weird is it that Linux numbers its devices starting at 1?  I would have expected the first partition to be /dev/sda0 (which of course to an OpenBSD guy such as myself seems dyslexic as it should be /dev/sd0a like we do it).  🙂

Before rebooting, you have to enable the crypto in the kernel by adding “encrypt” and “lvm2” between “block” and “filesystems” in the /etc/mkinitcpio.conf file and then regenerate the initramfs by issuing the “mkinitcpio -p linux” command.  Install the simple bootloader via “bootctl install”.  Finally in /boot/loader/entries/arch.conf file, add the following:

linux /vmlinuz-linux
initrd /initramfs-linux.img
options cryptdevice=/dev/sda2:crypt ro root=/dev/mapper/lvmpool-root rw

Set your hostname in /etc/hostname, install the necessary wifi software with “pacman -S iw wpa_supplicant dialog” and change the root password with “passwd”.  Press CTRL+D to get out of your chroot, unmount /mnt (where you just installed Arch) and reboot.  You should be presented with a prompt to enter your crypto password and then you boot to the console just as you normally would.

Now to install OpenBSD.  I boot from the install60.fs USB stick as I normally would and immediately drop to the shell from the installer prompt.  Use fdisk to change the ext4 partition into an OpenBSD one, then use disklabel on that partition to create a swap file and the installer space.  Following this handy guide, I then created my softraid crypto filesystem and installed as I normally do.

Upon reboot, I am taken into Arch where I can install rEFInd from the console.  Another reboot and (although the icons and naming is pretty goofy by default on rEFInd in this case), I can safely switch between the two operating systems.

Mission accomplished!  I’ll clean things up with the rEFInd configuration, add puffy as an icon and get to it configuring my environments on both sides.  While it ended up being a long journey, I learned a lot on this and ended up having a test machine to now see if I can make the big switch away from macOS and Apple hardware.

Helping out an Internet Friend…

While I know everyone is probably waiting impatiently for my Fortran follow-up (I assure you all, I do have a working Fortran-based web application development framework – I just want to pretty it up a bit more <grin>), I thought I’d take a moment and diverge to talk about something I’m doing to help a fellow BSD aficionado out.

I’m in the process of standing up a BSD Users Group (BUG) in Indianapolis, IN (https://indybug.org) and in the process, met someone who was interested in dual booting a BSD and Linux on his HP laptop.  In the past, I cheated with my Thinkpad x220 (which is now an x230 but that’s another story) by just using two drives and leveraging the boot menu to decide which one to boot.  My primary SSD is OpenBSD-current and my m.SATA drive runs Arch for those times that I need something from the Linux world.

Well, this individual only has the ability to have a single drive in his laptop and would like to do the hard thing of dual booting.  Also, it turns out this is a UEFI machine.  Given that, I decided to roll up my sleeves and see what I could make happen.  Since we have EFI support in OpenBSD 6.0 now, I thought I’d give it a whirl in a Virtual Machine under VMWare Fusion on my Mac.

First things first, I needed to figure out how to set up a UEFI VM.  There were no obvious checkboxes to be found in the UI so, after some Googling, I found out that I just needed to add a line to the *.vmx file that contains the settings for the VM.  I created a generic 64-bit VM with 2 cores, a 40 GB IDE drive and 2GB of RAM, added firmware = “efi” to the VMX file and fired it up to see what happens.  Sure enough, I got an EFI boot screen from VMWare instead of the traditional BIOS one.  Looks like that first step was easier than I thought.

Next, I downloaded the OpenBSD 6.0 install60.iso image and booted.  No joy.  I fiddled around with it for some time and was unable to boot that image from the EFI version of VMWare Fusion.  Given that there is more than one way to skin a cat, I downloaded the latest Ubuntu 16.04 desktop ISO and was able to successfully boot that.  Therefore, I decided I’d install Ubuntu first and then layer on the OpenBSD install from that.

The only non default settings I chose were:

  • Install 3rd party software
  • Installation type of “something else” so I could control the partitioning
    • I created a 64MB EFI partition at the front of the disk
    • Next, I created a 20GB primary partition at the beginning of the space, mounted as the root (/) filesystem
    • I then added a 4096MB swap partition for Ubuntu
    • Finally, I used the rest of the free space to create a Reserved BIOS Boot AreaFAT32 partition that was not associated with a mount point – this is where I will be installing OpenBSD
      • You will get a warning about the FAT32 partition not being used and that’s fine.  We don’t want Linux to use it.  That will be where we’ll install OpenBSD
      • If you are asked about “forcing the system to be UEFI only” that’s what you want to do.  Remember, our goal is to create a UEFI dual booting system.
  • Everything else should be the defaults.  Just create your user and sit back and wait for the install to finish.

So I got that done and got ready to boot off of the OpenBSD installation ISO.  And that’s when it hit me.  Nothing has changed since I tried to boot off of it last time.  Doh!  OK, so I downloaded the install60.fs file and dd’ed it to a USB drive, mapped the USB drive to the VM and rebooted.  Hitting <ESC> quickly and repeatedly, I ended up at the UEFI boot device selection screen in VMWare and was able to boot off of the USB drive.  So I ended up at the initial install prompt.

I mashed through the defaults in the OpenBSD installer until I got to the disk partitioning.  Since I told VMWare to make my hard drive an IDE one, I knew I was playing around with wd0 and not sd0 (my USB key).  I dumped into fdisk by selecting to (E)dit the partition scheme and saw my setup from Linux.  First was the EFI partition (I am guessing I’ll have to copy my bootx64.efi file to that at some point), second was the Linux etx4 partition, third was my Linux swap partition and fourth was a weird looking one that is the “Reserved BIOS Boot” partition.  That’s the one I’ll fiddle with.

Issuing the command “edit 3” allowed me to fiddle with that partition #3 (remember, we start counting at zero).  I set it’s type to “A6” (OpenBSD) and then took the defaults with the exception of naming it “OpenBSD”.  A quick “write” followed by a “quit” allowed me to update my new partition and get back to the installer.

I took the (a)uto layout for the filesystem setup and let the installer create all of the filesystems.  Once that was done, I continued mashing defaults for the rest of the install and just let it do a full install of everything (including X) on the system with the files coming from the USB drive.  When I got dumped to the prompt at the end, that’s where I knew I needed to play around a bit.

I mounted the EFI partition (mount /mnt /dev/wd0i) and poked around.  It looks like the OpenBSD installer already created an /EFI/BOOT folder and stuck our boot loaders in it.  This might mean I can just reboot from the hard drive and it will just work.  I’ll give it a shot and see.  Sure enough, mashing <ESC> at the boot screen allowed me to choose between Ubuntu (the default) and an unlabeled hot mess that turned out to be OpenBSD.  Now I’m going to push the envelope and try to get rEFInd on this thing to have a pretty boot menu.

I rebooted to Ubuntu and hit the rEFInd web page so I could download the Linux installer.  I installed the .deb file and the ran “sudo refind-install”.  It installed just fine.  I rebooted and sure enough I could switch between Ubuntu and a weird icon in the middle that turned out to be OpenBSD.

I guess I have achieved my goal of dual booting Linux and OpenBSD on a UEFI install.  Now to see if my new friend can do the same by following this.

Wait… usenet is still… alive?!?!

In my previous post, I described getting re-acquainted with Fortran after 38 years and discovering that the language had really added some nice features over that time.  I also mentioned how my “curly brace bigotry” was starting to thaw out.  Well, now let’s talk about what I’ve learned over the past week.

First things first, like a good OpenBSD user, I found the mailing list for the GNU Fortran compiler and sent a question to it.  Wrong!  That list is for people working on the compiler itself, not for dumb n00b questions like mine.  Oh whelp.  They were pretty friendly about pointing me to the last place on earth I’d ever expect to be going – usenet.

For those of you who aren’t greybeards like me and have no idea what usenet is, I’ll enlighten you.  Back in the dark ages of the internet, when we used dial-up access, bandwidth was very dear to everyone – even the servers on the Internet itself.  Usenet was essentially a de-centralized message board that you could download information from in phases.  First, you pulled down the list of boards themselves, then when you selected your board, it pulled down the most recent topics, then when you clicked on a topic, you would get the actual text of the topic and any replies to it.

The topics were arranged hierarchically like comp.os.bsd and such.  There was a creepy part under the “alt” heading that had such jewels as “alt.wil.wheaton.die.die.die” for people who didn’t like the character he played on Star Trek TNG.  At one point, I could swear remembering that usenet went away and was really nothing more than an archive that some sites saved.  Apparently I was wrong because my friends on the compiler mailing list directed me to comp.lang.fortran.  On a side note, one of the folks on that list was very kind and helped kick start some of my early learning.

After a brief internet search, I found a link and clicked on it.  I don’t know if I should be seeking professional help or if others have dreams like this, but occasionally I have a “neglect dream” where (in the dream) I suddenly remember something I was supposed to have been doing but wasn’t – like feeding my non-existent giraffe.  Well, the feeling I have in the dream when I head out to where the giraffe lives (expecting at the least a thin and angry horned long-neck horse) was the feeling I had clicking on that link.  However, instead of a dead or murderous giraffe, I found a healthy message board that used Google Groups.

In addition to posting questions when I got stuck and reading code on the Internet, I also downloaded and read (yes – cover to cover) a Fortran programming book called “Introduction to Programming with Fortran:  Third Edition” on my Kindle web app so I could read it in my browser on OpenBSD.  After perusing this information, I was even more impressed with what Fortran has grown up into.  One of the folks on the comp.lang.fortran group pointed out that the oldest date he was aware of was October 15, 1956 (an old IBM 704 manual apparently) so we are essentially looking at Fortran’s 60th birthday in a few months.  That’s a long time for something to stay this relevant.

OK.  Time to dive in.  My use case was to create a string class and then add a subclass to it that uppercased all strings before storing them.  Pretty simple, eh?  You’d think until you learn that, while Fortran has great numerical and scientific intrinsic functions and features, it really lacks character handling capabilities.  No problem, the Internet to the rescue.  I found a string upper-casing and lower-casing implementation that I baked into what Fortran calls a “module”.  Essentially a module is a way to encapsulate types, code (functions and subroutines) and data.  Modules can be in stand-alone files and can be included into another piece of code using the “use” statement.

module foo_m
...
end module foo_m

program bar
use foo_m
...
end program bar

As you can see from the above example, this is quite similar to the “require” functionality that you see in Javascript programs of the Node.js variety, the “#include” feature of C/C++ and the “using” feature in C#.

After some pointers from the kind folks on the comp.lang.fortran mailing list, I cobbled together a bit of a style guide that I tried to follow as I built my example.  By the way, if you would like to see the finished product, take a look on github and you should have a fully buildable version on OpenBSD.  If you want to build this on another platform, you’ll probably have to change from “egfortran” to whatever the gfortran compiler is called on your platform (most likely gfortran).

My string_utility.f90 file ended up with the following structure:

module string_utility
implicit none
private
public :: ucase
public :: lcase
contains
pure function ucase(in_string) result(out_string)
character (len=*), intent(in) :: in_string
character(len(in_string) :: out_string
...
end function ucase

pure function lcase(in_string) result(out_string)
...
end function lcase

Some things to point out here.  The use of “implicit none” brought back some fond memories.  FORTRAN used to automatically type declared variables in a really cute way.  If the variable name started with a letter between ‘I’ and ‘N’, it assumed it was an INteger.  Ha!  Since we want to declaratively type things these days and not have “odd” side-effects from features like that, implicit none turns this feature off for the module.

Fans of other OO languages like C++, Java, etc. will recognize the use of “public” and “private”.  Fortran is public by default, so putting the private declaration at the top of the module flips this to the way I feel most comfortable.

The “pure” keyword isn’t really needed perhaps, but I threw it in to illustrate that feature of the language.  Fortran is very focused on parallel programming and this feature is essentially a compiler hint that the code in that procedure is deterministic and can be ran in parallel if necessary.  For a better description than a n00b like I can produce, take a look at this stack overflow post.

The last interesting bit in this code snippet is how I declared the return type from my function.  You can do it two ways, the traditional supply-the-type-before-the-name way (in which case you specify the return value by assigning your result to the name of the function in its body) or the way I did it here where I have a specific variable I declare to hold the result.  No preference, this just looks tidy to me.

By the way, it is probably worth pointing out that, unlike the C languages and their offspring, Fortran has two types of procedures – functions and subroutines.  A function returns a value and requires no special calling syntax and a subroutine returns no value but requires the use of the call subroutine_name() syntax to invoke.

The next thing we’ll talk about is how you build the equivalent of a class from the curly-brace languages.  That is done in Fortran by creating a type in a module.  Take a look at the following code skeleton:

module string_m
implicit none
private
type, public :: string_t
private
character(:), allocatable :: str_m
contains
private
procedure, public, pass(this) :: get_value => get_value_string_t
...
end type string_t
...
end module string_m

Ok.  There are many interesting things going on in this code.  In the type declaration, you see me creating a type called “string_t”.  The “_t” bit is just convention, it isn’t syntactically necessary.  The private variable “str_m” is an “allocatable” character array, or what I would call a “string”.  You also see a public procedure called “get_value” that is aliased to a local function I called “get_value_string_t”.

Finally the “pass(this)” part is some cool magic.  This is where you declare the equivalent of a “this” pointer in C++ that contains a reference to the object in who’s context your procedure is being invoked.  The slick thing is you can call it whatever you want so you aren’t tied to a particular name.  Plus (to me at least) the fact that you have to explicitly declare it takes away some of the “magic stuff” in C++.

To subclass this little gem, it is pretty straightforward.  Take a look at the following code skeleton to see how that happens in another module:

module ustring_m
use string_m, only : string_t
use string_utility, only : ucase
implicit none
private
type, public, extends(string_t) :: ustring_t
contains
procedure, public, pass(this) :: set_value => set_uvalue
...
end type ustring_t
...
end module ustring_m

OK.  Lots of stuff going on here.  For one, did you notice the new “use” feature I pulled out of the air?  If you say “, only :: ” and list a set of components, only those components are used from the included module.  This helps you avoid unintended name space collisions from crud that happens to live in a module.

You can see from the “extends” piece of syntax, that this is how you subclass your base class.  Finally, if you look at the procedure, you see that I am overriding the “set_value” procedure from the base class and mapping it to the local function that I named “set_uvalue”.

At this point, I knew enough to be dangerous and, as is my speciality, decided to soar over the tips of my skis and crash into the mountain – I had read about operator overloading and decided to create an overload of the equals operator!

Boom!

Well, that was painful.  Back to the comp.lang.fortran group for some healing and education.  I learned that, while you can overload operators in Fortran, what I wanted to do was overload “assignment”.  After a lot of back and forth with some incredibly patient people out there, I discovered the solution that makes all of this make sense and be pretty simple to do.

You declare the assignment operator in your base class and use the ability to alias the names of procedures in your subclass to overload the actual methods performed.  Here is how it looks in the base class:

type, public :: string_t
private
character(:), allocatable :: str_m
contains
private
generic, public :: assignment (=) => string_t_assign_string_t, &
string_t_assign_character
! Procedure declaration for internal methods
procedure, private, pass(lhs) :: string_t_assign_string_t, &
string_t_assign_character
end type string_t

Believe me, this looks way cleaner than the messes I created along the way to learning how to do it.  Oh, and did I mention that Fortran uses the exclamation mark (bang) for comments?  It feels slightly ironic to me like, “Holy crap!  I’m actually putting a comment in my code!  Can you believe it?”  I used to work with a guy who said that anyone not smart enough to understand his code by reading it shouldn’t be in there in the first place – that’s why he never commented it.  😉

So the interesting bits here are the use of the “generic” keyword.  This allows us to genericise what can be on the right-hand side of that assignment operator.  Very similar to generics in other languages.  Finally, notice that we have to declare the actual procedures that do the assignment (one from a string_t type and a second one from a character or character array) twice.  Once in the mapping and a second time for the actual declaration.

The implementation of the procedures is pretty straightforward:

elemental subroutine string_t_assign_character(lhs, rhs)
class (string_t), intent (inout) :: lhs
character(len=*), intent (in) :: rhs
lhs%str_m = rhs
end subroutine string_t_assign_character

The only piece of magic here is the use of the keyword “class” so that you can tell the compiler you might want to allow someone to override this procedure later in a subclass.  If you don’t want that, just use “type”.  Finally, I picked this method to show the implementation of because of the use of “elemental”.  This tells the compiler that you can pass a single character or an array of characters.

One thing that a good object needs is a constructor in C++ to initialize internal data, etc.  Figuring this out in Fortran was a little challenging.  What I ended up with is as follows:

module string_m
implicit none
private
type, public :: string_t
private
! Internal private character store
character(:), allocatable :: str_m
contains
private
end type str_t
! Class constructor
interface string_t
module procedure string_t_constructor
end interface string_t
contains
type (string_t) function string_t_constructor()
string_t_constructor%str_m = ""
end function string_t_constructor

So the trick here is to declare the interface, then actually define the function in the contains section of the module, returning a type of “string_t”.  I’m not certain if the “_constructor” is convention or required.

Now I can build a little test program to exercise my classes like this:

program fortranString
! Pulls in public interface for our String module
use string_m, only : string_t
use ustring_m, only : ustring_t
! Prevent default I-N integer assumption
implicit none
! Declare local variables
type (string_t) :: string1
type (string_t) :: string2
type (ustring_t) :: string3
string1 = "Bonjour"
print *, string1%get_value()
end program fortranString

As you can see, we can call the methods in the classes, use the assignment operator and all sorts of fun things like that!

So, in conclusion, Fortran is a pretty cool language.  The syntax is a little different that a curly-brace guy like me is used to, but once you figure it out, it’s pretty easy to use and has a very nice feature set. Again, if you’d like to look at a functional complete example, check out my source repository on GitHub.

I’m going to do a third post in this series where I actually build a modern web application using Fortran for the middle tier (I’m thinking I need a cool name like LAMP or BCHS so maybe FARM – Fortran, Apache, REST and mySQL?) but that’s for another day.  Hope you enjoyed reading this as much as I enjoyed learning it.

Have you seen FORTRAN lately?

I’m familiar with the FORTRAN programming language.  Heck, the first formal programming class I ever took was in 1978 in FOTRAN-IV on punched cards (don’t laugh kids, I’ll tell you to get off of my lawn if you do).

My next exposure to this venerable language was in college when I took my intro to CS course in the amazing, whizz bang “new” FORTRAN-77.  In that class, I got done with my final so quickly that I spent the rest of the time using VT-100 escape codes on the VAX-11/784 to animate a beer truck across the terminal – again written in FORTRAN-77.

My final trip to FORTRAN-ville was a year later in an algorithm analysis class where we had to solve the “Towers of Hanoi” problem using recursion in both FORTRAN-77 and Pascal.  The only catch – FORTRAN-77 didn’t support recursion so we had to build and manage our own stack.

All of this left me with the feeling that FORTRAN wasn’t a very exciting language and was quite old-fashioned.  Not too long after that, I learned C and my path to the dark side began.  I went from C to C++ and eventually to things like Java, Javascript and C# – quite happy in my little curly-brace world.  And by the way, the only true way to nirvana lies in putting the curly brace on a separate line from the IF statement now doesn’t it my disciples?!?!?!  Bwahahaha…

OK.  Fast forward to a couple of weeks ago.  I had been reading about the BCHS (pronounced “beaches”) web programming stack (see this page for more details) which essentially is using C for your middle-tier code with CGI to write web applications.  Now, regardless of whether or not this was serious or a joke (there seemed to be some debate on this on Reddit at the time), it got me thinking.

I remember CGI programming from the mid 90’s.  Essentially the idea is that you have a chunk of code running on the web server (be it a script or compiled code, it really doesn’t matter) that kicks out HTML for its output.  This allowed you (way, way, way back in the day) to access crazy things like relational databases and such to generate dynamic web pages and perform I/O with users.

I went down the path that said, “Hey.  If you can do this with C, why not other languages?”  I tried thinking what the most absurd, crazy, old-fashioned thing you could use and came with COBOL!  Of course!  That would be the funniest thing imaginable.  Unfortunately, even doing a multi-year stint as a mainframe CICS programmer back in the 80’s and 90’s, I never used COBOL (I used PL/I for those who were curious – a pretty cool language for its time actually).  This left me with FORTRAN – the language I knew from college and my brief punch-carded middle-school experience.

I did some sniffing around in the ports tree and discovered that OpenBSD has the GNU version of FORTRAN and it appears to be fairly recent.  I did some further investigation and hit a wall – we only support FastCGI with our httpd web server and surely that wasn’t the same thing as straight up CGI that I remembered.  After some searching, I confirmed my suspicion – FastCGI counts on a long-running process that the web server communicates with via sockets so that it doesn’t have the overhead of firing up and tearing down a process for each web transaction (boy they sure got smarter after the mid 90’s on this web stuff <grin>).

Not to be deterred, I decided to use the Apache web server from ports.  Installing that was pretty simple via pkg_add and turning on CGI access wasn’t that tough either.  All I had to do was uncomment the LoadModule call to the cgi_module, add a ScriptAlias to a virtual /cgi-bin directory (to contain my scripts outside of the htdocs tree) and add the ExecCGI option to my htdocs directory tree.

From there, I wrote a little shell script to test things and stuck it in my /var/www/cgi-bin directory as testcgi.cgi:

#!/bin/sh
echo "Content-type text/plain"
echo ""
echo "Hello world!"

I then invoked http://127.0.0.1/cgi-bin/testcgi.cgi and got the content “Hello world!” in my web browser.  Note the blank line after the content-type header – if you miss that, you will suffer with 500 errors until your web searching fu teaches you the error of your ways.

Now for the fun, let’s use FORTRAN to write our CGI script.  This should be crazy.  I can’t wait for all of the uppercase characters and sequence/line numbers in my xterm under vim.  I’ll fee like Indiana Jones in the Temple of Doom or something – a crazy technology archaeological expedition!  Off to wikipedia and the web to re-learn this language enough to write out strings.

Well, the first thing I learned was we are much more polite in this millennium than we were in the last – we don’t shout the name of the language any more.  FORTRAN (which stood for FORmula TRANslation) was now Fortran.  Also, there were new standards that were established after the 1977 version I was used to.  They had one in 1990, 1995, 2003 and 2008.  Apparently there was even a 2015 version undergoing standards ratification.  Huh.  Looks like people still use this crazy old thing.

Some more research turned up an interesting fact – 15% of the world’s software is written in Fortran.  I started looking at some of the language features that had been added over the years (more out of curiosity than for any other reason) and was shocked, nay horrified to find out that they added recursion.  That would have made my data structures class in college a heck of a lot easier!

I kept looking and discovered other interesting things.  For one, user defined types!  We didn’t have those in FORTRAN-77.  Also fun things like object oriented features, operator overloading, free-form formatting, case-insensitive intrinsic functions, generics…

Holy crap!  I suddenly felt like the prototypical old guy at the high school reunion who discovered that the nerd you remembered had grown up to be attractive and successful.  My world-view (the one that said all curly-brace languages were naturally superior to all others) was in jeopardy.

The only thing that could save me would be if our Fortran compiler on OpenBSD was too archaic for any of this funny-business to work.  Then I would be justified again.  My heart rate started to settle down to a more normal level at this thought.

I started sniffing around in the ports tree and discovered I already had most of what I needed installed.  With the latest (in our tree at least) gcc installed, all I had to do was add the g95 compiler via pkg_add.  I did so, and then tried invoking it with a simple ‘g95 –version’.

No joy.  No binary with that name on my system.  Huh?  I did a ‘find / -name g95’ and still came up dry.  I then did some web searching (notice I don’t say “googling” any more now that I’ve switched to duck duck go for my web searching) and discovered that, silly me, it isn’t called “g95” it is called “gfortran”.  OK, I tried that ‘gfortran –version’.

No joy.  No binary with THAT name on my system.  WTF?  After some more searching, I discovered that we call it “egfortran” on OpenBSD and all was right with the world.

I tried my little test program that I did before with a shell script, but this time did it in Fortran as testcgi.f and came up with:

program testcgi
print *, "Content-Type: text/plain"
print *
print *, "Hello World!"
end program testcgi

When I used the command-line ‘egfortran testcgi.f’ I was expecting to run a.out and move on with my life.  Nope!  All sorts of crazy error messages.

After some more searching, I found the ‘-ffree-form’ compiler flag.  It worked.  (Edit:  I later have learned that if you use a .f90 file suffix, the switch is not necessary.)  I copied the resulting a.out binary into my /var/www/cgi-bin directory and renamed it to testcgi.cgi and went back to my browser.  Drum roll please….

Fail!  I got a 500 error.  This really stymied me for quite some time until I picked up on a subtle thing.  When I ran my binary from the command-line, I suddenly noticed that every line of text was preceded with a blank space in column 0 (or would that be column 1 in a Fortran world?).  Anyhow, after some web searching I discovered that I could use a format specifier and all would be fine.  This left me with the following, which worked:

program testcgi
print '(a)', "Content-Type: text/plain"
print '(a)'
print '(a)', "Hello World!"
end program testcgi

So now that I have scratched that initial itch, I’m curious to see how far Fortran can take me with a more interesting problem to solve.  Look for an upcoming post that will delve into that.

Smaller is better

My home office (where my network and servers live) is a warm, noisy place.  So much so that I really wasn’t enjoying being in the room.  Since I don’t have an air conditioned datacenter with a raised floor in my house, I decided that I wanted to do something about this problem.  Interestingly enough, the biggest source of heat in the room turned out to be my two 24″ Apple Cinema Displays from circa 2006-2008.

One of the others was the little PC I built to be my OpenBSD router / firewall.  I did some research and discovered the APU2 board.  This little beauty is air cooled, about the size of a CD jewel case, and has 4 cores and up to 4GB of RAM with 3 gigabit Intel ethernet ports on it.  Sounds like a winner to me.

I acquired one of these from Mini-Box (http://www.mini-box.com/ALIX-APU-2C4-AMD-G-Series-GX-421TC?sc=8&category=754) along with the case and power supply.  I picked up a cheap mSATA 64GB drive for storage and spent about 3 minutes with a screw driver assembling the thing.  The end result was a small case about the size of three audio CD jewel cases stacked on top of each other.  After that, I popped a USB stick with OpenBSD 5.9-current on it and booted.

The system (which I was connected to with a serial cable to the serial port on the APU2 board) booted up, I saw the boot prompt for OpenBSD and then it booted up, I saw the boot prompt for OpenBSD, and then it booted up…  You see where this was headed.

I reached out for help on the mailing lists and very quickly had two folks clue me in.  When OpenBSD’s kernel starts to load, it looks for a console and if it can’t find one, it exits, creating this boot loop.  At the boot prompt, a few commands had be booting to the installer:

boot:  stty com0 115200
boot: set tty com0
boot: boot /bsd

I decided to do this router “right”.  Since it was just going to be a router and firewall and I wouldn’t run anything else on it, I wanted to go with a small attack surface so I chose to not install any of the X packages.  This turned out to be a “Bad Idea”(TM) since you can’t build any ports on the system if you don’t have X installed.  Since I use tarsnap and it can only be built from source currently due to the licensing model, I went back, installed the X packages from the installer and was good to go.  Make sure you disallow root logins over ssh (the default) as this is going to be internet facing in the end.

Once I logged in as root the first time, I needed to give my unprivelaged user some juice so I created an /etc/doas.conf file, allowing anyone in the “wheel” group to run commands as root (preserving their environment).

# echo "permit keepenv :wheel as root" >/etc/doas.conf

I then added my unprivelaged user to wheel, wsrc and staff and then logged out.  At this point, I shouldn’t need to log in as root at all.

# usermod -G wsrc <your user> (and so on)

Since I have my dotfiles in a github repository, I needed to now log in as my unprivelaged user and generate their SSH keys:

$ ssh-keygen -t rsa -b 4096 "<your email>"

I then copied the ~/.ssh/id_rsa.pub file’s contents and pasted it into a new SSH key in my GitHub account’s settings.  Now I’m good to clone my dotfiles repository and have it set up my environment the way I like it to be set up.  However, first I need to install git.

I temporarily export the PKG_PATH that I like to use:

$ export PKG_PATH=http://openbsd.cs.toronto.edu/pub/OpenBSD/5.9/packages/amd64

Then I install the git package:

$ doas pkg_add git

… and it didn’t work.  Ah!  I forgot to actually connect to the network after rebooting.  A simple:

$ doas ifconfig em0 up
$ doas dhclient em0

and I was good to go.  I added git and then cloned my repository:

$ git clone <your user>@github.com:/<your user>/<your dotfile repo>.git

Then I ran the shell script to set up my dotfiles, logged out and back in to pick up the changes and I was good to go.  Now I needed to actually configure this box as a router and firewall the way I like it.  To do this, I stand on the shoulders of giants and use the awesome OpenBSD FAQ.

The APU2 numbers its ethernet ports from left to right as you look at the back of the case (meaning that em0 is the port closest to the DB9 serial connector).  I’m using port 0 for my WAN interface and port 1 for my internal.  My goal is to use the third port as a private interface to a second setup just like this and use CARP to make it redundant.

So, for my router, I start by adding the following to /etc/sysctl.conf:

net.inet.ip.forwarding=1
net.inet.ip.redirect=0
kern.bufcachepercent=50
net.inet.ip.ifq.maxlen=1024
net.inet.tcp.mssdflt=1440

Then, we need to enable the dhcp daemon:

$ doas rcctl enable dhcpd
$ doas rcctl set dhcpd flags em0

Now I create this as my /etc/dhcpd.conf file:

option domain-name-servers 192.168.1.1;
subnet 192.168.1.0 netmask 255.255.255.0 {
option routers 192.168.1.1;
range 192.168.1.4 192.168.1.254;
}

Following the tutorial, we will be also using unbound for local DNS caching.  Enable it as follows:

$ doas rcctl enable unbound

Create a /var/unbound/etc/unbound.conf file like they did in the tutorial:

server:
interface: 192.168.1.1
interface: 127.0.0.1
do-ip6: no
access-control: 192.168.1.0/24 allow
do-not-query-localhost: no
hide-identity: yes
hide-version: yes
forward-zone:
name: "."
forward-addr: 127.0.0.1@40

Add dnscrypt-proxy from the packages tree and enable / configure it:

$ doas pkg_add dnscrypt-proxy
$ doas rcctl enable dnscrypt_proxy
$ doas rcctl set dnscrypt_proxy flags "-l /dev/null -R dnscrypt.eu-dk -a 127.0.0.1:40"

Finally, prevent your upstream ISP from changing your DNS resolution via DHCP:

# echo 'ignore domain-name-servers;' >> /etc/dhclient.conf

Now we need to set up the firewall.  I used this for my /etc/pf.conf file:

int_if="{ em0 em2 }"
set block-policy drop
set loginterface egress
set skip on lo0
match in all scrub (no-df random-id max-mss 1440)
match out on egress inet from !(egress:network) to any nat-to (egress:0)
antispoof quick for (egress)
block in quick inet6 all
block return out quick inet6 all
block return out quick log on egress proto { tcp udp } from any to any port 53
block return out quick log on egress from any to { no-route $broken }
block in all
pass out quick inet keep state
pass in on $int_if inet
pass in on $int_if inet proto { tcp udp } from any to ! 192.168.1.1 port 53 rdr-
to 192.168.1.1
pass in on egress inet proto tcp to (egress) port 222 rdr-to 192.168.1.2
pass in on egress inet proto tcp from any to (egress) port 2222

Finally, I added “noatime,softdep” to my /etc/fstab for my non-swapfile mount points because I’m running an SSD drive.  I also disabled the sound server to further reduce attack surface on this box:

$ doas rcctl disable dnsiod

Next, I set a static IP address for ethernet port 0 (em0) so that dhcpd could bind to it:

# echo 'inet 192.168.1.1 255.255.255.0 192.168.1.255' > /etc/hostname.em0

Finally, I told ethernet port 1 (em1) to get its IP address via dhcp from my upstream ISP’s cablemodem:

# echo 'dhcp' > /etc/hostname.em1

At this point, I did a reboot, plugged in my upstream router to ethernet port 1, plugged my home network into ethernet port 0 and tested to ensure everything was working the way I expected it to.

The one thing I did have to do was reboot each machine on the home network to pick up the new DHCP stuff.  It all worked like a champ.

Oh my God! I killed Kenny… er the VAX

Have you ever had one of those days where nothing seems to go right?  Like the fictional character from the old “Pebbles & Bam Bam” cartoon “Bad Luck Schleprock” where a cloud follows you around and everything you touch turns to… well crap?

I think I had a day like that yesterday.

It all started when I read a posting on Undeadly.org announcing that OpenBSD was dropping support for the VAX platform:

http://undeadly.org/cgi?action=article&sid=20160309192510

As I read the posting, especially after seeing the part where it said “after much internal discussion”, I groaned.  Literally groaned out loud.

You see, if we roll the clock back to January, I had picked up a dirt cheap VAXstation 3100 on eBay and was working on getting OpenBSD-current up and running on it.  When I looked at my favorite mirror, however, I couldn’t find packages in the “snapshot” subdirectory for the VAX architecture.

Being the n00b that I am, I posted to the misc@ mailing list and asked the question that I fear started this whole snowball rolling:  “I can’t find -current packages for the VAX, are we dropping support in 5.9 for it?”

https://marc.info/?l=openbsd-misc&m=145357927822796&w=2

Really quickly, someone answered back that we weren’t building packages on that platform because it took so long and that we typically build them near the end of the release cycle.  No big deal, I went on with my day.

Well, if you follow the thread (and I clearly wasn’t doing so), it goes on until we see Theo weigh in sometime later and… I fear I was the guy who accidentally kill the VAX support in OpenBSD.  I’m like the people who keep killing Kenny on South Park.

RIP OpenBSD VAX