Sometimes, processors need love too

I have a confession to make.  As a person who has had a passion for computers and programming since the late 70’s, I have spent a lot of time on some really fun, albeit old, platforms.  For example, I started out on a Cyber 171 programming in BASIC and Pascal, then “graduated” to a TRS-80 model 1 doing BASIC as well.

When I was able to buy my first computer, I went with the Apple ][+ (and yes, that’s the right way to spell it!) and eventually upgraded to an Apple //e.  Once I realized the 32-bit revolution was on, I transferred my allegiance to the Commodore Amiga 1000, then the A2000 with its ability to install expansion cards.  Into one of those lovely slots so reminiscent of my old Apple ][+ went a Bridgeboard.  First the 80286 version (on which I ran DOS) and then finally the 80386sx version that I ran Windows 3.x and OS/2 on.

Along the way, I worked on a VAX 11/784 (and was a system administrator) where I learned such useful things as VT100 escape codes that would allow me to write programs that did simple animations on the screen.  I lovingly recall finishing my FORTRAN-77 final early in college and spending the extra time writing an animated beer truck that chugged across the screen and had a little guy get out and dance at the end.  (Of course I turned in both programs to the professor!)

I cut my teeth in the UNIX world working on SunOS 4 on a variety of Sparcstations, as well as logging in remotely to a MicroVAX running Ultrix (as I recall).  That was where I first discovered that these machines were connected over a global network and I could bounce from machine to machine.  Welcome to the Internet!

I finally sold out and went down the dark path of C/C++/VB/C# software development on the Windows platform (starting with Windows 3.x and then moving to Windows NT 3.1 beta).  After many years on that platform, my wife bought me an iPod for Christmas.  As I was thinking to myself “she doesn’t know this won’t work on Windows” she said, “This is the new version that works on Windows.”  (Not the first time she has proven herself to surpass me, nor will it be the last!)

That iPod was the gateway drug that got me to try out a 32-bit PPC Mac Mini.  It was simple to use and pretty useful.  I could even run Windows on it in an emulated (aka slow) mode.  I then discovered that it had UNIX under the hood and my spinal cord started typing things that surprised me at the terminal prompt from my old SunOS 4 days.  When I heard Steve Jobs announce the move to Intel, I told myself that this was the end of the line for the Mac.  Just to be a “kind soul”, I enrolled in the Intel Mac beta program and received a G5 tower that had nothing in it but a tiny little PC motherboard and a running copy of OSX 10.4 on that architecture.

Given that time has proven me wrong there as well (never take a stock tip from me BTW), I was knocking around on various Intel-based Mac desktops and laptops ever since.  Then, I discovered OpenBSD and fell in love.  One of the things I discovered was that it ran on a variety of platforms.  Not like our friends over at NetBSD do (“my toaster runs NetBSD!” <grin>), but they were tier-1 platforms nonetheless and many of them were my old friends (sparc64, ppc32, VAXen, etc.)

Since I had that 32-bit PPC Mac Mini in the closet and an old i386 laptop, I tried installing OpenBSD on these two “exotic” architectures and found it to be drop-dead simple.  I actually had to set the hostnames and the prompt in ksh to remind me I was on a different architecture.

When browsing eBay and finding a SunBlade 100 there for under $30, I couldn’t resist and pulled the trigger.  After a brief amount of fiddling, I had a sparc64 processor in my stable now too!  This was when I noticed something was amiss.  Many of these other platforms had much fewer ports/packages available for them, which makes them marginally less useful.

Given this observation, and an email response I received on the HPPA mailing list from none other than Theo DeRaadt himself where he told me that some of these platforms just need “some love”, I decided this might be a worthy quest for someone who now has a few ports under his belt.  But where to start…

I decided that polishing up my ksh scripting skills might make this more fun so I first set my sites on getting a sqlite3 database with all of the ports tagged for each platform.  I used curl to pull down the list of packages from one of the mirrors, jammed it through a sed script to extract just the package name and used some more sed magic to turn it into an insert statement for sqlite3.  Now I have a database of all of the ports for each platform.  What to do next?

I decided to do some sniffing around to find out how I could determine the dependencies for each port because I discovered that many of these ports are missing on the platforms because their dependencies aren’t there.  Knowing the interrelationships of the ports themselves would make my job easier / more fruitful.  How could I determine this though?

After exercising my Google-fu, I found out that you can use the pkg_info with a ‘-f’ flag to list the dependencies for a given port.  Some chaining of that with grep and cut and voila – I had a little script that, given a port name, could spit out the dependencies for me.

I then wrote a third script to iterate over all of the distinct port names from my sqlite3 database and iteratively call my dependency finding script, storing the results in a second table in my sqlite3 database.  Now I have all of the ports by platform, and a list of the (at least top-level) dependencies for each port.

Given that, it was a simple matter to create an ugly query to give me a text file for each processor of which ports were missing from that architecture, along with the number of dependencies each port had, ordered to make the simple ones (zero dependencies) sort to the top of the list.

Where does that leave me?  Well, my marching orders are pretty clear.  I need to start plowing through some of these ports and see what the collateral impact of this work is in terms of getting more parity in the ports trees between the various architectures.

If you have read this and are interested in the scripts I created, I’ll point you towards my github project for this.  The only caveat is you can’t mock my sketchy SQL statements or even sketchier scripts:

http://github.com/bceverly/openbsd-ports-db

That’s all for now.  Wish me luck as I start giving some love to these old friends.