Thinkpad T14 (AMD) Gen 2 – A Brave New World!

As long-time readers of this blog are aware, I’m a bit of a Thinkpad fanatic. I fell in love with these durable machines when I was working for IBM back in the late 90’s and accidentally had one fall out of my bag, bounce down the jetway stairs and hit the runway hard – amazingly enough it had a few scuffs but zero damage! After the purchase of the brand by Lenovo, I was a bit worried, but they continue to crank out (at least in the Thinkpad T and X model lines) high-quality, powerful machines.

Thinkpad T480 – RIP

I ran into a nasty problem with my Thinkpad T480 where the software on the machine actually physically damaged the hardware. I know! I thought that was impossible too (other than the 70’s PET machine that had a software-controlled relay on the motherboard that you could trigger continuously until it burned out) but nope – the problem is real.

Essentially, the Thunderbolt I/O port on the machine is driven by firmware running out of an NVRAM chip on the motherboard that can be software-updated as new firmware comes out. As with any NVRAM chip, there are a finite number of write-cycles before the chip dies, but the number of times you will update your firmware is pretty small so it works out well.

Unfortunately, Lenovo pushed out a firmware update that wrote continuously to the NVRAM chip and if you didn’t patch fast enough (they did release an urgent/critical update), then the write-cycles would be exceeded, the chip would fail and the bring-up code would not detect the presence of the bus and thus you had no more Thunderbolt on the laptop. Well, I didn’t update fast enough so “boom” – it is now a Thunderbolt-less laptop.

The New T124 (AMD) Gen 2

Well, enter the need for a new laptop. I decided to jump ship from the Intel train and try life out on the “other side” but ordering a Thinkpad T14 (AMD) Gen 2 machine with 16gb of soldered RAM (there is a slot that I will be populating today that can take it up to 48gb max – I’m going with 32gb total by installing an $80 16gb DIMM) and the Ryzen Pro 5650U that has 6 cores and 12 threads of execution. The screen was a 1920×1080 400 nit panel and looks really nice.

When the laptop showed up, I booted the OpenBSD installer from 6.9-current and grabbed a dmesg and discovered that I lost the Lenovo lottery and had a Realtek WiFi card in the machine. Well, the good news was that I had upgraded the card in my T480 to an Intel AX200 so I swapped it for the one I took out of the T480 and then used it in the T14 to replace the Realtek card. Worked like a charm.

The Ethernet interface on this machine is a bit odd. It’s a Realtek chipset as well, but it shows up as two interfaces (re0 and re1). The deal is that re0 is the interface that is exposed when the machine is plugged into a side-connecting docking station and re1 is the interface that is connected to the built-in Ethernet port. The device driver code that is in 6.9-current as of this writing works just fine with it, however, so I’m happy.

Now for the bad news. Every Thinkpad I have owned for the last decade allows me to plug an m.2 2240 SATA drive into the WWAN slot and it works great. I assumed that would be the case with this machine. While I had the bottom off to replace the WiFi card, I slipped the 1TB drive from the WWAN slot of my T480 into the WWAN slot of the T14 and booted up. I was immediately presented with an error message stating effectively that the WWAN slot was white-listed by Lenovo and would only accept “approved” network cards. I was beyond frustrated by this.

Given that I want to get this machine into my production workflow, I decided that I’d slog along for the time being by putting a larger m.2 2280 NVMe drive in, installing rEFInd to allow me to boot multiple partitions from a single drive and then clone the 512gb drive that is in the machine to the 1GB drive out of the T480. Then, the remaining space on the new drive will contain an encrypted partition for my OpenBSD install.

Installing rEFInd

I followed the instructions from the rEFInd site on how to manually install under Windows 10 and the steps I followed included downloading and unpacking the ZIP file and then running the following commands from an administrative command prompt:

C:\Users\xxxx\Downloads\refind-bin-0.13.2\> mountvol R: /s
C:\Users\xxxx\Downloads\refind-bin-0.13.2\> xcopy /E refind R:\EFI\refind\
C:\Users\xxxx\Downloads\refind-bin-0.13.2\> r:
R:\> cd \EFI\refind
R:\EFI\refind\> del /s drivers_aa64
R:\EFI\refind\> del /s drivers_ia32
R:\EFI\refind\> del /s tools_aa64
R:\EFI\refind\> del /s tools_ia32
R:\EFI\refind\> del refind_aa64.efi
R:\EFI\refind\> del refind_x64.efi
R:\EFI\refind\> rmdir drivers_aa64
R:\EFI\refind\> rmdir drivers_ia32
R:\EFI\refind\> rmdir tools_aa64
R:\EFI\refind\> rmdir tools_ia32R:\EFI\refind\> rename refind.conf-sample refind.conf
R:\EFI\refind\> mkdir images
R:\EFI\refind\> copy C:\Users\xxx\Pictures\mtstmichel.jpg images
R:\EFI\refind\> bcdedit /set "{bootmgr}" path \EFI\refind\refind_x64.efi

That next to the last line is because I wanted to have a picture of my “happy place” (Mount Saint Michel off of the northern coast of France) as the background for rEFInd. I edited the refind.conf file and added the following lines:

banner images\mtstmichel.jpg
banner_scale fillscreen

A quick reboot shows that rEFInd is installed correctly and has my customized background. Don’t be alarmed that the first time you boot up with rEFInd is slow, I think it is doing some scanning and processing and caching because the second and subsequent boots are faster.

Cloning the Drives

The process that I am going to follow, at a high level, is to first clone the contents of my primary 1TB 2280 NVMe drive in my T480 to a spare 256GB drive. I will then erase the 1TB drive and clone the contents of my T14’s drive to it (it’s only 512GB). I will then erase the 512GB drive and clone the 256GB drive back to it. Finally, for good operational security (OpSec) purposes, I’ll use the open source Windows program Eraser erase the 256GB drive. At this point I should have a bootable T480 (with a fried Thunderbolt bus – grr…) on the 512GB drive, and a bootable T14 on the 1TB drive.

I’m using Clonezilla, an open source tool that I burn to a bootable USB drive to do the cloning. For hardware that I am using to accomplish all of this, first I use a Star Tech device that allows me to plug m.2 drives into a little box that then acts as a 2.5 inch SSD drive. I plug that into a Wavlink USB drive docking station that can hold either 3.5″ or 2.5″ drives.

Another piece of software that I use as part of this process is GPartEd Live – an open source tool that allows you to create a USB drive that boots into the GPartEd software (the Gnu Partition Editor). This allows me to view the partition structure of one drive and create an analagous partition structure on another drive. The built-in tools for Windows to do this work (Disk Manager for example) can create hidden partitions under the covers that can cause problems with this process. I prefer to use GPartEd to ensure that I can see and control everything that is going on.

Step One is to take the T480, boot it into Windows and connect the Wavlink device to it with the 256GB NVMe drive plugged into it via the StarTech adapter. While I’m using Eraser to wipe the 256GB drive, I also go into Windows settings and decrypt the Windows disk by turning off BitLocker for it. This may not be necessary but it makes me feel more comfortable to do the cloning with unencrypted Windows drives because the key for the encryption is store in the TPM device on the motherboard and I’m not sure if the fact that the underlying hardware changes would muck that up. After the erase and decrypt is finished, I shrank the partition using “Disk Management” on Windows to be smaller than the new physical disk. If you don’t do this, then Clonezilla won’t allow you to clone from a larger partition to a smaller one.

Next we will need to reboot the machine to GPartEd Live. For the destination drive, you will need to use the “Device” menu and create a new GPT partition table. Take a look at the source drive and make a note of the various partitions, their flags, and their sizes. On the destination drive, recreate that partition structure with the same flags and the same or slightly larger size. I generally bump up the size of the partition by just a bit in order to avoid getting into trouble with rounding the size for display on the screen. If you get it wrong, don’t worry, Clonezilla will yell at you and you’ll have to go back and do this over again. 🙂

When launching Clonezilla, since I have the high resolution display on the T480 (a mistake I’ll never make again, HiDPI is a PITA in everything but Windows) I had to use my cell phone to zoom in on the microscopic text and select the “use 800×600 with large fonts from RAM” option. With readable text, I then make sure that I’m choosing “device-device” from the first menu (not the default). Next, select “Beginner Mode” to reduce the complexity of the choices you’ll have to make. After that, you want to select “part_to_local_part” to clone from one partition on the source drive to the corresponding partition on the destination drive. Finally, select the source partition and the destination partition. I recommend you do the smaller partitions first and then let the main C: partition (the largest one) grind because it can take a long time to clone.

After cloning the T480 drive, I removed it from the machine and was ready to clone the T14’s drive to it. This is where I ran into a “keying” problem with m.2 drives. Some are “B” keyed, and some are “B+M” keyed. This refers to the number of cutouts where they plug into the slot. Well, it looks like the NVMe drives in both the T480 and the T14 don’t fit the StarTech adapter. After some juggling around I found an old 256MB drive that I was able to use to get the swap completed.

Creating the OpenBSD Partition

To do this, I will use “Disk Manager” on Windows and shrink the NTFS partition (if necessary) to make room for OpenBSD and then create a new partition on the drive that takes up the remaining space. If you check the “don’t assign a drive letter” box and the “don’t format the partition” box, you’ll get a raw, unformatted partition that takes up the remaining space on the disk.

That new raw partition will be changed in OpenBSD to be the home of the encrypted slice on which I’ll be installing the operating system. After creating that partition, it’s time to download the 6.9-current .IMG file for the latest snapshot and use Rufus on Windows to create the USB drive and reboot from it.

Once in the OpenBSD installer, drop immediately to the shell and convert that NTFS partition into an OpenBSD partition. That will be where we we put the encrypted slice that we will be installing to. To do this, run the following commands:

# cd /dev
# sh ./MAKEDEV sd0
# fdisk -E sd0

sd0: 1> print
sd0: 1> edit 4
Partition id: A6
Partition offset <ENTER>
Partition size <ENTER>Partition name: OpenBSD
sd0*: 1> write
sd0: 1> exit

The print command above should show you the 4 partitions on your drive (the EFI partition, the Windows partition, the WindowsRecovery partition and your fourth partition that will hold OpenBSD that you created above).

Now that you have a partition for OpenBSD, you’ll want to copy the EFI bootloader over to your EFI drive. You’ll later make a configuration change in rEFInd to not only display it on the screen, but also show a cool OpenBSD “Puffy” logo for it!

# cd /dev
# sh ./MAKEDEV sd1
# mount /dev/sd1i /mnt
# mkdir /mnt2
# mount /dev/sd0i /mnt2
# mkdir /mnt2/EFI/OpenBSD
# cp /mnt/efi/boot/* /mnt2/EFI/OpenBSD
# umount /mnt
# umount /mnt2

Now that you have an OpenBSD EFI bootloader in its own directory on the EFI partition, you’ll want to create the encrypted slice for the operating system install:

# disklabel -E sd0

sd0> a a
sd0> offset: <ENTER>
sd0> size: <ENTER>
sd0> FS type: RAID
sd0*> w
sd0> q

# bioctl -c C -l sd0a softraid0
New passphrase: <your favorite passphrase>
Re-type passphrase: <your favorite passphrase>

Pay attention to the virtual device name that bioctl spits out for your new encrypted “drive”. That’s what you will tell the OpenBSD installer to use. To re-enter the installer, type “exit” at the command prompt. Do your install of the operating system as you normally do. When you reboot, go into Windows.

First, download an icon for OpenBSD from here (or pick your favorite elsewhere). Next, bring up an administrative command prompt and use the following commands to mount the EFI partition and add the icon for OpenBSD:

C:\Windows\system32> mountvol R: /s
C:\Windows\system32> r:
R:> cd \EFI\refind
R:\EFI\refind> copy "C:\Users\<YOUR USER>\Download\495_openbsd_icon.png" icons\os_openbsd.png

Save your changes, exit notepad and then reboot. rEFInd is smart enough to find your OpenBSD partition and use the icon you just added. When you select it from the rEFInd UI, you should be presented with your OpenBSD encrypted disk password and be able to boot for the first time. I ran into a weird thing with my snapshot where it couldn’t download the firmware. I formatted a USB thumb drive as FAT32, downloaded the amdgpu, iwx, uvideo and vmm firmware from the site, mounted the drive in my OpenBSD system and ran fw_update -p /mnt to get the firmware.

At this point, you should be able to reboot and select either Windows or OpenBSD from your rEFInd interface. My hope is that Lenovo will remove this absurd white-listing of the WWAN devices from their UEFI/BIOS code and I’ll be able to plug drives into it again; however, if (and this is more likely) they do not, I’ll at some point buy a 2TB m.2 NVMe drive for this machine, repeat this process and be able to add Linux to it.

I hope folks find this guide helpful.

OpenBSD 6.9 – Help with the “Failed to install bootblocks” issue

Hi everyone!

I purposely chose a non-catchy title so that it would be more easily found by the search engines as this one has been a challenge for me in my last several laptop installs and I always manage to fix it after fiddling around for a while. This time around, I thought I’d actually produce a decent (hopefully!) write-up on just how I go about addressing the problem from scratch. This will provide two benefits: 1) I’ll have a nice step by step the next time I install my machine <grin>; and 2) It might help some other intrepid soul who is running into the same issue!

While the FAQ is always the best place to go for the most up to date steps on formatting and installing a system, I tend to run a “weird” setup that it seems like confounds the installer and most easily-accessible information. What I normally do in my Thinkpad laptops is install a second (or third) SSD or NVMe drive and then dedicate the entire disk to a given operating system. For example, if I’m running Windows 10 and OpenBSD 6.9 on my Thinkpad T480, I install Windows on the first drive (so that if my machine falls into evil hands and they power it on, it will just default boot into Windows and they might not even suspect OpenBSD is on the machine) and then I install OpenBSD onto the second drive. I then use the UEFI or BIOS boot menu to choose the OpenBSD drive to boot from.

Install Windows

I started off by installing Windows from a USB key to the primary drive in the laptop. As is my custom, after install, I put on all of the drivers and used the group policy editor to increase the BitLocker encryption from 128-bit AES to 256-bit AES. I also edited the registry to allow Outlook’s OST file to expand beyond the pitiful limit that it defaults to. After a reboot, I start the BitLocker encryption process and connect my email accounts.

If you are installing OpenBSD on a drive that has previously had something on it, it’s always a good idea to erase that drive. I use an open source tool for Windows called Eraser if I’m on Windows or good old dd if I’m on Linux. Eraser’s UI is a bit weird. It requires that you create a task that you can “run manually”, select the disk to be erased (in my cased “Hard disk 1”) and then select an erasure method (I use Pseudorandom 1-pass), then run the task manually.

I then download the install69.img file from my favorite mirror (https://openbsd.cs.toronto.edu/pub/OpenBSD) and use Rufus to transfer it to a bootable USB drive. I reboot, hit <F12> to get a boot menu from the UEFI, select my USB drive and then boot into the OpenBSD installer.

Install OpenBSD

The first thing I do is look at my dmesg to see what devices my drives have been attached to:

# dmesg | grep -i sd

This shows (in my case) that my Windows drive is connected to sd0, my blank drive that I will put OpenBSD on is connected to sd1 and my USB installer device is connected to sd2. Next, I need to create the necessary /dev devices:

# cd /dev
# sh ./MAKEDEV sd1
# sh ./MAKEDEV sd2

If you do a quick ls, you should see that the MAKEDEV script created the necessary device files and you should be good to proceed to the next step. Next, we want to initialize the sd1 drive to a GPT partitioning scheme and create the initial EFI partition on the disk. Fun fact, the EFI partition (while its own partition type) is formatted using FAT32 so thanks Windows 95! Here’s how you do this:

# fdisk -iy -g -b 960 sd1
# newfs_msdos /dev/rsd1i

Note my use of the /dev/r device (the raw device) and not the /dev/sd1i (normal device) in that second command. I’m not entirely sure if that is necessary, but the nice Reddit post that sparked me to think about how to do this did so why not, eh? If you get a weird error message trying to run newfs_msdos, it is likely that you have some previous partitioning data on that drive and it would be a good idea to completely erase it (see above).

Now, we need to mount the new partition, create the necessary directory structure that UEFI looks for and put the UEFI loader file from our installer USB drive into that directory:

# mount /dev/sd2i /mnt
# mount /dev/swd1i /mnt2
# mkdir -p /mnt2/efi/boot
# cp /mnt/efi/boot/* /mnt2/efi/boot

Now, we need to create the slice in the OpenBSD partition for the encrypted filesystem (you can skip this if you want to not have an encrypted drive):

# disklabel -E sd1

a a [ENTER]
offset: the default given
size: *
type: RAID
w [ENTER]
q [ENTER]

At this point, we have a slice set up as type “RAID” so we need to use the bioctl program to set up the encryption information along with the drive’s encryption password:

# bioctl -c C -l /dev/sd1a softraid0

You should see in the response to the above command the name of the new “virtual” encrypted disk. That is the disk that you will be installing OpenBSD onto. When you reach the question in the installation program about “Which disk is the root disk?”, enter that value (in my case, sd4). When i tasks whether or not you want to “Use (W)hole disk MBR, whole disk (G)PT or (E)dit?”, pick the MBR option (I know, this is counter intuitive but trust me here).

After the installer reboots the system, I press the [F12] key to get the boot menu (your key might be different if you aren’t running a Thinkpad) and select the disk I have installed OpenBSD on. I am immediately presented with the password prompt to decrypt the encrypted slice “virtual” disk and, upon entering it, I get the boot prompt. Everything proceeds as normal from that point forward and I am presented with the login prompt for my new system.

Updated Laptop Setup

If you are still with me and want to see how I set up my OpenBSD desktop (I get criticized slightly for making it “too heavy” with “too many packages” but I have to use Ubuntu as well for what I do and I like to have the UI be as consistent across the two operating systems as I can. Therefore I install Gnome 3 along with some gnome tweaks and plugins that give me the same theme and dock as Ubuntu.

To start out, I log in as root and enable my user account:

# echo "permit persist keepenv [my_non_root_user] as root" > /etc/doas.conf

At this point, I log out and back in as my unprivileged user account and work from there using the doas command to escalate privileges when needed. I start out by updating my system:

$ doas syspatch

Now, set up power management (this is a laptop):

$ doas rcctl enable apmd
$ doas rcctl set apmd flags -A
$ doas rcctl start apmd

I also add the following line to /etc/rc.conf.local (I haven’t cracked the code on how to do this with rcctl yet):

ntpd_flags=""

Now I need to make sure that I have the right level of resources available to my non-privileged user for tools like nextcloudclient (which opens a TON of files during its synchronization process). To do this I typically put myself in the “staff” and “operator” groups:

$ doas usermod -G staff MY_USERNAME
$ doas usermod -G operator MY_USERNAME
$ doas usermod -L staff MY_USERNAME

I then make the following changes to the “staff” section in /etc/login.conf:

...
staff:\
  :datasize-cur=4096M\
  :datasize-max=infinity\
  :maxproc-max=512:\
  :maxproc-cur=256:\
  :openfiles-max=102400:\
  :openfiles-cur=102400

I then have to add a line to /etc/sysctl.conf to take complete the work on allowing more open files on this system:

kern.maxfiles=102400

Now that I have modified all of this stuff and patched the system, it’s a good time to reboot.

Next, I add all of the packages I can’t live without (I know it seems like a small list, but they pull in a lot of others):

$ doas pkg_add gnome gnome-tweaks gnome-extras firefox chromium libreoffice nextcloudclient keepassxc \
   aisleriot evolution evolution-ews tor-browser shotwell gimp vim colorls cups reposync

A few changes to /etc/rc.conf.local are needed to boot into Gnome3:

$ doas rcctl disable xenodm
$ doas rcctl enable multicast messagebus avahi_daemon gdm cupsd

To avoid taking a kernel panic in my use-case (I have multiple monitors connected through a Lenovo Thunderbolt/USB-C dock), I have to manually switch to the Intel DRM driver in my /etc/X11/xorg.conf by adding the following section:

Section "Device"
  Identifier "Intel Graphics"
  Driver "intel"
EndSection

At this point, it’s time to reboot and go into GUI land. If you run into a situation where you have a monitor mirrored and no way to turn that feature off, I have found that turning all of the monitors off and back on generally fixes things. Once I have everything the way I would like it, I then download the yaru-remix-complete theme and install it manually by doing this:

$ cd ~
$ mkdir .themes
$ cd .themes
$ mv ~/Downloads/yaru-remix-complete-20.04.tar.xz .
$ unxz yaru-remix-complete-20.04.tar.xz
$ tar xf yaru-remix-complete-20.04.tar
$ mv themes/* .
$ rmdir themes
% doas mv icons/* /usr/local/share/icons
$ rmdir icons
$ doas mv wallpaper/* /usr/local/share/backgrounds/gnome
$ rmdir wallpaper
$ rm yaru-remix-complete-20.04.tar

Now launch gnome-tweaks and from the “Extensions” tab, turn on “user-themes”. Restart gnome-tweaks, go to the “Appearance” tab and select “Yaru-remixt” for applications, icons, and shell. On the “Top Bar” tab, enable “Battery Percentage” and “Weekday”. In the “Window Titlebars” tab, enable “Maximize” and “Minimize”.

Next, we want to put the wonderful extension Dash-To-Dock into the environment. To download it, go to https://extensions.gnome.org/extension/307/dash-to-dock/ and pick the right sehll version and extension version to match your install of Gnome shell. You will have to manually install it because the Gnome shell extension integration doesn’t appear to be enabled for OpenBSD:

$ cd ~/Downloads
$ unzip dash-to-docmicxgx.gmail.com.v67.shell-extension.zip
$ cat metadata.json

The value for “uuid” in that file is what you want to use in the next step:

$ mkdir -p ~/.local/share/gnome-shell/extensions/dash-to-dock@micxgx.gmail.com
$ cd ~/.local/share/gnome-shell/extensions/dash-to-dock@micxgx.gmail.com
$ unzip ~/Downloads/dash-to-docmicxgx.gmail.com.v67.shell-extension.zip

At this point, reboot to pick up the changes you’ve made, log in and launch gnome-tweaks again. On the “Extensions” tab, enable dash to dock. From the settings gear icon, select “extend to edge” and “show on all monitors” and you should have a very serviceable dock that is quite similar to the one in Ubuntu.

I then switch the terminal to “White on Black” for a better look and a 16-point font, and pin my favorite apps to the dock. Now for some terminal-level tweaks. I typically edit my ~/.profile file and add a couple of things:

export PS1="\[033[01;32m\]\u@\h\[\033[00m\]:\[033[01;34m\]\w[\033[00m\]$ "
export ENV=$HOME/.kshrc
export CVSROOT=/home/cvs

I then edit the ~/.kshrc file to add some aliases:

alias ls="colorls -G"
alias vi="vim"

A couple of other changes I typically make include turning off suspend when I’m plugged in (Settings | Power | Automatic Suspend), setting Firefox as my default browser (Settings | Default Applications), and setting my Time Format to “AM/PM” instead of “24-hour” (Settings | Date & Time).

I also take a moment to switch to “View -> User Interface -> Tabbed” in the Write, Calc, and Present applications in LibreOffice. This gives an interface reminiscent of the one in Microsoft Office – which I find helpful in terms of standardizing my workflow across operating systems.

After installing the appropriate browser security plugins and configuration changes from my favorite https://privacytools.io site, it’s time to set up CVS on my system for development purposes. To do this, I always double-check the AnonCVS link from the OpenBSD website left navigation panel and follow the steps to:

  1. Pre-load the source tree (for src, sys, ports and xenocara)
  2. Follow the instructions to give your non-root user write access to the src, ports and xenocara directories
  3. Mirror the repository with reposync (Note: I have had the best luck using anoncvs.comstyle.com as my mirror)

I then typically will add a crontab entry to keep things in sync:

$ doas crontab -e

...
0    */4    *    *    *    *    -n su -m cvs -c "reposync rsync://anoncvs.comstyle.com/cvs /home/cvs"

After syncing up my NextCloud data and my email data, I now have what I consider to be a secure, fully-functional OpenBSD laptop, configured the way I like it.

Let’s Talk Password Vaults

When “civilians” asks me what the most important thing they can do to protect the security of their home computers, I always answer the same way – make sure you patch and do so automatically! However, as Windows 10 finally has started defaulting to this behaviour (and they seem to be taking security way more seriously at Microsoft these days), my next favourite recommendation for folks is that they invest energy in a password vault.

For the uninitiated, a password vault is a piece of software that stores the passwords you use on various services and then encrypts them with a master password so that they are safe. “Since I use password123 as the password for all of the sites I visit, why would I need that”, you might be saying. Arrrrggggghhhh!!!!

You should use a unique, long, complex and randomly-generated password for every site you visit! How can anyone who is not superhuman do that? Well, it’s a bit circular but see above – a password vault. The good ones will even help you generate passwords and give you a health report on the ones you store in it, indicating that they might not be long enough, etc.

Ah, but my fellow paranoids might be thinking that this puts all of your eggs in one basket. And if you store them in the cloud (someone else’s computer) then OMG! Doomsday scenario! Well, I have a plan for that (no, I’m not secretly Elizabeth Warren)! Be all self-hosted with it!

So, how do I recommend setting things up? First things first, you need a place to store the password file. You could put it on your local hard drive but that would make it difficult to use it across your multiple devices (most everyone has a smart phone these days and you want to be as secure on that device as you are on your home computer). As I always recommend, put that file on a server in a country that has strong privacy laws and isn’t part of the dreaded Fourteen Eyes. Switzerland is a good choice and there are Swiss owned VPS providers who will give you a small virtual server for a reasonable monthly fee.

I recommend the open source project “NextCloud” as a good self-hosted service to run for this purpose. It is incredibly flexible and has a very active community around it creating all sorts of plugins, etc. You can buy space on a public NextCloud server but that would defeat the whole purpose of having the control of the server yourself and putting it in a country that is safer. There is a great tutorial on DigitalOcean for setting up NextCloud on an Ubuntu LTS release that I’d recommend you reference. While you are at it, take a look at their “initial Ubuntu server setup” for some other security recommendations. Add to it a LetsEncrypt certificate with automatic renewal set up and you have a pretty decent platform for storing your files.

OK. There are two ways you can get your password file to/from the server. You can either share it directly from your NextCloud server using WebDAV or you can just install the NextCloud desktop client software (available for pretty much every operating system) to sync a local folder with a folder on your NextCloud server. Typically I use the sync solution for desktops/laptops and the WebDAV solution for my mobile devices.

Now, you have a place to store files but what file are you going to store there? More specifically, what password vault software do I recommend if you want to go the self-hosted route. Well, I really really really like KeePassXC as my password vault software and file format of choice. It’s well-written, free, open source, what isn’t there to love about it!

To set it up, install the software on your desktop/laptop and create a new password database in your directory you are syncing to NextCloud. Make sure you pick a complex, random, long password that you can remember without writing it down as the master password for the vault. If you want to get even more secure, check out the Yubikey option for two factor authentication for your vault. You can also set up the browser extension for it if you want the convenience, but keep in mind it does increase your attack surface so you might just want to go old school and copy paste the credentials from the KeePassXC client software.

For Android and iOS, I use the app “Strongbox” and, as I mentioned above, use WebDAV over https as the way I read and write the file from my NextCloud server. The end result is that I have a single, secure password file that, even if my NextCloud server is compromised, is encrypted and would be a nightmare to try and hack your way into given the length, complexity and randomness of my master password.

KeePassXC has some really nice features you probably want to start leveraging right away. It has a great random password generator so that you can create crazy complex passwords that are unique for each service you use. In addition, it has a “Health Check” report that you can run to check up on your stable of credentials to make sure you aren’t re-using any of them or have some that are not complex enough.

In addition there is an integration with “HaveIBeenPwned” that allows you to check to see if any of the credentials you use have been exposed in a data breach. It does so by sending a secure, cryptographic hash of part of your password to the service so your risk is minimal other than your IP address being exposed to the service. All in all, I trust the author of the service and think it’s a great thing to do periodically.

Finally, I recommend taking a look at the security settings in your KeePassXC client or your Strongbox app. There is a feature that clears your clipboard and logs you out of the application after a period of inactivity. That’s literally the first thing I turn on when I install either of the applications because it keeps you from having your device stolen and being logged into the most secure thing you probably have.

All in all, I hope you enjoyed this post. I really do think that password vaults are an incredibly important development in the field of cybersecurity and would encourage everyone to use them, even if you want to go with a commercial one that you don’t have to self host.

Fast Follower – Even More Privacy Centric DNS!

After posting this blog entry, I had a number of people reach out to me to tell me that, while my configuration recommendations were good, there was something new in the DNScrypt protocol that I could take advantage of to make my DNS even more obfuscated – “Annonymized DNS”.

The way this protocol works is that you send your encrypted DNS request to a relay (which can’t decrypt it). The relay then sends the encrypted request to your resolver. The resolver decrypts the request, resolves it and sends the encrypted answer back to the relay. The relay (which can’t decrypt the answer either), sends the encrypted answer back to you where you can decrypt it and stick it in the cache.

If you followed all of that, the relay (even if it is evil and logging the heck out of everything you do) can’t know what your request is. It only passes it along. The resolver (again, it could be evil and logging stuff too) does know your request but doesn’t know that it is coming from you! It only knows that the request came from the relay.

This is a pretty cool thing when it comes to making your DNS resolution even more private!

To make this work with the setup (regardless of the operating system), you simply need to add one more section to your dnscrypt-proxy.toml configuration file:

[anonymized_dns]routes = [
{ server_name=’cs-ch’, via=[‘sdns://gRE1MS4xNTguMTY2Ljk3OjQ0Mw’] },
{ server_name=’faelix-ch-ipv4′, via=[‘sdns://gRMxODUuMjUzLjE1NC42Njo0MzQz’] },
{ server_name=’yofiji-se-ipv6′, via=[‘sdns://gS5bMmEwMjoxMjA1OjM0ZTc6OGUzMDpiMjZlOmJmZmY6ZmUxZDplMTliXTo4NDQz’] }
]

The server_name fields should be recognizable because they are the ones that you listed for your resolvers. The sdns mumbo-jumbo is the name of the relay you are asking to use for each one of them. They are published on this page. I’d recommend picking ones in countries you like and that are different providers from the resolvers. If you have an evil service provider that runs both a relay and a resolver, they could piece the traffic back together. If they are different entities, it is far less likely that they would.

Enjoy!

Fixing an OpSec Hole…

As return readers of this blog know, I try pretty hard to maximize my privacy and security online and also share what I’ve learned with the readers. One (in retrospect) painfully obvious hole, however, in my operational security (OpSec for the cool kids) is that I use the same bloody username for most of my online accounts. It doesn’t take a data brokerage genius to figure out that all of these accounts are owned by the same person. Duh!

So if you are a creature of habit like I am and would like to improve your operational security, I thought this would be a helpful post. I’m going to outline just how to do this along with some tools that are pretty useful as well. First off, what’s the best way to come up with a new username for a service that won’t give away who you are? Turns out, there are a variety of websites that will generate readable, random usernames for you. One that I found particularly helpful was from LastPass, a password vault application.

Thus armed, it’s now down to the laborious process of figuring out if you can rename your account on a variety of services and, if not, deleting and recreating said account by hand. You might also want to consider deleting some of these accounts if you don’t use them any more. That will reduce your personal attack surface in the event that one of these services is breached.

Just for fun, I have a set of links below that I discovered that should save you some time if you frequent these sites / services. What’s surprising is how many of these services do not let you rename your account. For those, it’s best to delete it unless there are specific digital purchased tied to that account (damn you Steam!).

After you think you are done, do yourself a favour and do a DuckDuckGo search of your commonly-used username. You might find some accounts out there that you had forgotten about. Chances are, if you forgot them you probably don’t use them so take the opportunity to delete them and decrease your attack surface further.

How To: Privacy-centric DNS

For those who aren’t as technically minded, it’s worth talking about what happens when you type a URL into your web browser and how that impacts privacy. The first thing is that the string you type in, say https://mycoolsite.com needs to be turned into a numeric IP address. This is done using a protocol called “Domain Name Service” or DNS that is as old as the hills. When you get an IP address from your Internet Service Provider (ISP), it’s usually done using something called Dynamic Host Configuration Protocol, or DHCP. At the same time you get that IP address, the ISPs DHCP server will generally pass other configuration parameters such as your DNS server.

OK, so that’s the mechanics of it. What is the privacy implication. Well, ISPs like to make money. And while they make money from you by providing you with Internet service, they also like to make money other ways. One of those is selling information on you to data brokers and advertisers. Since they are turning your URLs into IP addresses using their servers, it’s a pretty easy thing for them to sell the URLs that you like to visit to those data brokers and advertisers. Given that they have to log this information in order to sell it, they can also turn it over to government agencies when they are either subpoenaed or (worst case) handed a national security letter which they can’t even disclose that they were given.

In other words, your ISP knows every web site you visit and happily tells other people about it that you might not want told. Let that sink in for a moment. All of the work you might (or might not <sigh>) do to stay private online goes out the window when you type that address into your URL. By the way, this also goes for your mobile service provider. When you are out and about (i.e. not on WiFi), your cell service provider is your ISP and they have all of that same power and information. Yikes, eh?

So, what can you do about it? Well, using some other regular DNS provider like Google (with the famous 8.8.8.8 DNS server) or CloudFlare doesn’t really help because who trusts their motives? You could use a privacy-respecting DNS provider that doesn’t log their information and has a warrant canary (a mechanism where they regularly post on their website that they haven’t been issued a national security letter until they stop posting that – which means they have), but since the DNS protocol is as old as the hills, that traffic is sent unencrypted over the internet and an adversary could capture it anyhow.

What to do? Well, there is a clever multi-part solution that I’m going to outline here for Linux, OpenBSD (of course) and Windows. It involves running a local DNS resolver on your laptop or desktop machine (why let your ISP resolve them when you can do so yourself) and when your local resolver doesn’t know the answer, using a protocol called DNScrypt to send that DNS request in an encrypted fashion, upstream to a DNS server that doesn’t log and is privacy respecting. It’s not airtight (that upstream server could be lying about being privacy respecting or could get compromised and not even know it) but like most privacy and security, the goal isn’t to be perfect, it’s to make yourself a much harder target so that the bad guys look for easier sheep to fleece.

Linux

For my example in Linux, I’ll be using Ubuntu 20.10 as the target operating system and version so if you are on a different distribution, your mileage may vary but the building blocks will be the same. I based this how-to on a combination of a Reddit post that I found while searching DuckDuckGo (you REALLY should stop using Google and who uses Bing anyhow?), a great article on LinuxConfig that I found the same way and finally

First things first, you need to install the necessary open source tools:

# apt install dnscrypt-proxy dnsmasq

Next, add the line “dns=127.0.0.1” to the [main] section in your local /etc/NetworkManager/NetworkManager.conf file. This tells NetworkManager to force the DNS manager to be the one installed locally (hence the “localhost” bit) on your machine.

After that is out of the way (please don’t reboot now because you won’t have the necessary other bits configured, please be patient <grin>), edit your local /etc/dnscrypt-proxy/dnscrypt-proxy.toml configuration file to set up the upstream server you want to use. I’d recommend taking a look at this list because 1) it’s from the documentation site for DNScrypt; and 2) it has a list of servers that are privacy respecting and don’t log connections. I’d also recommend refreshing your memory as to who the “Fourteen Eyes” nations are who do intelligence sharing with the US and try to pick a resolver that is in a country not on that list. Just sayin’…

Anyhow, add the following lines in the global section of the file:

listen_addresses = [‘127.0.0.1:53000′,'[::1]:53000’]
server_names = [‘cs-ch’,’faelix-ch-ipv4′,’yofiji-se-ipv6′]

I chose to use multiple servers in my configuration to make it resilient in case one of them happens to fail. I’m making the assumption (check my math on this) that the first two (the ones in Switzerland – my privacy-respecting country of choice) are the primary and the other one is there as a back-up in case the first two don’t respond.

Because I couldn’t get the “listen_addresses” bit working from the config file (I left it in there just in case), I also edited the /etc/systemd/system/sockets.target.wants/dnscrypt-proxy.socket file to specify the port I want this service running on. In the [Socket] section, I modified the two lines with the port 53000 setting that I have in the config file:

ListenStream=127.0.2.1:53000
ListenDatagram=127.0.2.1:53000

Now that we have DNScrypt set up, we need to set up the dnsmasq service to be our local, lightweight caching resolver that forwards its upstream resolution requests to DNScrypt. To do this, we need to edit it’s configuration file /etc/dnsmasq.conf and add the following lines (the default file has everything commented out so instead of searching for them and un-commenting them, I just slammed this at the end of the file):

no-resolv
server=::1#53000
server=127.0.0.1#53000
listen-address=::1,127.0.0.1

After setting this up, there is a REALLY IMPORTANT STEP you need to do and that is to disable systemd-resolvd:

# sudo systemctl disable systemd-resolved

After you have done this, you should be able to reboot and resolve DNS names (easy test, go to a site you rarely visit in your browser). To verify that it is your privacy-respecting configuration that is doing this, try the following tests:

# sudo lsof -iTCP:53000 -sTCP:LISTEN

You should see several lines showing tha t the proxess “dnscrypt” running as user “_dnscrypt-proxy” is listening on this socket.

Next, check who is listening on port 53 (the “regular” DNS port):

# sudo lsof -iTCP:53 -STCP:LISTEN

You should see that the process ‘dnsmasq’ running as user ‘dnsmasq’ is listening on this socket. So far so good.

Now, use the ‘dig’ command to force the resolution of a URL that you don’t normally visit:

# dig -t A microsoft.com @127.0.0.1

You should get a display of the IPs that match that URL. Finally, if you are really paranoid (and I am), verify that by turning off dnscrypt-proxy that you CANNOT resolve DNS queries:

# systemctl stop dnscrypt-proxy
# systemctl stop dnscrypt-proxy.socket
# dig -t A oracle.com @127.0.0.1
# dig -t A oracle.com

This should fail to resolve things. Make sure you restart the two systemd services after you have verified this.

Congratulations. You have just significantly improved the privacy of your machine. Now go do this on all of your Linux systems.

OpenBSD

I listed these sections in alphabetical order so don’t read into my choice of having “Linux” first as an indication that I don’t love me some OpenBSD! For this part, I used my coreboot+tianocore Thinkpad T440p running OpenBSD 6.8 off of a secondary drive (the primary drive had the fresh Windows 10 install on it for the Windows version of this how-to).

First things first, we need to install dnscrypt-proxy:

# pkg_add dnscrypt-proxy

The configuration file is logically /etc/dnscrypt-proxy.toml and you need to edit it and add our server_names and listen_addresses to it:

server_names = [‘cs-ch’,’faelix-ch-ipv4′,’yofiji-se-ipv6′]
listen_addresses = [‘127.0.0.1:53000′,'[::1]:53000’]

Once you have that saved, you should enable the daemon using the following command:

# rcctl enable dnscrypt_proxy

Note the underscore instead of a dash there for the daemon name. Now start it up:

# rcctl start dnscrypt_proxy

To verify that the daemon is running and listening on our desired port, simply use netstat:

# netstat -an | grep LISTEN

You should see that there is a process listening on port 53000 in both localhost IPv4 and IPv6.

Now we need to get a local resolver running and forwarding to DNScrypt. Fortunately, OpenBSD has a really nice builtin one called unbound. Enable and start it using the following commands:

# rcctl enable unbound
# rcctl start unbound

Now we need to tweak our dhcp configuration (if you are not running a static IP address) to override any “suggested” name server from your dhcp server. To do that, edit the /etc/dhclient.conf file and add the line:

supersede domain-name-servers 127.0.0.1;

Restart your network with the following command:

# sh /etc/netstart

And then check your /etc/resolv.conf file to verify that the name server is in there along with no other.

Use the following commands to verify that your DNS resolution is working using unbound locally:

# dig -t A microsoft.com @127.0.0.1
# dig -t A microsoft.com

Assuming that goes well, now we need to edit unbound’s configuration file to use localhost port 53000 for our upstream resolver. That file is in /var/unbound/etc/unbound.conf and you need to edit the forward-zone section to look like this:

forward-zone:
name “.”
forward-addr: 127.0.0.1@53000

In addition, in the “server:” section of the file, you need to add the following line in order to get the forwarding working properly between the unbound daemon and the dnscrypt-proxy daemon. Without it, localhost will be ignored for queries:

do-not-query-localhost: no

Use the following commands to restart the service and validate that name resolution is working:

# rcctl restart unbound
# dig -t A oracle.com @127.0.0.1
# dig -t A oracle.com

Now, to verify that you have unbound listening on port 53 and dnscrypt-proxy listening on port 53000, run the following:

# netstat -an | grep LISTEN

If you see listening processes on both the IPv4 and IPv6 ports, you should be good. Finally, to verify that the upstream requests are being forwarded to dnscrypt-proxy, stop its daemon and test using dig:

# rcctl stop dnscrypt_proxy

Windows

OK. Whew. I’m going to be a bit out of my depth on this one (I’m not a big-time Windows power user any more) but the Internet has some good stuff on it so here goes nothing. For my setup, I had a dead-fresh install of Windows 10 on my coreboot + tianocore Thinkpad T440p that I’m using. I figured that way no extra configuration would get in there that might mess things up.

I figured there are a lot of bad actors on the Internet who might post some bad info on how to set this up in such as way as to route all of your traffic to them (forgive me, I’m paranoid <grin>), so I thought the best place to go would be the GitHub repository for the DNScrypt-proxy project.

From there, I clicked the link to download the latest version of DNScrypt-proxy for Windows. Given that, I read the documentation on how to install the service myself, not using a possibly compromised “helper”. Also, this helps me learn where the files are and how to configure things which is always good. I’ll document my process here. From this point forward, assume that I’m running a PowerShell command prompt with the “run as Administrator” option.

I copied the contents of the “win64” directory to C:\Program Files\DNScrypt-proxy to start things off. Best to have it in a “standard” (I think) place. I then move to that as my current directory. Next, copy the example-dnscrypt-proxy.toml file to “dnscrypt-proxy.toml”:

PS > copy example-dnscrypt-proxy.toml dnscrypt-proxy.toml

Edit the file using notepad (I guess) to have the same server_names line that we had for the Linux install above:

server_names = [‘cs-ch’,’faelix-ch-ipv4′,’yofiji-se-ipv6′]

Now, let’s make sure everything is working by running ./dnscrypt-proxy from that PowerShell prompt:

PS > ./dnscrypt-proxy

You should see some diagnostic information finishing up with “dnscrypt-proxy is ready – live servers: 2” in your PowerShell prompt.

The instructions are a little off on the version of the page I was looking at (or maybe my Windows 10 fu is weak). Anyhow, I went old-school and went to my “Wi-Fi” settings in the UI and selected “Change adapter options” from the link on the right. I then went to my “Ethernet” and “Wi-Fi” icons, right clicked on them, selected “Properties” and then went to the “Internet Protocol Version 4 (TCP/IPv4)” item and clicked the “Properties” button.

For both adapters (Ethernet and Wi-Fi), I changed the page to “Use the following DNS server addresses” radio button and entered 127.0.0.1 in the “Prefered DNS server” address and “9.9.9.9” as the “Alternate DNS server” per the instructions. Just for shits and giggles I checked the “Validate settings upon exit” checkbox and hit OK. I didn’t get any error messages so I’m assuming it’s good.

I then repeated the process with the IPv6 properties, setting the “Preferred DNS Server” to “::1” (without the quotes) – the equivalent of localhost in IPv6 land. I also set the “Alternate DNS server” to “0:0:0:0:0:ffff:909:909” which is the IPv6 address of that server.

Back to the PowerShell prompt, I hit “Ctrl+C” to break out of the dnscrypt-proxy process that was running and executed the following command:

PS > ./dnscrypt-proxy -resolve example.com

This successfully verified that I was able to resolve the DNS query using my configuration file. Now for the fun part, installing the service. To do this, run the following command:

PS > ./dnscrypt-proxy -service install

If you don’t get any error messages, start the service:

PS > ./dnscrypt-proxy -service start

Assuming you are error free there, you actually have a working (without a local caching resolver) install of DNScrypt-proxy. The final step is to fiddle with a Windows 10 group policy setting for the Network Connect Status Indicator (NCSI) to prevent it from showing your network as “offline”.

To do this, run “gpedit.msc” to launch the Group Policy Editor. From there, drill down to:

Computer Configuration -> Administrative Templates -> Network .> Network Connectivity Status Indicator

In the right-hand pane, select the “Specify global DNS” policy and click the “Enabled” radio button. Now check the “Use global DNS” checkbox and hit OK.

At this point you should be safe to reboot and verify that things are working. After the reboot, bring up a “run as Administrator” PowerShell prompt and run the following command:

PS > netstat -a -b

You should see dnscrypt-proxy.exe listening on port 53 of the localhost IP address. Bring up a browser and hit a site you rarely visit to verify that you have name resolution working.

I don’t have the local cache part of this working on Windows yet but all I’m losing there is efficiency because I’m doing the DNS lookup each time I need to resolve something. All said, I think I improved the privacy of my Windows 10 install by a non-zero amount! 🙂

Mobile?

So what about mobile? That tends to be a platform that we all spend a lot of time on these days and I’d hate to leave it out. While my experience and expertise doesn’t allow me to truly test that this works like I can on desktop operating systems where I can use tcpdump, turn services on or off, etc. some sites that I trust lay down some recommendations that appear to be working when I test them. Therefore, use this at your own risk and do testing that you think makes sense for you and your threat model.

For iOS, I found the following link from PrivacyTools.IO (one of my favorite sites) that walks you through how to use an app on the iPhone to run dnscrypt-proxy as your local DNS resolver on iOS. Essentially, you need to install the app, click on the “edit” button in the toolbar when you launch it and add our standard “server_names” line below, then hit the checkmark button in the toolbar to save it:

server_names = [‘cs-ch’,’faelix-ch-ipv4′,’yofiji-se-ipv6′]

Then, you will want to click on the “hamburger button” in the toolbar on the left and turn on the “Connect On Demand” general option. After that I rebooted my phone just for good measure and I appear to be up and running.

For Android, I found an application on the Google Play store called Quad9Connect. It appears to use secure DNS to access servers but it lacks the fine-grained control I would like in order to specify which countries I want those DNS queries to be tunnelled through.

Another Android app that I have seen recommended on a variety of sites is InviZible Pro. This app allows you to have more control over your DNScrypt-proxy setup. The settings I’m running with are to turn off the Tor support, leave the DNSCrypt support on and set it to run at boot-time. I also turn on the “require_nolog” feature in the DNSCrypt Settings page. Finally, I go into the Network & Internet settings in Android, selecct VPN, select settings for InviZible Pro and enable “Always-on VPN”.

Save Me from the Data Brokers!

In looking back over my most recent posts on this blog, I’m noticing a pattern. I’m tending to focus on technical aspects of privacy and security and am leaning more towards the security side of things. Given that I want to be balanced in the information I share here at FunctionallyParanoid, I thought it would be a good time to start making a concerted effort to share some information that isn’t focused on installing operating systems or configuring software. In this post, I’m going to talk to you about the evil role of “data brokers” in the world today and what you can do to limit their nefarious impact on your life.

Data brokers are not a new invention of our modern computer age. They have been around for thousands of years. As long as a given piece of information had value to someone else, they work to maximize that value and monetize it to their benefit. For example, if you knew the location of an invading army, you could sell that location to the country being invaded, regardless of whether it was the twenty-first century or the first century. For that matter, you could then turn around and sell the fact that the element of surprise was lost to the invading army and suggest they take a different route for their invasion and so on. Nice work if you can get it, eh?

The modern data brokers also set themselves up to sell the same information multiple times. What they do is comb through tons of data – things like mailing lists for catalogues, marketing databases, etc. They then correlate that information and assemble a dossier on each person that they can tease out. Once they assemble that dossier, they can then sell it to firms that want to market to you, to people trying to track you down to collect on a debt, whatever. The information is for sale to the highest bidder.

Yes, there are some limitations to what they can do, but it really depends on where you live. For example, if you live in California, there is a new data privacy law that places some realistic limits on these firms; however, if you live in Montana, you are pretty much on your own. There really isn’t a lot of privacy protection at the federal level.

Given that, what can you do? Well, for one, you can work to either correct or delete the information that these data brokers have on you. While there are services you can pay to do this on your behalf, I thought I’d do this the old fashioned way and see how good a job I can do using a little elbow grease and trying my hand at getting my information deleted.

The first thing you need to do is get a list of who the data brokers are. That is surprisingly difficult to do in a canonical way. However, after a series of DuckDuckGo searches (of course I use them and not Google – I’m FunctionallyParanoid after all), I was able to come up with the following list. What I’ll do is keep this post in draft form as I go through the process of requesting that my information be deleted and add tips and additional information on each broker as I knock them out.

As a sidebar, there is a service called Blur that allows you to get a free email forwarder that masks your email address. Essentially you put in your “blur” email address and then the email actually comes to you but the sender doesn’t really know what your true email address is. Might not be a bad idea to use a service like this for the removal emails where they require confirmation. Just a thought.

Here we go…

Acxiom

This is one of the big ones. You need to navigate to their opt out page and wade through a plethora of text about why this is a bad idea and you really should love having your data sold all over the place without you knowing who gets it. You then have to fill out a form at the bottom of that screen where the difference between the text and the white background is about +1 on the RGB scale – in other words, it is nearly indecipherable. After you squint your way through that, you will get an email that requires you to click a link to ensure that the information is deleted. As with all of these, set a calendar reminder to check back and ensure that your information has truly been removed.

BeenVerified

This broker seems to be the easiest to deal with and pretty quick in terms of deleting the information. Go to their opt out page and search for your information. Once you find your record or records (I found old addresses listing me with different middle initials), click on each one, supply an email address (yes, you have to do this because they require you to click a link in the email they send you to actually kick off the removal process), help Google or whoever identify busses or fire hydrants, and then wait for that email to show up. Once you click the link, they will begin the process and have you removed in 24-48 hours. Put a calendar reminder out there to check that you have been removed and see if there are any other records that need to be deleted.

Epsilon

Another big one. These folks have not only the ability to sell your data for offline purposes, they can do it online. And, they purport to be able to connect the two. Yikes! To get your data removed, go to their opt out page and fill it out. Just like with Axciom above, you have to wade through some text to get to the form. Again, don’t forget to set a calendar reminder to check back and ensure that the data has been deleted.

Intelius

This one is probably the easiest of the bunch. You simply navigate to their opt out page, search for yourself and then request that they delete the information. As with all of these, put a calendar reminder in your calendar to ensure that the information has been deleted.

MyLife

This one is a pain. You have to “join” their service (which to me seems to indicate that they have more accurate information on you) but you can obfuscate that information. Anyhow, once you join you can then search for your information and send an email to privacy@mylife.com to have them remove you. Include the URL from your search in the email and don’t forget to put a calendar reminder in to verify that it has been deleted.

Raradis

Similar to the others, you first have to search for your record on their site. You then copy the URL from your browser’s address bar and paste it into the opt out form to have them remove your information. They will notify you when the information is removed. When they did, I actually checked and my information was NOT removed. An email to customer service corrected the issue and my information was removed.

Spokeo

This one is pretty similar to many of the others. Go to their search page and look for yourself. Once you find a record you wish to remove, click on it, save the URL from your browser’s address bar and supply it to their opt out page. As with all of these, you’ll have some email interaction with them and they will let you know you have started the process. I strongly recommend taking their timeframe for removal and putting a reminder in your calendar to check up on it and make sure the information is truly gone.

WhitePages

This one is more pernicious. They seem to have lots of different records of me that need to be deleted. The process is pretty straightforward, however. You go to their search page and look for your information. After you find your listing, if you are fortunate enough to see one with an arrow at the right, click on that and copy the resulting URL from the address bar. You can then go to their opt out page, provide the URL and go through the request process. Since this broker sources from public records, I can’t find a way to delete their “premium search” and they still list your personal information (such as home address) on their search pages. To get around this, I tried using their CCPA opt out form because my records can be found by searching an address in California.

Conclusion

To net this out, it’s a few minutes worth of work but the end result is that your data isn’t being monetized by some faceless corporation somewhere. If you then have good hygeine on your privacy, perhaps you can reduce your data footprint to keep from being an interesting target for these brokers. Either way, it is probably worth checking in on your status on these sites a couple of times a year or more.

Quick and Dirty OpenBSD Version Upgrade on a Running System

I probably should have checked the mailing lists before writing my last two blog posts on how I install and “beautify” an OpenBSD bare-metal install on a laptop because sure as I was done with the second one and shared it to my friends on Reddit, I saw the announcement that 6.8 was available. Well, probably a good time to show how I do an in-place upgrade of a running system. I heavily leverage this post from the main site, so I thought I should give it a shout out for its helpfulness!

First things first, I make sure I have patched up to the latest and greatest kernel of what I’ll now be calling the “old version” of OpenBSD:

# syspatch

I then make sure my firmware is fully upgraded:

# fw_update

Then, just for double-dog-sure’edness (I just made that compound word up on the spot), I reboot:

# reboot

By the way, I always have my laptop set to boot into Windows by default so that if it is “checked” at a border crossing (I always approach security with all of my devices powered down) it boots into a benign and mostly unused install of Windows. I know, it’s a bit “security through obscurity” but what the heck, right?

If this is the first upgrade for the system, I create a directory called /root/upgrade and then within it create subdirectories for the versions where I copy the files. I generally clean out the old one when I’m setting up a new one just to preserve a little disk space:

# mkdir /root/upgrade
# cd /root/upgrade
# mkdir 6.8
# cd 6.8

I now download all of the install files from the CDN using the built-in ftp client (I used to use wget but why install another port if you don’t need it <grin>):

# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/base68.tgz
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/comp68.tgz
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/game68.tgz
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/man68.tgz
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/xbase68.tgz
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/xfont68.tgz
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/xserv68.tgz
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/xshare68.tgz
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/bsd
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/bsd.mp
# ftp https://cdn.openbsd.org/pub/OpenBSD/6.8/amd64/bsd.rd

For those who are extra paranoid like me, I always make sure I have a total of 11 files in this directory. Remember the name of the blog after all… <grin>

Now, you are about TO DO SOMETHING DANGEROUS to your system so don’t blame me if it goes awry. The sequence is very critical because you need to preserve an “old kernel” reboot command and only update the “base” install last. Otherwise you will get in a situation where you have the wrong userland files for the kernel you are running and things will get messy. So, here goes everything in the correct sequence:

# ln -f bsd obsd && cp bsd.mp /nbsd && mv /nbsd /bsd
# cp bsd.rd /
# cp bsd /bsd.sp
# sha256 -h /var/db/kernel.SHA256 /bsd
# cp /sbin/reboot /sbin/oreboot
# tar -C / -zxphf xshare68.tgz
# tar -C / -zxphf xserv68.tgz
# tar -C / -zxphf xfont68.tgz
# tar -C / -zxphf xbase68.tgz
# tar -C / -zxphf man68.tgz
# tar -C / -zxphf game68.tgz
# tar -C / -zxphf comp68.tgz
# tar -C / -zxphf base68.tgz

I then comment out all of my cool stuff in /etc/rc.conf.local and reboot using the old reboot command:

# /sbin/oreboot

You then need to update some other things as root:

# cd /dev
# ./MAKEDEV all
# installboot sd1
# sysmerge
# fw_update

I typically reboot (just to be safe) and then log in as root and update my packages:

# pkg_add -u

Finally, remove the old reboot command:

# rm /sbin/oreboot

Uncomment all of the cool stuff in your /etc/rc.conf.local and reboot. At this point you should be AOK and running the new kernel, userland and packages from the release version!

If you get stuck somewhere along the way, remember you can boot into single-user mode with:

boot> boot -s

You can then manually mount your filesystem and poke around to see what you need to fix. This should serve to reinforce why having physical possession of a machine trumps most of your security preparations and also reinforce the need to run a full-disk encryption (if you aren’t already).

I hope you found this post helpful!

Fast follower post – making OpenBSD UI a bit “prettier” (as I see it)

In my last blog post, I shared how I set up my OpenBSD laptop from the bare metal. In retrospect, there are a handful of UI tweaks that I also perform to make my UI and workflow more consistent between my OpenBSD machines and my Linux (Ubuntu) machine. I know that the purists out there are crying “boo hiss” but then they probably were crying that when I talked about using Gnome instead of cwm and xenodm for my window manager <grin>. Here are some things that I like to do.

First things first, I had forgotten that I prefer to use the “regular” (not dark, not light) version of the Yaru-remix theme from https://gnome-look.org because it keeps the menu bar and window titlebars dark while still leaving things like Evolution’s list of folders, preview pane and list of messages pane with a light background. Again, more like what I see on Ubuntu in my Linux machines.

For my terminal prompt, I had forgotten at the time I did the nuke and re-pave that I had moved away from the old “green screen” look (green on a black background) and went with white on a black background to get the base look set up. I then use the ability to inject color into the PS1 prompt so that the machine name shows up in green and the current working directory shows up in blue with a white dollar-sign prompt by making this change in my ~/.profile file:

export PS1=”\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]$ “

Next, I need to install the colorls package using “pkg_add” and enable a .kshrc file by adding this line to my ~/.profile file:

export ENV=$HOME/.kshrc

In the ~/.kshrc file itself, I add an alias to make a regular issuance of “ls” actually fire the colorls command:

alias ls=”colorls -G”

and just for giggles, I use “pkg_add” to import the vim (non-x11) package and make my old timey SunOS4 muscle memory not betray me when I’m trying to edit a file and mistype “vim” as just straight-up “vi”:

alias vi=”vim”

I also add this line to my /etc/rc.conf.local file (I haven’t figured out the magic syntax for rcctl to do this with no flags):

ntpd_flags=””

and then start the NTP daemon:

# rcctl start ntpd

Since I maintain several ports on OpenBSD (yeah, I know, I need to update some of them – I’m working on it! <grin>), I like to set up anonymous CVS access to the ports tree by adding this to my .profile:

export CVSROOT=anoncvs@anoncvs1.ca.openbsd.org:/cvs

and then do an initial checkout of that and the rest of the OpenBSD source code for good measure by running:

# cd /usr
# cvs -qd anoncvs@anoncvs1.ca.openbsd.org:/cvs checkout -rOPENBSD_6_7 -P src ports xenocara

I also make Firefox my default web browser and follow the recommendations at https://privacytools.io to ensure that I have it configured to maximum privacy mode.

One thing I’ve noticed is that it takes Gnome Evolution 3-4 launches to correctly pick up folder names in my EWS and IMAP mail services. I’m not sure why that is (it seems like some sort of network timeout more than anything else) but it’s just a minor annoyance. Keep re-launching it after it settles down and eventually it will get it right.

The other Evolution annoyance is that it doesn’t correctly identify the SPAM, TRASH and INBOX folders so some of the nice shortcuts you get from the context menu to expunge things (for example) and some of the nice automatic sorting of “Inbox” to the top of the folders list doesn’t happen with EWS accounts. Again, annoying, but it doesn’t limit the basic functionality of the email client.

Since it takes several hours (days?) to sync all of my NextCloud documents and email archive data, I generally try to switch off suspend when plugged in until everything is synced the first time. After that, I go back to a suspend after an hour of inactivity mode for my normal use. While you are in the settings app changing that, it’s a good time to turn off the notifications on both the main screen as well as the lock screen if you find them as annoying as I do.

In conclusion, these little tweaks and configuration allow me to have a pretty consistent workflow across all of my laptops/workstations so that I can continue the programming of my spinal cord that has been ongoing since the 80’s when it comes to working on *nix systems. I hope you found this to be a useful read.

OpenBSD Laptop

Hi, I know it’s been a while. I recently had to nuke and re-pave my personal laptop and I thought it would be a nice thing to share with the community how I set up OpenBSD on it so that I have a useful, modern, secure environment for getting work done. I’m not going to say I’m the expert on this or that this is the BEST way to set up OpenBSD, but I thought it would be worthwhile for folks doing Google searches to at least get my opinion on this. So, given that, let’s go…

After downloading the install67.fs image from my favorite site (https://openbsd.cs.toronto.edu/pub/OpenBSD/6.7/amd64) I write it to a USB drive on Linux with:

$ sudo dd if=install67.fs of=/dev/sdc bs=1M

I then boot off of that USB drive and drop to a (s)hell at the install prompt. From there, I create the full disk encryption container for my install:

# cd /dev
# chmod +x ./MAKEDEV
# ./MAKEDEV sd1
# fdisk -iy sd1
# disklabel -E sd1

Create one big “a” partition for the whole drive but specify RAID as the partition type.

# bioctl -c C -l /dev/sd1a softraid0

I then enter my decryption password and type ‘exit’ to restart the installation. The only tweak I make to the installer is to set up my wifi (it won’t work in the installer or on reboot for me because I have an Intel ‘iwm’ device that needs the firmware installed – I do that later) and make sure that my /usr partition is at least 200g in size (I probably should but I don’t create the subdirectory mount points, just one big /usr).

After the first reboot, I go back to my Linux box with the same USB drive, turn it into a single partition FAT32 drive and download the firmware for my Intel ‘iwm’ device from https://firmware.openbsd.org/firmware/6.7 as a single .tgz file that I place on that USB drive. I then mount the drive on OpenBSD:

# mount /dev/sd2i /mnt

Next, I copy the .tgz file to the /etc directory and expand it there. It will automatically put the right files in the /etc/firmware directory. I then reboot and should have WiFi. I then update the other firmware:

# fw_update

Next, I run syspatch to ensure that I’m fully patched on the kernel and userland I’m running:

# syspatch

That takes a bit of time, just be patient. At this point, it’s best to reboot because so much has changed on the disk from what is running in memory. You should see in the dmesg output that all of your firmware installed successfully and that the kernel relinked to help fight off potential ROP attackers.

First things first, I create an /etc/doas.conf file so that I can run everything in my regular user account:

# echo “permit persist keepenv MY_USERNAME as root” > /etc/doas.conf

Next, I set up power management (because I’m running on a laptop):

# rcctl enable apmd
# rccl set apmd flags -A
# rcctl start apmd

Since I run development tools and the owncloud desktop, I have to tweak some values in /etc/login.conf to make things work the way I want them to. First, I add myself to the “staff” group:

# usermod -G staff MY_USERNAME

I then edit /etc/login.conf in the “staff” section to make the following changes:

# pkg_add vim
# vim /etc/login.conf


staff:\
:datasize-cur=4096M:\
:datasize-max=infinity:\
:maxproc-max=512:\
:maxproc-cur=256:\
:openfiles-max=102400:\
:openfiles-cur=102400

I also modify /etc/sysctl.conf to include the line:

kern.maxfiles=102400

I then will typically reboot to pick up the changes.

Next, I want to install the Gnome desktop environment and the Gnome display manager. It’s what I’m most comfortable in and I know it’s a pain in the you know what for the port maintainers to keep it working on OpenBSD because of the Linux-ism’s that keep creeping in so I want to put in a shout out to them for all of their hard work on this.

# pkg_add gnome gnome-tweaks gnome-extras
# rcctl disable xenodm
# rcctl enable multicast messagebus avahi_daemon gdm

Install my favorite applications and utilities:

# pkg_add firefox chromium libreoffice owncloudclient
# pkg_add keepassxc aisleriot evolution evolution-ews
# pkg_add tor-browser shotwell gimp

I will reboot at this point and go into Gnome with the default theme. That needs fixing so I download the yaru-remix-complete theme from https://www.gnome-look.org and install it manually by doing the following:

$ cd ~
$ mkdir .themes
$ cd .themes
$ mv ~/Downloads/Yaru-remix-complete.tar.xz .
$ unxz Yaru-remix-complete.tar.xz
$ tar xf Yaru-remix-complete.tar
$ mv themes/* .
$ rmdir themes
$ doas mv icons/* /usr/local/share/icons
$ rmdir icons
$ doas mv wallpaper/* /usr/local/share/backgrounds/gnome
$ rmdir wallpaper
$ rm Yaru-remix-complete.tar

Fire up gnome-tweaks and from “Extensions” turn on “user-themes”. Close and restart gnome-tweaks, go to the “Appearance” tab and select “Yaru-remix-dark” for “Applications”, “Icons” and “Shell”. ON “Top Bar”, turn on “Battery Percentage” and “Weekday”. In “Window Titlebars” enable “Maximize” and “Minimize”.

Enable the extension “Dash to Dock” by downloading it from https://extensions.gnome.org/extension/307/dash-to-dock/ and pick the right shell version and extension version to match your install of Gnome shell. You will have to manually install it because the Gnome shell extension integration doesn’t appear to be enabled for OpenBSD:

$ cd ~/Downloads
$ unzip dash-to-docmicxgx.gmail.com.v67.shell-extension.zip
$ cat metadata.json

The value for “uuid” is what you want to use next:

$ mkdir -p ~/.local/share/gnome-shell/extensions/dash-to-dock@micxgx.gmail.com
$ cd ~/.local/share/gnome-shell/extensions/dash-to-dock@micxgx.gmail.com
$ unzip ~/Downloads/dash-to-docmicxgx.gmail.com.v67.shell-extension.zip

Reboot to restart Gnome shell, log in, start gnome-tweaks, navigate to the “Extensions” tab and enable dash to dock. From the settings gear icon, select “extend to edge…” and you should have a very serviceable dock that is quite similar to Ubuntu 20.04’s.

Finally, switch the terminal to “Green on Black” for a better look in the terminal. Pin your favorite apps to the dock and you should be good to go.