headshot of Chris Tonkinson
Chris Tonkinson

tech {enthusiast, practitioner, leader, podcaster, mentor, entrepreneur}


Refactored Podcast

I co-host Refactored, a casual, ongoing conversation between two technology leaders trying to suck a little less every day.

Career Schema Project

I founded the Career Schema Project with the mission of developing open standards and technology to reduce the hassle of online jobseeking.

RSS Subscribe

© 2021 Chris Tonkinson

FreeNAS-Generated Certificates: Buyer Beware

I recently upgraded my FreeNAS from 9.3-STABLE to 9.10-STABLE. The process is easy — you simply adjust the update train, Check Now, and then Apply Pending Updates. This will require a system reboot.

Upgrade FreeNAS to 9.10

In my scenario, I have TLS enabled through a self-signed certificate generated by FreeNAS.

After the reboot, Firefox got angry: SEC_ERROR_REUSED_ISSUER_AND_SERIAL, which for the uninitiated, is not an error you can “I know what I’m doing” around.

Chrome and IE failed as well, with different takes reporting on an “invalid certificate” error.

What the frick?

I tried clearing Firefox’s certificate cache (the infamous cert8.db, along with a few others for good measure) knowing that it wouldn’t help because the other browsers hadn’t ever been used to access the FreeNAS Web GUI.

To the command line! Because I couldn’t access the site (why configure TLS certificates and then allow HTTP fallback?) I had to power down the box with the case power button, hook up an input and display, and then power back up, booting from the previous 9.3 image.

Once in (the certs now worked as before) I immediately enabled HTTP-fallback so I could play around and not need to reboot constantly.

First I tried simply generating a new Certificate using the existing CA. Same error. Then I tried generating a new CA, and a new Certificate with that. Nope. I executed openssl s_client -connect which was able to parse the cert and I couldn’t see anything amiss, so I figured it was actually not a format error, but some content problem.

Back to 9.3, so I could examine a working certificate, whereupon I noticed that the serial number was being reported as 01. This stuck out at me because

  1. typically the serial is a much larger number
  2. didn’t Firefox complain about the serial? Oh, right! SEC_ERROR_REUSED_ISSUER_AND_SERIAL!

You aren’t able to configure the serial number upon either CA nor certificate generation in the FreeNAS Web GUI. On a whim, though, I double-clicked the CA entry (as there is no Edit/View button at the bottom of the list). Not only can you make changes to the CA, but at the end of the form is a serial input. I randomly changed this to 231, generated a new certificate, and BAM, no more errors.

Changing the CA serial to some random number

The generated certificate, upon examination, had serial number 00:E7 (which is 231 in hex) and looking at the CA again, I see that the serial field has incremented to 232). Whew, okay, that was unexpected, but at least I’m back on track.

Until I looked at my plugin status page. After any change I’m always anxious to verify that my dependent services survived the upgrade.

FreeNAS plugins are all ‘off’

Crap. But wait, Plex is alve and well.

Plex is functional

It wasn’t until I looked down at the mini log window that I realized I wasn’t out of the woods yet:

Mar 30 12:48:41 rogers manage.py: [freeadmin.navtree:567] Couldn't retrieve <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>

The plugins were fine, it was that FreeNAS 9.10’s own internal APIs are broken with respect to self-signed certificates. Even with HTTP enabled. Long story short, this particular FreeNAS sits on a trusted network with restricted admin access, so for the sake of seeing accurate status information I’ve simply disabled HTTPS for now.

I lost this battle, but I will win the war.