DRYer strong_parameters in Rails

This post documents a very simple pattern for DRYing up strong_parameters knowledge to make your Rails controllers cleaner and virtually maintenance free (at least in the face of model attribute changes).

Starting point

A typical Rails app will have a model with some attributes.

class Window < ActiveRecord::Base
  # Window has attributes :manufacturer, :thickness, and :glass_type
end

Following the conventional pattern, the WindowsController will probably have a private window_params method to whitelist the attributes allowed by Rails (v4 and up) for mass assigmnent.

class WindowsController < ApplicationController

  private
    def window_params
      params.require(:window).permit(:manufacturer, :thickness, :glass_type)
    end

end

If later you add an attribute to Window, say, :num_panes, you'll need to append it to your permitted attributes.

class WindowsController < ApplicationController

  private
    def window_params
      params.require(:window).permit(:manufacturer, :thickness, :glass_type, :num_panes)
    end

end

Adding complexity

There are two latent problems (not problems per se, but... perhaps inconveniences?) with this approach.

First, the diff for your commit will include a patch to app/controllers/windows_controller.rb when in reality it's very likely you made no substantial changes to that file. Mechanically you DID alter the code there, but there was no logical modification (in the abstract). The point here is minor - perhaps even philosophical - but it makes for unnecessarily noisy commits, which makes for more difficult debugging later.

Second, suppose your app had a House concept.

class House < ActiveRecord::Base
  has_many :windows
  accepts_nested_attributes_for :windows
end

Which had the same whitelisting in its controller.

class HousesController < ApplicationController

  private
    def house_params
      params.require(:house).permit(
        :address, :color, :value,
        window_attributes: [:manufacturer, :thickness, :glass_type],
      )
    end

end

Oops! Not only are we duplicating knowledge (the list of window_attributes), but we already forgot to add a reference to :num_panes in HousesController#house_params. No bueno.

DRY out this knowledge

There's a quick and easy solution, which is to declare a class method on each model returning an array of whitelisted attributes for that model.

class Window < ActiveRecord::Base
  # Window has attributes :manufacturer, :thickness, :glass_type, and :num_panes

  def self.window_params
    [:manufacturer, :thickness, :glass_type, :num_panes]
  end
end

Now our WindowsController can be cleaned up.

class WindowsController < ApplicationController

  private
    def window_params
      params.require(:window).permit(*Window.window_params)
    end

end

As can our HousesController.

class HousesController < ApplicationController

  private
    def house_params
      params.require(:house).permit(
        :address, :color, :value,
        window_attributes: Window.window_params,
      )
    end

end

And neither needs to be changed as we modify information in our Window model. If we apply the same technique to the attributes of House as well we can remove even more knowledge that doesn't belong from that controller.

class HousesController < ApplicationController

  private
    def house_params
      params.require(:house).permit(
        *House.house_params,
        window_attributes: Window.window_params,
      )
    end

end

Adding [more] complexity

Suppose we now introduce a Vehicle class. We decide, in our infinite wisdom, that we'll share code using a concern. We can reuse the same pattern to keep our controllers tight.

module Windowable
  extend ActiveSupport::Concern

  included do
    has_many :windows
    accepts_nested_attributes_for :windows

    def self.windowable_params
      { window_attributes: Window.window_params }
    end
  end
end

With all of our models using this pattern, our final HousesController is completely oblivious to the implementation details of our models.

class HousesController < ApplicationController

  private
    def house_params
      params.require(:house).permit(*House.house_params, House.windowable_params)
    end

end

And we can live happily ever after knowing that as we add, remove, and modify attributes on our models, we have a single source of truth for whitelisted attributes. AND this truth is grouped with all the other authoritative information about the state of the model; that is to say, it lives in the model itself. Win-win.

A word about purism

There is at least one possible argument against this approach — the pattern I've outlined does violate an academically strict understanding of minimal coupling. Rails 4 introduced strong_parameters with the express intent of moving parameter whitelisting behavior OUT of the model and INTO the controller. While this pattern doesn't violate that intent it does keep the configuration of said whitelisting in the model. You might even call it an antipattern.

But no matter which way you slice it, either you'll be duplicating knowledge or violating a high-brow interpretation of encapsulation. So you'll have to decide which brand of Evil™ you prefer to drink.

Constructive feedback and/or discourse? Comment below or tweet me. Send all hatemail to /dev/null (if you don't, I will).

Jekyll Plugin: Automatically Optimize Images

Jekyll is a great platform for managing static sites (like chris.tonkinson.com). It does the Right Thing™ out of the box, is simple to initialize and deploy, and it plays very nicely with some popular hosting options like GitHub Pages, S3, and Heroku. If roll-your-own-blog (the "Hello World!" of web development) doesn't sound productive enough, WordPress doesn't sound secure enough (cough cough it isn't), and Ghost doesn't sound mature enough, Jekyll might be your sweet spot.

Jekyll has the advantage of a huge plugins ecosystem. One example of a great Jekyll plugin is image_optim-jekyll-plugin — brainchild of @chrisanthropic — which is a simple but robust plugin built around @toy's excellent image_optim meta-project. image_optim is a clever Ruby Gem that wraps about a dozen different low-level image manipulation libraries and provides a clean unified interface such that you can simply chuck in any image file you might have on hand, and it will intelligently delegate your request to the corresponding library.

Check out the image_optim-jekyll-plugin README, but the TL;DR is that after you install this plugin, it automatically optimizes all of your Jekyll images at build-time. I recently contributed some documentation as well as functional changes so you can control which images get optimized, and where the plugin stores its artifacts.

It's just one more way you can cajole Jekyll into silently doing more work on your behalf.

How To Recover 20GB of Disk Space Back From Git

TL;DR — Don't be a packrat. But if you must, use gc/repack/prune regularly.

I love open source software for myriad reasons. Some are ideological, some are academic, and some are practical. One of the most beneficial practical aspects of open source software is that you can easily introspect what would otherwise be a black box.

When you've identified a business need, have searched for and installed software, and subsequently received some unhelpful and undocumented error message while configuring said software, your path forward is governed by whether or not you have source access. You've already searched Stack Exchange, mailing lists, etc. for solutions and tried several, all of which failed with different, but still maddeningly opaque errors.

With closed software, you're left to call customer support, and proceed to bash your head against the nearest keyboard(s)†. With open software, however, you can inspect the code, find the source of the error, and actually understand some of the context around the issue.

Searching for code is faster with find, grep, ack, or my new favorite ag (the_silver_searcher) than with Google (DuckDuckGo for me). Once you've found the file you're looking for, browsing/exploring around the codebase is faster and easier with Atom or Vim rather than clicking through a web UI such as on GitHub, GitLab or Bitbucket. Go figure - local operations are faster than network operations. :trollface: For this reason, I clone just about every project I use under ~/repo/vendor — and a bunch that I don't use directly but find helpful or interesting for whatever reason.


To ensure this code is up-to-date and relevant when I need it, I run a cron job nighly that pulls down the latest from upstream. Something like

#!/bin/bash
for repo in ~/repo/vendor/*; do
  # Update.
  git -C $repo fetch --all --quiet
  git -C $repo reset --hard --quiet
  git -C $repo merge --quiet
  # Dependencies.
  git -C $repo submodule update --checkout
done

Today I noticed disk usage approaching 60% on my primary workstation (running 2x Intel 730 240GB 6Gb/s SSDs in RAID0) and I wasn't surprised to find out that the glut of third party code I keep was consuming 65GB all by itself.

Turns out that git, for all it's superlatives, can get pretty gummed up under certain circumstances (such as when cloning a large project with a long and complex history such as linux, chromium, and gcc.

I decided to test a worst-case-scenario to see if adding in any routine maintenance would yield benefit. My local clone of the chromium repository (from https://chromium.googlesource.com/chromium/src.git) was 20GB alone, so I did a quick search of the man pages and wound up running git gc --aggressive --prune=all.

Three hours later... pregnant pause... yes, three hours later, the whole time pegging all 12 logical cores of my Intel i7-5930K and utilizing upwards of 70% of my 64GB DDR4 RAM, it was finished. The result was a 13.5GB reduction in space usage, down to a mere 6.5GB. Did I mention it took over three hours?

It didn't take long before I came to learn that gc --aggressive is rarely the right answer. See http://stackoverflow.com/questions/28720151/git-gc-aggressive-vs-git-repack for a full discussion of why that is, and how you might proceed differently in your case. I wound up adding a few extra tasks on the end of my nightly update script:

#!/bin/bash
for repo in ~/repo/vendor/*; do
  # Update.
  git -C $repo fetch --all --quiet
  git -C $repo reset --hard --quiet
  git -C $repo merge --quiet
  # Dependencies.
  git -C $repo submodule update --checkout
  # Maintenance.
  git -C $repo gc
  git -C $repo repack -Ad
  git -C $repo prune
done

I'll probably run git gc --aggressive --prune=all as I clone new repositories down (just to be sure they've been correctly packed since we don't know what GitHub, et al. do behind the scenes), but other than that, it's a waste of time.

I saved about 19GB running these tasks over everything under ~/repo/vendor, and another 1GB or so doing the same thing to all the other code I have locally, for about a 20GB total disk savings.

All told, rm -r is the fastest way to recover disk space, but if you're a packrat like me, git gc/repack/prune is second best.

If you know of similar tricks for optimizing Mercurial, Subversion, or CVS repositories, I'd love to hear them. Most everything I have is in git, but there are some disk-hungry notable exceptions (top of mind are webkit, which uses Subversion and sits on about 15GB, and the mozilla-central repository which uses Mercurial and eats about 3.8GB).


† The probability of getting an error, and subsequently being unable to make progress against it without outside help because of Software Documentation Insufficiency Disorder (SDID) while using proprietary systems is approximately 73.2%, according to my research experience.

The Bureau of Silence

Bruce Schneier explains the cost/benefits of the FBI throwing in the towel in the Apple case. Per his reputation for thoughtful analysis, this is well worth reading. At one point Schneier repeats what the industry has been crying for weeks:

This case has always been more about the PR battle and potential legal precedent than about the particular phone.

Intuitively, we all know this to be true. Objectively, however, we only suspect it to be true. Thing is, science is not about intuition and suspicion. Good science requires evidence and proofs. Not that we can realistically expect to get any of that in this case, but I have an idea.

Now that the FBI has "successfully accessed the data" on the iPhone according to a recent court filing (pdf) and the legal battle has been abandoned, we can actually set about trying to clear the record.

Apple, and the public at large, should demand the FBI disclose their method(s). Many are pointing this out, such as Robert Knake on the CFR blog. If — as they've maintained all along — this isn't about anything more than this particular device, then there's no conflict of interest in them responsibly disclosing their methods to Apple. But according to Knake, because this exploit likely came from an un-named third party:

All the FBI can likely tell Apple is what they have already made public: there’s a vulnerability in iOS. Good luck finding it.

Which is likely true. Apple, along with everyone everywhere, would love to know what the vulnerability is, but when they decided to play hardball with the government, they likely cemented the contract details and NDA's the government signed with their vendor so that they'll never actually find out.

So the drama continues. Now everyone will be hunting for clues to identify the FBI's vendor. Maybe we find out, maybe we don't. One thing is for sure though - if we circle back to the whole concept of "evidence," we'll never know what was on Farook's iPhone; You won't hear anything more about it from the FBI. Here's a basic truth table for the possible outcomes:

Possible Outcomes for the FBI (annotated with highly scientifically derived and totally true and accurate probabilities)

FBI Found Actionable Intel (5%) FBI Got Bupkis (95%)
They lie about it (80%) <silence> "Data proved useful, plz crucify Apple, kthxbye"
They don't lie (20%) <silence> <silence>

If — if just maybe you're a cynic like me — you believe there never was actionable intel to be found on Farook's work phone to begin with (since he destroyed his personal phone) and you — again, maybe you're just a little cynical — don't trust the FBI to be honest about it either way, then the most probable outcome (at 76%) is that they pretend to have gained operational intelligence from the exercise and mount a PR campaign to smear Apple for slowing them down...


[An internal conversation at FBI headquarters]:

Agent Smith: You know, instead of spending OUR budget trying to convince Apple to play ball — you know how hard-headed those guys are — why don't we just convince the public that they're irresponsible? Let them do our work for us?

Agent Jones: It's crowdsourcing! Like on Kickstarter!

Agent Smith: Exactly. Let's take this up the chain.

[Later, in the corner office]:

Agent Smith: ... and that's why we should leak some backchannel information indicating we actually did find something useful on the iPhone.

Agent Anderson: That's brilliant. Just one thing, intsead of tying Farook to a future plot, tie him to a previous event. Really cement the connection for the public; make it hit home.

Agent Jones: We're on it.


Now obviously I'm having fun here. I don't believe our law enforcement personnel are that dense, but I do have a healthy suspicion for what happens behind closed doors, funded by my tax dollars, into which I have zero oversight. Whether or not the Bureau is honest about it, my prediction is:

The FBI will claim they found some worthwhile information on the device (but won't share any details) and then publicly shame Apple for interfering. Moving forward, you'll scarcely hear them referencing this ordeal.

FreeNAS-Generated Certificates: Buyer Beware

I recently upgraded my FreeNAS from 9.3-STABLE to 9.10-STABLE. The process is easy — you simply adjust the update train, Check Now, and then Apply Pending Updates. This will require a system reboot.

Upgrade FreeNAS to 9.10

In my scenario, I have TLS enabled through a self-signed certificate generated by FreeNAS.

After the reboot, Firefox got angry: SEC_ERROR_REUSED_ISSUER_AND_SERIAL, which for the uninitiated, is not an error you can "I know what I'm doing" around.

Chrome and IE failed as well, with different takes reporting on an "invalid certificate" error.

What the frick?


I tried clearing Firefox's certificate cache (the infamous cert8.db, along with a few others for good measure) knowing that it wouldn't help because the other browsers hadn't ever been used to access the FreeNAS Web GUI.

To the command line! Because I couldn't access the site (why configure TLS certificates and then allow HTTP fallback?) I had to power down the box with the case power button, hook up an input and display, and then power back up, booting from the previous 9.3 image.

Once in (the certs now worked as before) I immediately enabled HTTP-fallback so I could play around and not need to reboot constantly.

First I tried simply generating a new Certificate using the existing CA. Same error. Then I tried generating a new CA, and a new Certificate with that. Nope. I executed openssl s_client -connect 1.2.3.4:443 which was able to parse the cert and I couldn't see anything amiss, so I figured it was actually not a format error, but some content problem.

Back to 9.3, so I could examine a working certificate, whereupon I noticed that the serial number was being reported as 01. This stuck out at me because

  1. typically the serial is a much larger number
  2. didn't Firefox complain about the serial? Oh, right! SEC_ERROR_REUSED_ISSUER_AND_SERIAL!

You aren't able to configure the serial number upon either CA nor certificate generation in the FreeNAS Web GUI. On a whim, though, I double-clicked the CA entry (as there is no Edit/View button at the bottom of the list). Not only can you make changes to the CA, but at the end of the form is a serial input. I randomly changed this to 231, generated a new certificate, and BAM, no more errors.

Changing the CA serial to some random number

The generated certificate, upon examination, had serial number 00:E7 (which is 231 in hex) and looking at the CA again, I see that the serial field has incremented to 232). Whew, okay, that was unexpected, but at least I'm back on track.


Until I looked at my plugin status page. After any change I'm always anxious to verify that my dependent services survived the upgrade.

FreeNAS plugins are all 'off'

Crap. But wait, Plex is alve and well.

Plex is functional

It wasn't until I looked down at the mini log window that I realized I wasn't out of the woods yet:

Mar 30 12:48:41 rogers manage.py: [freeadmin.navtree:567] Couldn't retrieve  https://1.2.3.4/plugins/plexmediaserver/1/_s/treemenu: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>

The plugins were fine, it was that FreeNAS 9.10's own internal APIs are broken with respect to self-signed certificates. Even with HTTP enabled. Long story short, this particular FreeNAS sits on a trusted network with restricted admin access, so for the sake of seeing accurate status information I've simply disabled HTTPS for now.

I lost this battle, but I will win the war.