How to Search and Change Text with Javascript
comments

Last updated 27 July 2016

Replacing text in javascript is easy, just use replace (i.e. 'wat'.replace('a', 'ha'); //=> 'what'). Sometimes, though, you want to do a search and replace on text in a whole html document that includes html tags and things that aren’t just text. I had to do that recently and found it wasn’t as obvious as I first thought.

Let’s say I wanted to replace all occurrences of hats with rats. Simple enough.

My first thought was to do something like this:

function hatsToRats() {
  var html = document.innerHTML;
  var newHtml = html.replace(/hats/, 'rats');
  document.innerHTML = newHtml;
}

This has a few problems. First, it changes all occurrences of hats to rats, even those inside html tag properties to <a href="mysite.com/hats">hats</a> becomes <a href="mysite.com/rats">rats</a>. Ideally, I would want <a href="mysite.com/hats">rats</a> so the link isn’t broken. The other problem is that if there were any javascript events bound to elements on the page, I’ve just replaced all the elements so those events will no longer be bound. I wanted this function to be independent of any other javascript so it shouldn’t know about what events it might need to rebind after running.

I wasn’t sure what to do so I did a little searching and came across Phrogz’s stackoverflow answer to a related question that explained document.createTreeWalker as a solution. This gives us access to each text node in the document like so:

function hatsToRats() {
  var html = document.querySelector('html');
  var walker = document.createTreeWalker(html, NodeFilter.SHOW_TEXT);
  var node;
  while (node = walker.nextNode()) {
    node.nodeValue = node.nodeValue.replace(/hats/, 'rats')
  }
}

What this does is finds all text nodes inside the html tag on the document and replaces its value with the newly hats/rats-swapped text. This won’t affect binding on any element on the page (even if you crazily have an event bound to a text node) and leaves tag attributes (like the aforementioned href) alone.

Hopefully this article saves you some time if you ever need to do some html find and replacing!


The Basics: Migrating Production Schemas
comments

Last updated 20 July 2016

The basics is a series where I explain a basic programming concept or term for new computer programmers or those who need a refresher.

When you start learning programming you are building toy apps or just apps for yourself and it is easy to get in the habit of dropping your entire database and starting from scratch every time you need to change the database schema. Once you are working with real user’s data, though, it’s no longer an option to lose all of your data every time you change your mind about the proper structure for the database.

Your only options are to always get the database schema design perfect the first time (impossible!) or to have a strategy for rearranging a database table’s columns without losing anything, ideally without downtime. Obviously the second option is the only real one. Even if you always make the right decisions about the database, new information or feature requests might make you want to change it.

The steps needed to change the database schema without losing data or having downtime are pretty much always the same. Each of these code-change steps should be followed by a deploy.

  1. Create a new column and start writing to it.
  2. Backfill old data to have the right value in the new column.
  3. Start reading from the new column.

Let’s look at an example. Here’s a simple Rails blog app with posts and comments:

# app/models/post.rb
class Post < ActiveRecord::Base
  has_many :comments
end

# app/models/comment.rb
class Comment < ActiveRecord::Base
  belongs_to :post
end

# app/controller/comments_controller.rb
class CommentsController < ApplicationController
  def create
    post = Post.find(params[:post_id])
    post.comments.create!(comment_params)
    redirect_to post_path(post)
  end

  private

  def comment_params
    params.require(:comment).permit(:body)
  end
end

# db/schema.rb
# some stuff
create_table "comments" do |t|
  t.integer "post_id"
  # various other fields
end

create_table "posts" do |t|
  # various fields
end
# more stuff

Now a new requirement comes in to allow users to comment on other comments in addition to posts. This means that comments need a polymorphic association to both comments and posts. The post_id field will no longer cut it because you’ll need a commentable_type and commentable_id column on the comments table.

Step 1 looks like this:

# app/models/post.rb - no change
class Post < ActiveRecord::Base
  has_many :comments
end

# app/models/comment.rb
class Comment < ActiveRecord::Base
  belongs_to :post
  belongs_to :commentable, polymorphic: true
  has_many :comments, as: :commentable
end

# app/controller/comments_controller.rb
class CommentsController < ApplicationController
  def create
    post = Post.find(params[:post_id])
    if params[:comment_id]
      commentable = Comment.find(params[:comment_id])
    else
      commentable = post
    end
    commentable.comments.create!(comment_params(commentable))
    redirect_to post_path(post)
  end

  private

  def comment_params(commentable)
    comment_params = params.require(:comment).permit(:body)
    comment_params.merge(commentable: commentable).permit!
  end
end

# db/schema.rb
# some stuff
create_table "comments" do |t|
  t.integer "post_id"
  t.string "commentable_type"
  t.integer "commentable_id"
  # various other fields
end

create_table "posts" do |t| # no change
  # various fields
end
# more stuff

Here we add the new fields to comment and start writing to them by setting a comments commentable. I just add some more logic to the CommentsController to handle the fact that the comment might be on a comment or post (as an aside, this code in the CommentsController is pretty bad and should probably be moved into its own class). At this point the code should be deployed to production.

Step 2 is a backfill:

# in a migration
Comment.where(commentable_id: nil).find_each do |comment|
  comment.update!(commentable: comment.post)
end

Once this backfill is run in production we are ready to start reading from that new column in the Post class:

# app/models/post.rb - no change
class Post < ActiveRecord::Base
  has_many :comments, as: :commentable
end

# app/models/comment.rb
class Comment < ActiveRecord::Base
  belongs_to :commentable, polymorphic: true
  has_many :comments, as: :commentable
end

Once this is deployed the post_id column can be deleted as it is no longer needed. While this is certainly more steps than just dropping the database and starting over, it isn’t too bad to do things this way and avoid losing any data or having any downtime. I hope you found this helpful and are not scared to refactor your database schema the same way you refactor your code.


Getting Deep into Ruby's Include and Extend
comments

Last updated 13 July 2016

When learning ruby people usually explain include and extend as follows. Use include if you want to add instance methods:

module DoIt
  def do_it
    "I'm doing it!"
  end
end

class Doer
  include DoIt
end

doer = Doer.new
doer.do_it # => "I'm doing it!"

And use extend if you want to add class method:

module DoIt
  def do_it
    "I'm doing it!"
  end
end

class Doer
  extend DoIt
end

Doer.do_it # => "I'm doing it!"

While these code examples work, they don’t really explain what is going on. A better way to understand the difference is that extend is used to add a module’s methods to a specific instance while include is like copying and pasting the instance methods from the module into the place where you call it.

To clarify, in the first example above using include, it is as if the def do_it code from the module DoIt was inside the class Doer as a regular def. That means it is applied to instances of the Doer class. In the extend example, on the other hand, the methods are added to that particular instance of the class Class, in this case the Doer class. That means the methods are added to the class Doer itself as class methods.

Similarly, if you have an instance of a class, you can call extend on it to apply a module to that instance only:

module DoIt
  def do_it
    "I'm doing it!"
  end
end

class Doer
end

doer = Doer.new
doer.extend(DoIt)
doer.do_it #=> "I'm doing it!"
Doer.new.do_id # raises NoMethodError

That means you can use extend to methods to an instance of a class, the exact opposite of how people usually explain extend working. You can also use include to add class methods like so:

module DoIt
  def do_it
    "I'm doing it!"
  end
end

class Doer
  class << self
    include DoIt
  end
end

Doer.do_it #=> "I'm doing it!"

You can even do something similar to use include to add methods to a single instance of a class:

module DoIt
  def do_it
    "I'm doing it!"
  end
end

class Doer
end

doer = Doer.new
class << doer
  include DoIt
end
doer.do_it #=> "I'm doing it!"
Doer.new.do_it # raises NoMethodError

So you can see that extend and include have nothing really to do with whether instance or class methods are added to a class, but more how those methods are applied.

How is any of this useful? Besides being a lot (A LOT) of fun, this can frequently be helpful knowledge when dealing with a DSL, usually from a gem. Recently I needed to edit some code in a rails codebase that used a DSL to generate a sitemap. Most of the code appeared in a block and for some reason all of the rails url helpers were available, but not some custom url-related helpers we’d defined in other rails helpers. We determined that the DSL was most likely using some sort of instance_eval-based magic so we were able to use extend inside the block to get the other helper methods we needed by adding them to the instance that the code was being evaluated in.

I hope you found this closer look at include and extend illuminating or interesting and can use it next time you need to deal with some overly-metaprogrammed ruby code.


Nouns and Verbs: How Naming Your Classes Determines Effects Other's Behavior
comments

Last updated 06 July 2016

Recently I was reading a blog post by Arlo Belshee about naming that gave a good insight into how naming effects what methods other programmers will add to a class:

If you want a thing to collect functionality and grow, name it by what it is. If you want it to split and shrink, name it by what it does.

This is fundamentally about class names that are classes vs. verbs. Take a noun class name like User. What methods should go on that class? Potentially anything related to users in you app. It might be the Rails/ActiveRecord expectation of .save and .find, methods related to authenticating a password, or methods related to whether the user has paid his invoice yet. Depending on how core the concept of a “user” is to your app this can lead to the single responsibility principle going straight out the window. Naming the User class by what it is collects functionality.

Name tags

On the other hand, you have class names that are verbs. If you have some system for charging invoices to users you could have a class called InvoiceUser. What methods would be on that class? Probably just a method called invoice. If you had to check if a user had been invoiced or whether their invoice has been paid, you would be inclined to define a new verb-based class to do this for you. Naming a class by what it does resists adding functionality. This is great for classes with a single responsibility but it can lead to primitive obsession since there isn’t a clear place to collect common functionality between various invoice-and-user-related classes.

One way out of this is with a role. Think about the noun-based class name InvoiceableUser. Ignoring the fact that “invoiceable” isn’t really a word, this is a class that collects multiple pieces of functionality (unlike the verb-based class names), but resists being a dumping ground for totally unrelated functionality the way a generic noun (User) does.

None of these ways of naming classes is the “right” way. It really depends on your application what name is appropriate for a class. If users don’t have very much behavior in your code, maybe “everything related to users” is a reasonable single responsibility. If invoicing a user is relatively complicated, it makes a lot of sense to have an InvoiceUser class to encapsulate the process while resisting gaining any other functionality. The role-based noun approach is a good one to keep in your back pocket for situations where your needs are somewhere in between.


The Basics: N+1 Queries
comments

Last updated 29 June 2016

The basics is a new series where I explain a basic programming concept or term for new computer programmers or those who need a refresher.

Imagine that you are a waiter in a diner. You are serving a group of 10 people at a single table. One of them asks for a refill, so you walk to the back of the restaurant, refill their drink, and return to the table. As you deliver the drink another person at the table realizes they need a refill too, so you go through the same steps. Again, when you get back to the table another person wants a refill so you find yourself spending all your time walking back and forth from the table to the back of the restaurant, each time carrying only a single drink on a tray that could easily carry more.

After a while you realize you should ask the whole table before going to the back, “Does anyone else need a refill? Are you absolutely sure?”, your voice rising to a shrillness level usually reserved for small children. That way you can carry and fill multiple drinks, saving your time and sanity.

An N+1 query is when you do this same thing to your database. Lets see an example from a typical rails app:

# app/models/user.rb
class User < ActiveRecord::Base
  has_one :address
end

# app/models/address.rb
class Address < ActiveRecord::Base
  belongs_to :user
end

# app/controllers/users_controller.rb
class UsersController < ApplicationController
  def index
    @users = User.all
  end
end

# app/views/users/index.erb
<ul>
  <% @users.each do |user| %>
  <li>
    <%= user.name %>:
    <%= user.address.zip_code %>
  </li>
  <% end %>
<ul>

Everything looks great, you are looping through the users in the system and displaying their name and zip code. The problem, though, is that you are performing N+1 queries.

The “1” in N+1 is for User.all. The data for the users is fetched in a single query and all is well with the world. The problem comes from the “N” which means you are doing a database query for each user’s address separately (you do “N” lookups for “N” addresses). As a waiter, the solution to this problem was to get all the refill requests at the same time. The solution is the same here. Update your controller as follows:

# app/controllers/users_controller.rb
class UsersController < ApplicationController
  def index
    @users = User.includes(:address).all
  end
end

This causes Rails to look up all the addresses at once so you go from N+1 queries to 1+1 queries (more commonly known as 2 queries). Regardless of the technology you are using, you should look for N+1 queries and eliminate them by looking up all the records you will need as a group instead of individually.

I hope this little explanation will help you understand what other programmers are talking about when they say “N+1” and help you avoid this common mistake.


Tmux Scripting
comments

Last updated 16 April 2016

If you use tmux in your terminal you probably have a default set-up you use. For example, I always have the same windows open with the same names split up in the same way at work, so I decided I might as well make a script to avoid all that manual work each time.

The easiest way to do this is with a bash script. The basic trick is to start a tmux session detached and then send it signals. Here’s a very basic script to start a tmux session named vim with vim running in it and then attach:

#!/bin/bash

tmux new -s vim -d
tmux send-keys -t vim 'vim' C-m
tmux attach -t vim

The first line of this script (after the shebang bash line) says to make a new tmux session named vim but don’t attach to it. The second line sends the text vim to the target session vim then hits enter (represented by C-m). That will cause vim to start in the opened tmux session. Finally, we attach to the session vim. Simple, right?

Let’s see an example with more windows and splits:

#!/bin/bash

tmux new -s work -d
tmux rename-window -t work vim
tmux send-keys -t work 'vim' C-m

tmux new-window -t work
tmux rename-window -t work server
tmux send-keys -t work './bin/rails s' C-m
tmux split-window -v -t work
tmux send-keys -t work './bin/sidekiq' C-m
tmux select-window -t work:1
tmux attach -t work

This script is a little more complicated. First we start a new session called work and remain detached. Then we rename the window to vim. Then we start vim in that window. Next, we open a new window, rename it server and send the command ./bin/rails s to it (this generally starts the web server in a rails project). Next, we split the window and send ./bin/sidekiq to that split. Finally, we select window number 1 (I have my tmux windows start numbering at 1, but your might start at 0) and attach to the tmux session.

Some things to know that might help you:

  1. When you open a new window to split, your cursor is assumed to be in that pane for when you use send-keys. You generally want to send the keys to your current pane before splitting or opening a new window.
  2. If you don’t like to have your windows split into even-size panes there are ways to specify the size, but I find they can have wildly different effects if you use the same script on computers with different size monitors. As a result, I don’t worry about resizing panes in the script.
  3. You can put the name of your session in a variable (i.e. SESSION=work; tmux new -s $SESSION -d) so you can easily rename it in the future.

I think you can see how these scripts are starting to get useful. I had a problem, though. At my job we run a lot of our development stuff in a vagrant vm. So I wanted to start vagrant, and then run several commands that depend on vagrant having successfully started. Vagrant takes a while to start, however, so I needed to find a way to wait for a single command to complete before proceeding. Luckily, tmux can help us here.

#!/bin/bash

tmux new -s work -d
tmux send-keys -t work 'vagrant up; tmux wait-for -S vagrant-up' C-m\; wait-for vagrant-up
... other commands

The second line of this script does exactly what we need. The way it works is this: tmux send-keys sends a command to tmux and doesn’t care what happens as a result. After the send-keys command, though, we say wait-for vagrant-up, which means the script will wait until a wait-for command with the arbitrary name vagrant-up is sent from the tmux session. The way you send this command is tmux wait-for -S vagrant up, so our tmux session is actually sending that back to us after the vagrant up command has completed. Since the wait-for command has been sent, the script can continue with other commands that can assume vagrant is running.

I hope you have fun writing your own tmux scripts!


Ready, Set, Vim! Part 2 - Movement
comments

Last updated 23 November 2015

Now that you know how to move between normal and insert modes it’s time to actually do something in normal mode. We’ll start with the basic commands used to move around a file.

Your new arrow keys

One of the things vim enthusiasts love most about vim is that it keeps your hands on the home row as much as possible. One of the tricks to this are the basic “arrow” keys of vim: h, j, k, and l. You’ll notice that those keys are all in a row, under your right hand. In normal mode in vim h moves the cursor to the left, j moves it down, k moves it up, and l moves it to the right.

h and l are pretty obvious since those are the keys to the far left and right of the “arrow” keys. j and k are a little less intuitive. I don’t have a mnemonic device and, honestly, if you were to ask me point blank which of those keys moves up and which moves down I’d have a hard time telling you. In practice, though, you’ll quickly be hitting the j key to move down and the k key to move up without thinking about it, to the point where you’ll want every application to use those shortcuts (did you know that gmail uses them already?).

When you are first practicing these movement keys, you’ll frequently find yourself slipping into using the arrow keys. That’s fine. Just try to use h, j, k, and l whenever you remember and quickly your hands will gravitate towards using them since they are already right under your hand!

Searching around

Moving your cursor one line or character at a time will quickly get cumbersome in a big file. The other movement keys we will be learning today are / and ?. They used for searching inside a file. Type / and some characters to perform a search forward in a file (i.e. /test and then enter to move to the next instance of the characters “test” in that file) and use ? to do the same search but backwards (towards the top of the file). The pattern of using one key to move forward and shift plus that key to move backwards will come up again in vim.

What you know so far:

i # enter insert mode
esc # return to normal mode
h # move left
j # move down
k # move up
l # move right
/ # search forward in file
? # search backwards

Using these shortcuts in normal mode, you’ll already find that you can spend a lot more time with your hands on the letters of the keyboard, right where they belong, and away from those cumbersome arrow keys. Practice these commands until they become natural. Try to search at every opportunity. Sometimes I find myself searching to find something just a few words away because it can often be the fastest way to get there.

Next time we’ll learn some more movement commands that are line specific, but in the meantime: Get vimming!


Ready, Set, Vim! Part 1 - Modal Editing
comments

Last updated 22 November 2015

Programmers who don’t use either vim or emacs secretly wonder if they are missing out one some sort of text editor nirvana. Well, they aren’t. But I do think vim and emacs are better than the other editors I’ve tried. Not to mention all the cred you get for using an old school text editor! Since no one wants to use ed in their daily life you are left with only vim and emacs as your old school options.

While I’ve tried out emacs, I haven’t used it enough to tell anyone how to get started. Since I use vim every day, though, I thought it’d be helpful to set out the step-by-step program to learning vim I wish I had when I was starting. Most people will tell you to start with the excellent vimtutor (you can type that in the terminal to get a vim tutorial), but I think it gives you too much to remember too fast when you are an absolute beginner.

Installing vim

The first thing you need to do is install vim. Assuming you are coming from a non-terminal-based editor, I think you should start with a GUI version of vim such as mvim for macs and gvim for linux (probably just apt-get/yum install gvim?). Using a GUI vim means that you can use your mouse and copying and pasting like you are used to from other editors. You’ll probably stop using your mouse at some point, but it doesn’t make sense to worry about that as a first step.

Modal editing

The first thing to learn as a vim user is the key to what people love and hate about vim - modal editing.

Modal editing just means that your keystrokes mean different things to vim depending on what mode you are in. The main modes you will use are normal-mode, insert-mode, visual-mode, and command-line-mode. Right now don’t worry about all of those. We’ll just be learning about normal-mode and insert-mode. Insert-mode is the mode that other editors are in all the time. When you type a letter that letter appears on the screen. Normal-mode is the secret to vim’s power since typing letters doesn’t cause them to appear in the text, but instead executes vim commands. Some day you’ll be turning a_constant into A_CONSTANT by typing viwU, but that’s for the future. Normal mode is the scary mode you’ve hear about that causes people to open vim and not be able to type in anything.

Don’t be scared, though! I’ll ease you into normal mode. At first I expect you’ll move around using the arrow keys and mouse just like you are used to.

A big concern people have when hearing about vim is that they won’t know what mode they are. Like, “Am I in normal mode? Insert? Visual? I won’t know what my typing will do!” That doesn’t really happen in real life, though, because of what we are going to learn in this lesson. That lesson is: you should always be in normal mode unless you are actually typing in text.

So, this is the first thing you will do when using vim: When you are ready to enter text, type i to enter insert-mode. As soon as you have finished typing that text immediately press esc to return to normal mode. That’s it. The only other things you might need to know are :w to save and :q to quit, but in GUI vims you should be able to do ctrl/cmd+s to save and ctrl/cmd+q to quit (cmd being for macs and ctrl for linux).

What you know so far:

i # enter insert mode
esc # return to normal mode

I know what you are thinking, “That is stupid. I’m just typing more because of vim’s weird modes. I’ll just stay in insert mode all the time.” Okay, I understand that it is stupid right now, but if you get in this habit everything afterwards will be much easier and you’ll never be confused about what mode you are in.

Next time we’ll learn about moving the cursor the vim way, but in the meantime: Get vimming!


Digging Into Sass Placeholders
comments

Last updated 10 November 2015

Most examples of how to use placeholders in Sass are very simplistic and don’t really show why you might want to use them beyond some programmer-y sense of obligatory correctness. You usually see something like this:

%blue {
  color: blue;
}

.blue {
  @extend %blue;
}

Which compiles to:

.blue {
  color: blue;
}

That’s obviously not super useful, so I was wondering what placeholders are really for. Turns out you can use placeholders as part of more complex selectors, and then those get passed down to the extending selector.

That probably doesn’t really make sense, so let’s use an example:

p%blue {
  color: blue;
}

.blue {
@extend %blue;
}

Compiles to:

p.blue {
  color: blue;
}

Now we’re starting to see the use.

You can even have multiple selectors using the same placeholder.

div%blue {
  color: white;
  background-color: blue;
}

p%blue {
  color: blue;
}

.blue {
  @extend %blue;
}

p.text-blue {
  @extend %blue;
}

Compiles to:

div.blue {
  color:white;
  background-color:blue;
}

p.blue, p.text-blue {
  color:blue;
}

That definitely seems useful now. I hope this encourages you to be a little more interesting when using placeholders!


Sass's Leaky Abstraction
comments

Last updated 07 November 2015

Sass is an excellent tool for keeping css more concise and readable. It creates an abstraction on top of css allowing you to write other code that is then compiled to css.

All abstractions are leaky abstractions, however. I’ve been trying to up my front-end game lately and that has meant working on my sass skills. Recently, while learning more about the differences between extend and include, I’ve learned that sass doesn’t totally shield you from knowing how it is compiled since that knowledge is part of why you might choose one or the other.

First, let’s see how they are used.

Include

Sass uses mixins for including. Like this:

@mixin text() {
  font-size: 12px;
  font-weight: 400;
}

.normal {
  @include text();
}

p {
  @include text();
}

Extend

Extend is sort of like inheritence:

.normal {
  font-size: 12px;
  font-weight: 400;
}

p {
  @extend .normal;
}

You can also use placeholders (sort of like classes that don’t actually get used) with extend. That looks like this:

%text {
  font-size: 12px;
  font-weight: 400;
}

.normal {
  @extend %text;
}

p {
  @extend %text;
}

Now, these look essentially the same, right? But for some reason you can pass arguments to a @mixin and you can’t use @extend inside a media query. Why is that? What is the difference between these almost identical ideas?

The answer is that the difference doesn’t really have anything to do with sass. The difference is in how they are rendered as css.

Include is like copy-pasting. This sass:

@mixin text() {
  font-size: 12px;
  font-weight: 400;
}

.normal {
  @include text();
}

p {
  @include text();
}

Turns into this css:

.normal {
  font-size: 12px;
  font-weight: 400;
}

p {
  font-size: 12px;
  font-weight: 400;
}

So if you use an argument to a mixin sass can easily use that argument to generate the output since it is only used for that one selector.

Extend, on the other hand, works very differently. This sass:

%text {
  font-size: 12px;
  font-weight: 400;
}

.normal {
  @extend %text;
}

p {
  @extend %text;
}

Turns into this css:

.normal, p {
  font-size: 12px;
  font-weight: 400;
}

Since the css is only in one place, it can’t take arguments (it always needs to be the same) and it can’t appear in media queries. This makes the generated css much smaller, however.

So, learning when to use these sass features requires you to forget about sass’s abstractions and start thinking about the css. The general rule for sass is to use @extend unless you need one of the features that it doesn’t support since it generates slightly smaller stylesheets than using all @mixins.

Sass is an excellent tool, just make sure you spend some time thinking about the generated css since sass doesn’t let you forget it!