jsborjesson Yet another dev blog

Vim macros behave strangely sometimes

03 Oct 2017

The vague title signifies how I've been categorising this issue in my head for a long time. There's basically a ghost messing with some of the macros I try to write, making them do something completely different, and making me look stupid and eventually giving up.

Broken macro

Unsuccessful Vim macro usage

In this gif I try to create a macro to change some comment headers in my dotfiles to a different style (one that creates Vim folds).

Basically I want to change this:

########################################
# Copy / paste
########################################

To this:


# }}}
# Copy / paste {{{

But somehow it just deletes the first line and then stops.

Infuriating.

The problem

If we inspect the recorded macro (you can paste it into the buffer with "qp, "paste from the q register"), we can see that this is what Vim saved:


C# }}}^[jA {{{^[jdd

Those ^[ are how Esc is represented in the terminal. It is not two characters, it is a so called control sequence. In fact, this is how all combinations of control and some other key are represented.

Pressing Ctrl + K is represented as ^K.

Pressing Esc is the same as pressing Ctrl + [.

It is frequently recommended to use Ctrl + [ instead of Esc in Vim, since it can be more ergonomic to press. This is not a separate mapping, Vim simply cannot tell the difference.

Even further, combinations of Alt (or Meta, or Option) and other keys, are represented as Esc followed by that key.

Pressing Alt + J is represented as ^[j

Pressing Alt + J is the same as pressing Esc followed by J.

Herein lies the issue. When Vim executes the macro, it thinks that we are pressing Alt + J, and not Esc followed by J. This issue reveals itself only in the macro, since when we push the actual keys there's a delay between pressing Esc and J. This timeout can be set very low, but a macro is always too fast. (See :help ttimeout)

The workaround

Using Ctrl + C will also get you out of insert mode, and using that in the macro works perfectly:

Successful macro usage

I've tried playing with the timeout settings to no avail, and <C-c> does not exactly equal escape, so mapping escape to that might cause some other issues. I hope understanding this issue can reduce the frustration for someone else dealing with this hiccup, but currently I don't know a reliable way of fixing it other than being aware and carefully avoiding it.

If anyone knows a way to fix this for good, please let me know.

Git reset refresher

21 Jun 2017

git reset is a confusing command, it is used for several different things in Git, and sometimes they seem unrelated.

  • git reset file.txt unstages file.txt
  • git reset --hard HEAD~ deletes the current commit

I've often come back to this article, which explains in detail how it works - I strongly recommend reading it. What follows is a short summary that makes sense to me, maybe it will help you as well.

It basically comes down to this:

Overview

Command move branch unstage changes discard changes
git reset --soft ✅    
git reset --mixed ✅ ✅  
git reset --hard ✅ ✅ ✅
git reset --   ✅  

Defaults

  • Running git reset <ref> is the same as running git reset --mixed <ref> (ref could be a branch name or a commit sha)
  • Running git reset <path> is the same as running git reset -- <path>

Summary

git reset works in three steps, on the three "trees" relevant to working with git:

  1. The ref (your current branch)
  2. The index (added/staged changes)
  3. The working directory (local changes to files)

Based on the mode (--soft/--mixed/--hard), it will go through this list, starting with changing where the current branch points, then updating the index to match, then discarding local changes and cleanly checking that out.

When provided a file argument, optionally separated by --, it skips the first step, since it doesn't make sense to point an entire branch to changes of specific files.

Differences from checkout

It seems like there should be a row that only has a check mark in the last column, what if you only want to discard your local changes without touching anything else? That's done with git checkout -- <path>, or simply git checkout <path>.

But wait, git checkout <ref> also moves something, right?

Yes, checkout moves the HEAD, as opposed to reset which moves the branch that HEAD points to. You can think of moving HEAD as simply turning your head to look at a different box of stuff, and moving the ref as actually ripping off the label from that box and placing it on a different box.

Implementing equality in Ruby

05 Jun 2017

Equality in Ruby is, at first glance, deceptively simple. Overriding the == method will get you almost all the way there, but if you want your objects to be consistently good citizens of Ruby-land, you need to think of a few more things.

Contents

Skeleton

We will be implementing equality on a class called T, it simply holds a value and doesn't reveal it back to us.

class T
  attr_reader :value
  private :value

  def initialize(value)
    @value = value
  end
end

Our test harness is equally simple:

require "minitest/autorun"

class EqualityTest < Minitest::Test
  def test_equality
    assert_equal T.new(1), T.new(1)
    refute_equal T.new(1), T.new(2)
  end
end

From now on I'll show only snippets of what we change, the complete example is shown at the end.

==

To make this test pass we need to implement the == operator

def ==(other)
  self.value == other.value
end

This gives us an error:

NoMethodError: private method `value' called for #<T:0x007f8c293b21a0 @value=1>

Since value is private, we cannot access it on a different instance, even if we are inside the same class. To fix this, we set it to protected instead. This means that we can look at other T instances' value from inside T, but it will still not reveal it to anyone else.

protected :value

Now our test is passing.

Edge cases

Our implementations covers the most basic case, but it's not quite enough. Consider the following test cases:

refute_equal T.new(1), Object.new
refute_equal T.new(1), nil

Both of these will break with an error like this:

NoMethodError: undefined method `value' for #<Object:0x007ff51916df58>

Let's adapt our == implementation to account for these edge cases:

def ==(other)
  return false unless other.is_a?(self.class)
  self.value == other.value
end

Now we have a pretty solid implementation of equality.

equal?

equal? determines if two objects are the exact same instance. You should never override this.


t_one = T.new(1)

t_one == T.new(1)      # => true
t_one == t_one         # => true
t_one.equal?(T.new(1)) # => false
t_one.equal?(t_one)    # => true

hash

Two equal objects should have the same hash code.

def test_hash
  assert_equal T.new(1).hash, T.new(1).hash
  refute_equal T.new(1).hash, T.new(2).hash
end

In our simple example we can simply delegate the hash method to our value:

def hash
  value.hash
end

When you have more instance variables, an easy way to implement hash is to offload the work to Array which already has a convenient implementation for this:

# Not needed in this simple example
def hash
  [value1, value2].hash
end

eql?

In Ruby, you can use any object as the key in a Hash (note the capital H).

h = { "key" => "value" }
h["key"] # => "value"

"key" == "key"           # => true (they have the same value)
"key".hash == "key".hash # => true (they have the same hash code)
"key".equal?("key")      # => false (but they are not the same instance of String)

Let's make sure our T behaves the same way.

def test_hash_key_access
  h = { T.new(1) => :val }
  assert_equal :val, h[T.new(1)]
end

It seems something is still missing from our implementation:

  1) Failure:
EqualityTest#test_hash_key_access [equality.rb:37]:
Expected: :val
  Actual: nil

Turns out, Hash uses both the hash method, and the eql? method to determine which value to retrieve. To fix this, all we need to do is alias our == implementation to eql?:

Reference

def ==(other)
  return false unless other.is_a?(self.class)
  self.value == other.value
end
alias eql? ==

And our tests are passing again.

<=>

If we want our Ts to be comparable with other Ts, we can implement comparison operators. Let's write a test first:

def test_comparisons
  assert_operator T.new(1), :<, T.new(2)
  refute_operator T.new(2), :<, T.new(1)
end

This is easy enough and looks a lot like our == implementation.

def <(other)
  self.value < other.value
end

Unlike our ==, which we want to simply say "no" when compared with something completely different, we want < to raise an error, because the comparison simply doesn't make sense. We cannot answer yes or no to whether a T is smaller than nil for example:

assert_raises(ArgumentError) { T.new(1) < nil }
assert_raises(ArgumentError) { T.new(1) < 1 }
assert_raises(ArgumentError) { T.new(1) < Object.new }

Though slightly different, the solution to this looks a lot like it did in ==:

def <(other)
  fail ArgumentError unless other.is_a?(self.class)
  self.value < other.value
end

Now all of our tests are passing again. We could go on and implement > the same way, but it quickly gets ridiculous once you get to <= and >=.

def test_comparisons
  assert_operator T.new(1), :<, T.new(2)
  refute_operator T.new(2), :<, T.new(1)
  assert_operator T.new(2), :>, T.new(1)
  refute_operator T.new(1), :>, T.new(2)
  assert_operator T.new(1), :<=, T.new(2)
  assert_operator T.new(1), :<=, T.new(1)
  refute_operator T.new(2), :<=, T.new(1)
  assert_operator T.new(2), :>=, T.new(1)
  assert_operator T.new(1), :>=, T.new(1)
  refute_operator T.new(1), :>=, T.new(2)

  assert_raises(ArgumentError) { T.new(1) < 1 }
  assert_raises(ArgumentError) { T.new(1) > Object.new }
  assert_raises(ArgumentError) { T.new(1) >= nil }
end

Thankfully, Ruby has a good answer for this: Comparable.

include Comparable

The contract of Comparable

Comparable needs you to implement the spaceship operator, which looks like this: <=>. This method has a contract that says essentially:

"Return 0 if the two objects are the same, return -1 if the first is smaller, return 1 if the first is bigger, and return nil if they cannot be compared".

You can try this out yourself:

1 <=> 1          # => 0
1 <=> 2          # => -1
1 <=> 0          # => 1
1 <=> Object.new # => nil

Implementing spaceship

Let's replace our == and < implementation with <=>:

def <=>(other)
  return nil unless other.is_a?(self.class)
  self.value <=> other.value
end
alias eql? ==

Now all of our tests are passing!

We delegate our spaceship operator to the values just like in ==, and we return nil in case the types don't match. This follows the Spaceship Contract.

There are a few things to take note of here:

  • We still need to alias eql? to ==, although now == is being implemented in Comparable so we can't see it.
  • Our == still returns false when compared to something strange, and our < still raises an ArgumentError. Comparable is smart like that.

Full example

Now we have a class that behaves well in every equality situation in Ruby-land. Did I miss anything? Please open an issue.

class T
  include Comparable

  attr_reader :value
  protected :value

  def initialize(value)
    @value = value
  end

  def <=>(other)
    return nil unless other.is_a?(self.class)
    self.value <=> other.value
  end
  alias eql? ==

  def hash
    value.hash
  end
end

require "minitest/autorun"

class EqualityTest < Minitest::Test
  def test_equality
    assert_equal T.new(1), T.new(1)
    refute_equal T.new(1), T.new(2)
    refute_equal T.new(1), Object.new
    refute_equal T.new(1), nil
  end

  def test_hash
    assert_equal T.new(1).hash, T.new(1).hash
    refute_equal T.new(1).hash, T.new(2).hash
  end

  def test_hash_key_access
    h = { T.new(1) => :val }
    assert_equal :val, h[T.new(1)]
  end

  def test_comparisons
    assert_operator T.new(1), :<, T.new(2)
    refute_operator T.new(2), :<, T.new(1)
    assert_operator T.new(2), :>, T.new(1)
    refute_operator T.new(1), :>, T.new(2)
    assert_operator T.new(1), :<=, T.new(2)
    assert_operator T.new(1), :<=, T.new(1)
    refute_operator T.new(2), :<=, T.new(1)
    assert_operator T.new(2), :>=, T.new(1)
    assert_operator T.new(1), :>=, T.new(1)
    refute_operator T.new(1), :>=, T.new(2)

    assert_raises(ArgumentError) { T.new(1) < 1 }
    assert_raises(ArgumentError) { T.new(1) > Object.new }
    assert_raises(ArgumentError) { T.new(1) >= nil }
  end
end

Exemplary design

03 Nov 2016

I recently wrote a simple command line tool, the functionality of which is unimpressive and uninteresting, but it illustrates a principle I've had on my mind for a while.

The tool is called in that case and converts between different capitalization standards:

$ itc --snake inThatCase
in_that_case

It detects which convention is being passed in, converts it to the convention you requested, and can even be part of pipelines:

$ echo inThatCase | itc --dash
in-that-case

This post will use this to discuss the topic of Exemplary Design. (Exemplary as in it setting an example, not it necessarily being good)

Discovering the project

Imagine you come across this tool, it does what you want but is lacking one of the capitalization styles that you need. You might think that "there's probably just one file with a huge if-statement, maybe I can try to extend it…".

So you start poking around the project. You might think that there seems to be a lot of files for such a simple problem, and that it looks too complicated, but nonetheless you quickly stumble upon the conventions/ folder which seems to have a file for each convention it supports.

Adding a new style

You want it to support dot-case, which looks like this: in.that.case.

You don't feel like getting into the project that much, you just want this damn thing to support one more style, and at least this conventions/ folder seems easy enough to understand, so you decide to give it 10 minutes and break out your most sophisticated programming tool and copy paste one of the convention files. While you're at it you decide to copy paste one of the spec-files too. Can't hurt.

Now you have essentially added this:

Copy pasted files:

# lib/in_that_case/conventions/snake_case.rb
require "in_that_case/convention"

module InThatCase
  module Conventions
    module SnakeCase
      extend Convention

      module_function

      def convert(words)
        words.join("_")
      end

      def extract_words(str)
        str.split("_")
      end

      def matches?(str)
        !!(str =~ /\A[a-z]+(_[a-z0-9]+)+\z/)
      end
    end
  end
end

# spec/in_that_case/conventions/snake_case_spec.rb
require "spec_helper"
require "in_that_case/conventions/snake_case"

RSpec.describe InThatCase::Conventions::SnakeCase do
  include_examples "convention"

  it ".extract_words" do
    expect(described_class.extract_words("in_that_case")).to eq %w[in that case]
  end

  it ".convert" do
    expect(described_class.convert(%w[in that case])).to eq "in_that_case"
  end
end

Change everything blindly

All the files in the conventions/ folder look pretty similar, so you decide to go through what you copied and change the obvious things:

  • you rename the files and every occurrence of "snake" inside them to "dot"
  • you change the 2 specs you copied to have dots instead of underscores
  • you go through the implementation file and replace the underscores with dots too, there's a kind of nasty looking regex, but it has an underscore in it so let's take that one as well (a dot in regex has to be escaped \.).

Phew, that was really boring. Now you have some very similar looking files, with different names and very slightly changed implementations:

Altered files:

# lib/in_that_case/conventions/dot_case.rb
require "in_that_case/convention"

module InThatCase
  module Conventions
    module DotCase
      extend Convention

      module_function

      def convert(words)
        words.join(".")
      end

      def extract_words(str)
        str.split(".")
      end

      def matches?(str)
        !!(str =~ /\A[a-z]+(\.[a-z0-9]+)+\z/)
      end
    end
  end
end

# spec/in_that_case/conventions/dot_case_spec.rb
require "spec_helper"
require "in_that_case/conventions/dot_case"

RSpec.describe InThatCase::Conventions::DotCase do
  include_examples "convention"

  it ".extract_words" do
    expect(described_class.extract_words("in.that.case")).to eq %w[in that case]
  end

  it ".convert" do
    expect(described_class.convert(%w[in that case])).to eq "in.that.case"
  end
end

You try running the specs to make sure you at least didn't break anything:

$ bundle exec rspec

You see that all of your new specs are passing, and all the old ones still pass as well. Great! The boring part is over, let's see what's left to do to hook this in. You try running the binary just to see what breaks:

$ exe/itc --dot myTestingStuff
my.testing.stuff

What? It's already works? You look at the help message to make sure that you don't forget to add documentation:

$ exe/itc --help
In That Case

Usage:
  exe/itc (--camel | --dash | --dot | --pascal | --snake) (<input> | -)
  exe/itc -h | --help

Options:
  -h --help      Show this screen.
  --camel        Convert to camelCase.
  --dash         Convert to dash-case.
  --dot          Convert to dot.case.
  --pascal       Convert to PascalCase.
  --snake        Convert to snake_case.

Also already there apparently. You commit your changes and move on with your life.

Motivation

I wanted to write about exemplary code, and I think this project illustrates what that means. Not because it's particularly good, but because it sets a very clear example.

To add support for a new feature there was very little to understand. Most of the time was spent mundanely copy pasting a file, going through it and renaming stuff. None of the rest of the project had to be touched at all, and in the end the diff is exactly adding a new, very boring looking file, and an equally boring and obvious looking spec. Most Ruby programmers could probably have done this in their sleep.

This kind of boring is very powerful. It is respectful of people's time. They do not have to go digging for code that is of no interest to them, and if you hand this task to a less experienced developer, they would likely produce the same, obvious and boring code.

One who digs a bit deeper might say that this is over-engineering, that the code is too clever, and that there is too much of it to solve such a simple problem - and they would be right, for a project of this scale it is a bit overkill. Here's a list of stuff that I did to make this possible:

  • Dynamically require every file in the conventions/ directory.
  • Automatically add all the Convention modules from this directory to a list of supported styles.
  • Derive the name of the command line arguments from the class names.
  • Generate a representation of each of these conventions based on their class names and their own convert implementation.
  • Use these to dynamically generate the help text of the executable.
  • Metaprogram specs to check the matches? method against all the other conventions to make sure there's no overlap/order dependency.

This is a bit of extra work, but all of it falls on me. Anyone else wanting to touch the code in the future will have an easier time.

This is what I consider to be exemplary code. Even without understanding it, it leads you down the path of making correct decisions. Writing bad code would have been harder than writing good code. By simply existing it prevents the code quality from decaying.

The alternative

A big if-statement would have been faster, shorter, and probably easier to understand at first glance - the whole project could have been one file. You could very well argue that doing that would have been the correct decision for a tool like this. YAGNI. If it needs to be more complex we can refactor it. Don't do big up front design. Make the smallest/simplest thing that works. I've made all of these arguments myself.

Imagine adding another style if the project was designed like that - it invites legacy code. You know you should refactor it, but it's just another little if-statement. Just like that the code which was previously a well balanced compromise between time and functionality lays out a labyrinth of design decisions ahead of you, even though you just want to add this little thing.

Exemplary code paves the way.

Git merge a different repository

13 Jul 2016

Ever wanted to merge a completely different repository? It's actually not very hard, but you should consider whether it actually makes sense to do so.

# We want to merge repo1 into repo2
$ cd repo2

# Add the first repository as a remote and fetch its history
$ git remote add repo1 ../repo1
$ git fetch repo1

# Now merge it
$ git merge repo1/master
fatal: refusing to merge unrelated histories

# If the repos are entirely unrelated git will refuse
# Consider again whether this makes sense - you can force the merge with this option:
$ git merge repo1/master --allow-unrelated-histories

If the repos are not in neat non-conflicting folders you'll likely have some merge conflicts that you need to solve on the way.

# If you want to clean up a bit you can now remove the remote
$ git remote remove repo1

And you're done!