Proposal: Add a `hanami test` command

proposals
1

I propose we add hanami test as a command to hanami/hanami. We have rake now, but I think this is a good nice-to-have.

I’m willing to commit to building it, once we come to consensus on the requirements.

Rationale

This allows users to run a single test file/test in an agnostic way (instead of using rspec directly, e.g.)

It also signals that testing is a first-class concern of Hanami, appropriately.

Rails has this and it’s a nice convenience that I’ve used often. For reference, their defaults for just the test command are that only non-system tests are run, which I think is a reasonable convention to follow.

Slices

I think that running hanami test within inside the folder for a slice (or app/) should only run the tests in that slice.

Question: Should running hanami test in the main folder not only run tests in app/ but also run tests in the slices as well? I think so.

Regardless, we can also have an option to specify the slice from the main folder, e.g:
hanami test --slice admin only run the tests for the admin slice, then also hanami test --app can test just the main app (unless this is the default behavior, then it wouldn’t do anything).

Sub-commands

Looking at all the subcommands Rails has, I think we can get just implement a few key subcommands, as described below:

test                               Run all tests, except system tests
test:all                           Run all tests, including system tests
test:units                         Run tests in spec/actions, spec/models/, etc.
test:system                        Run all system tests

They also has test:db which resets the DB before testing, which seems fine and useful but not sure it’s needed for v1.

Name

Question: Should we stick to calling it hanami test and not support calling it hanami spec at all? This would greatly simplify things, especially in documentation and for this reason, I think we should. It also leaves the spec command namespace free for users to define as they wish.

1 Like
2

I wrote my own binstub to do something like the slice argument

#!/usr/bin/env ruby

ENV["HANAMI_ENV"] = "test"

if ARGV in ["ciam" | "scim" | "oauth", *]
  slice_name = ENV["HANAMI_SLICES"] = ARGV.shift
  ARGV.unshift "--tag", "slice:#{slice_name}"
end

require 'bundler/setup'
require 'rspec/core'
RSpec::Core::Runner.invoke

Proper framework support for test running with slice filtering would be nice.

3

This sounds like a good initiative for me, small but convenient addition that would save developers some scripting to have that functionality.

Should running hanami test in the main folder not only run tests in app/ but also run tests in the slices as well?

Seems to be in line with the overall convention so far, where the slice is a flag to be passed to the command, so no flag means a more general approach, flag means specific slice.

As for naming - many people use “test” and “spec” interchangeably. I usually try to differentiate, as per the simple definition of specs being short of “specification” meaning they are written per functionality and are more descriptive than general tests, also having a far more “human” description.

So, given that the command is more general than specific, I too would stick to naming it hanami test

I would be open to helping out in implementation if you go forward with it and need/want some help.

1 Like
4

Now that Hanami supports Minitest (--test=minitest), having a hanami test commmand seems even more valuable, since it’s not obvious how to run the minitest tests in a new Hanami app.

I’m currently relying on my own bin/test script to handle this:

#!/usr/bin/env ruby

ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__)
require "rubygems"
require "bundler/setup"

ENV["HANAMI_ENV"] ||= "test"
require "hanami/setup"
require "hanami/prepare"

$LOAD_PATH << Hanami.app.root.join("test")

paths = ARGV
  .reject { it.start_with?("-") }
  .map do |path|
    if Dir.exist?(path)
      Dir.glob(File.join(path, "**/*_test.rb"))
    elsif File.exist?(path)
      path
    else
      raise "missing `#{path}`"
    end
  end.flatten
paths = Hanami.app.root.glob("test/**/*_test.rb") if paths.empty?

paths.each do |file|
  Kernel.load file
end

require "minitest/autorun"

Usage:

bin/test
bin/test path/to/some_test.rb
bin/test test/features
bin/test -h
bin/test --pride --seed=42 -v test/features

Is there still interest in a hanami test command?

Coming from Rails, I’m really missing something built-in.

Also, I looked into adding the command to hanami-cli, but that definately warrants some discussion first - e.g, do we re-implement rspec/minitest options in hanami-cli, etc, etc

5

Maybe we should just go the hanami dev route, i.e,

  • generate a bin/test for the project (based on the user’s test framework)
  • have hanami test delegate to bin/test

That seems simpler, and avoids us having to try and mash everything into hanami test.