19: Testing your site (Part 1)

Summary: <h2>Goals</h2> In this lesson, we’re going to step back and look at a subject we’ve neglected: Testing. First, we’ll take a brief tour to understand the support Ruby and Rails provides us for using tests in our regular development. Then, we will take stock of the current status of our code and make sure it is on solid footing with testing. Please note that, while we’ve tried to make these notes complete, they aren’t the full tutorial; that’s in the screencast, which you can access via the link on the left. <h2>Setup</h2> We begin with the code with which we ended Lesson 18. These zip files contain the beginning and ending states of the code: <ul> <li><a href="/learningrails_18.zip">Learning Rails example app code as of the start of this lesson</a></li> <li><a href="/learningrails_19.zip">Learning Rails example app code as of the end of this lesson</a></li> </ul> <h2>A Brief Tutorial on Testing</h2> <h3>Why Write Tests?</h3> In our previous 10 screencasts, we’ve built a simple Ruby on Rails site. To move along as fast as possible, we’ve neglected testing, and in this lesson, we’re going to remedy that. Tests provide a safety net that helps you improve the quality of your code. You can certainly write significant Rails applications without writing tests — I have to admit I’ve done so myself, and I suspect that many Rails developers are continuing to do so. Rails doesn’t force you to write tests. But it is one of the hallmarks of professional-quality code that it has complete tests. In the short term, writing tests adds one more thing you need to do, and another set of syntax and technology you need to learn. There’s no doubt that this takes more time up-front. But in the long run, writing tests can save you time, since problems are easier to track down when they do occur. Many developers advocate writing tests even before you write the code, which forces you to think through exactly what your code is supposed to do. Tests are helpful in finding the corner cases that you might not remember to test manually, and in being able to automatically test all aspects of your code whenever you make a change. Without tests, it’s common for a change you make to fix one thing to accidentally break something else, which you might not notice right away. Tests are especially valuable when you’re doing major rework on your code, commonly called refactoring. With a good suite of tests, you can be confident after you’ve rewritten some code that it hasn’t broken anything. <h3>Testing Background</h3> Many studies have shown that the cost of fixing bugs rises with time. Imagine if you had a “quality safety net” that helped protect you by checking basic assumption about your code and notified you if something broke those assumptions. If you had such a tool, it would give you greater confidence as you added or refactored code in your program. The Ruby on Rails community has adopted many so-called Agile practices, and code testing is but one. The core Rails team has made it very easy to implement testing in your program by building in simple testing tools in the standard Rails installation. Whether you choose to write tests after coding parts of your program (“Test After Development” – TAD) or before you write a line of code (“Test First or Test Driven Development” – TDD), Rails will accommodate you. Ruby on Rails has built in support for a variety of testing scenarios. Rails currently uses the Ruby Test::Unit library to implement three kinds of tests: Unit, Functional, and Integration Tests. Rails’ usage of the names for testing terms is slightly different than common meaning. In Rails, Unit tests are tests that exercise Model objects. Functional tests focus on testing controllers. Integration tests are meant to test multi-step workflows that trigger one or more actions, potentially across multiple controllers. A fourth type, “acceptance tests” is sometimes used to mean tests that exercise workflows from a user perspective, usually by automatically triggering actual view code. We’ll skip that in this screencast. <h3>Testing in Rails</h3> When you generate a new Rails application, you will note that a test directory is created. Inside of test, you will see a number of directories: fixtures, functional, integration, mocks, and unit. Unit, functional, and integration directories match up with the test types we just defined. The fixtures directory is used to store test data that can be automatically loaded inside of test code to simulate a known state of your program. Commonly, fixtures are written in YAML, although they can be stored in other formats and can also use Erb to implement dynamic data generation. Mock objects are objects that stand-in for real code, replacing expensive (resource-wise), external, or state-changing code. The mocks directory was originally intended as a place you could drop code that would replace other code in your project during testing. This practice is largely deprecated in favor of using external libraries such as “Mocha” or “FlexMock”. We’ll passover this are in this screencast. When you use one of the script/generator scripts for making models, controllers, or even scaffold, they will generate pre-populated test code in the proper directory. You can make your own test files too. A test is simply a class that is derived from one of several parent class that Rails provides. Parent classes implement functionality appropriate for the kind of test, e.g. ActionController::TestCase sets up an environment that allows you to simulate calling a controller. Behind all of these Rails provided classes is the Ruby library called Test::Unit. A test class is made up of one or more methods. Each method examines one aspect of the object under test. The checks we do in a test method are called “assertions”. The testing framework provides a wide variety of helper methods that we can use to write assertions. Most assertions boil down to checking whether some condition is true or false. For instance, in our <code>links_controller_test.rb</code> file, we can find a test that checks whether we can get the <code>new</code> action’s form: <pre> def test_should_get_new get :new assert_response :success end </pre> We see one assertion using the <code>assert_response</code> helper method. <code>assert_response</code> examines the HTTP response object that comes back from doing an HTTP GET on the @LinkController@’s <code>new</code> action. We are asserting that the response code will be a success code, such as 200. We’ll see many different kinds of assertions as we write tests. You will note that certain naming convention are in play for tests, just like elsewhere in Rails. Test files end with the suffix “_test” and test methods inside of these files start with the “test_” prefix. As we look inside of the test subdirectories, we’ll see a lot of files that have been generated as we’ve worked on our CMS project. How do we use these tests? <h3>Running Tests</h3> Rails provides a complete environment for running your tests. Look inside the config directory and you will see an environment/test.rb file. This sets up testing specific options within the Rails framework to maximize the amount of testing checks that occur. Database.yml specifies a separate test environment as well. The test oriented database keeps your test data separated from your development or production data and allows the test tools to set up pristine testing conditions as needed. You can ascertain your test database is ready to go with the <code>rake db:test:prepare</code> command. Let’s do that now. Ruby and Rails provides a number of ways to run our tests. The easiest manual way is to fire up the <code>rake test</code> command. This command will run all of our tests: unit, functional, and if present, integration. We can run a subset of our tests by specifying the type, such as <code>rake test:units</code>. Let’s take a look at how the project is fairing at the moment: <pre> rake test </pre> This runs our tests and we see a lot of diagnostics go by. Let’s examine a few critical pieces of information. First, a test can be in one of three states: <ol> <li>Success, indicated with a period (<code>.</code>), sometimes also called “green”;</li> <li>Failure, indicated with an <code>F</code>, (called “red”), which means one of the assertions we wrote to test an assumption about our code did not work;</li> <li>Error, indicated by an <code>E</code>, (also called “red”), which means a program logic problem was detected in our test or application code.</li> </ol> The rake test results print out a sequence of progress characters for each test run: <pre> ....FFFF....E... </pre> Second, each test that fails or errors out will display relatively detailed information about what went wrong. This is your first line of defense to track the program down: <pre> 3) Failure: test_should_get_edit(CategoriesControllerTest) [./test/functional/categories_controller_test.rb:30:in `test_should_get_edit' /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/active_support/testing/default.rb:7:in `run']: Expected response to be a <:success>, but was <302> </pre> The testing environment in Rails also has its own log file, so you can peek inside of log/test.log for more information too. Armed with this information, let’s start cleaning up our CMS program’s broken tests. <h2>Cleaning up the tests</h2> <h3>Unit Tests</h3> We are working through our tests type by type. First up is unit tests and on running <code>rake test</code>, generally it looks good, only one problem, right? Yes and no. The error comes from our mailer test, which we’ll fix shortly. It turns out that the other model tests are largely empty. We’ll demonstrate adding a test to one of these files and leave it as an exercise to you to fill in more tests. Fortunately some plugins, such as the restful_authentication plugin we used, generate their own tests, and our user and session models start out with some useful tests that you can examine. Take a peek at user_test.rb. Lots of good tests for verifying that the User model is working. Let’s look at the tests for our Message model, since it is used by our mailer. The default scaffold generator creates a test file that contains an empty test: <pre> require File.dirname(__FILE__) + '/../test_helper' class MessageTest < ActiveSupport::TestCase # Replace this with your real tests. def test_truth assert true end end</pre> That isn’t too useful. Recall that the Message model does some validation. Let’s test those validators: <pre> class Message < ActiveRecord::Base validates_presence_of :name, :subject, :body validates_format_of :email, :with => /^(\S+)@(\S+)\.(\S+)$/ end </pre> We add a variety of tests in MessageTest: <pre> def setup @message = Message.create(:name => 'Bob', :email => 'bob@example.com', :company => 'Acme', :phone => '123.456.7890', :subject => 'test subject', :body => 'please test me') end def test_valid_model assert_valid @message end def test_missing_required_attributes assert_equal false, Message.new.valid? end def test_requires_name @message.name = nil assert_equal false, @message.valid? assert_equal "can't be blank", @message.errors[:name] end def test_requires_subject @message.subject = nil assert_equal false, @message.valid? assert_equal "can't be blank", @message.errors[:subject] end def test_requires_body @message.body = nil assert_equal false, @message.valid? assert_equal "can't be blank", @message.errors[:body] end def test_does_not_require_phone @message.phone = nil assert_valid @message end def test_poor_email_formatting @message.email = 'spammer-no-domain' assert_equal false, @message.valid? assert_equal "is invalid", @message.errors[:email] end </pre> The setup method is called before each test method is run and sets up a known correct example of our <code>Message</code> object for us to use. We check whether the object is valid, we see that a new empty <code>Message</code> is not valid (it fails validation on our required attributes), and proceed to test our required attributes and email formatting rules. We can use a different technique for setting up test data by using fixtures. First, lets set up the fixture files links.yml: <pre> learningrails: url: http://learningrails.com/ title: LearningRails podcast home page description: Pod and screencast dedicated to learning the Ruby on Rails framework categories: ruby, rails google: url: http://google.com/ title: Google description: Widely used search engine categories: search </pre> and categories.yml: <pre> ruby: title: Ruby description: Ruby the Programming Language links: learningrails rails: title: Ruby on Rails description: Rails is a cool framework links: learningrails search: title: Search Engine description: Used to find things on the web links: google </pre> Fixtures are smart about associations. We can specify attributes and associations. The associations are set up using the name of the association, then the id of the related fixture. The id is simply the first non-indented word of each YAML block. When the test code loads up these fixtures, it automatically sets up the associations. Now we can use these fixtures in our link_test.rb file: <pre> require File.dirname(__FILE__) + '/../test_helper' class LinkTest < ActiveSupport::TestCase fixtures :categories, :links def setup @link = Link.create(:url => 'http://buildingwebapps.com/', :title => 'BuildingWebApps.com', :description => 'Resource for Web Developers') @category = Category.create(:title => 'Programming', :description => 'All about programming') end # Replace this with your real tests. def test_valid_model assert_valid @link end def test_valid_from_fixture assert_valid links(:learningrails) end def test_has_categories link = links(:learningrails) assert_valid link assert !link.categories.nil? assert_equal 2, link.categories.length end def test_add_category assert_valid @link assert_valid @category assert @link.categories.empty? assert @category.links.empty? assert_difference "@link.categories.length" do @link.categories << @category end @link.reload @category.reload assert !@link.categories.empty? assert !@category.links.empty? assert_equal 1, @link.categories.length assert_equal 1, @category.links.length end end </pre> Note the <code>fixtures</code> call near the top. This explicitly tells the test code to load the named fixture files into the test database before each run of the tests. Like before, we also create a couple of hard-coded objects to use in some of our tests. <code>test_valid_model</code> is exactly like before. <code>test_valid_from_fixture</code> demonstrates how you can reference an instance of a fixture data object instead. Here we refer to the links fixture data with the id <code>:learningrails</code>. The rest of the tests exercise the <code>has_and_belongs_to_many</code> association. We’ll leave testing the other model’s to you. Before we leave unit tests, though, let’s fix up the mailer test, since it is a little different. <pre> 1) Error: test_message(ContactMailerTest): NoMethodError: undefined method `subject' for Thu Jun 26 14:53:52 -0700 2008:Time /Users/chaupt/Documents/Business Documents/CollectiveKnowledgeworks/Podcasts/sc11/learningrails_19/app/models/contact_mailer.rb:4:in `message' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:410:in `__send__' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:410:in `create!' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:403:in `initialize' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:351:in `new' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:351:in `method_missing' ./test/unit/contact_mailer_test.rb:10:in `test_message' /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/active_support/testing/default.rb:7:in `run' 18 tests, 30 assertions, 0 failures, 1 errors </pre> This gives us a clue as to what went wrong and where. The <code>ContactMailerTest</code> class’ <code>test_message</code> method had a programming error in it: an undefined method ‘subject’. This test is exercising the pseudo model that was created when we generated our contact mailer. The test inside of the test class looks like this: <pre> def test_message @expected.subject = 'ContactMailer#message' @expected.body = read_fixture('message') @expected.date = Time.now assert_equal @expected.encoded, ContactMailer.create_message(@expected.date).encoded end </pre> Mailer related unit tests are a little strange. What this code is trying to do is test to see if the mail message that gets generated by the <code>ContactMailer.message</code> method is identical to a canned email fixture. <code>@expected</code> is an instance variable created for us by the testing class that embodies a TMailer object and which we’ll load with data and subsequently ask to generate a properly formed email body and headers (that is what <code>encoded</code> does). Here we see that the other special calling convention on a mailer is being used: <code>create_message</code>. Like <code>deliver_message</code>, this method uses the <code>message</code> method we wrote, but rather than building a mail message and then actually emailing it out, this form just returns a string that contains the body of the email message. To fix this test, we are going to take a different approach. Since we generate a multipart email (plain text and html), we’ll generate a mail message then probe it for correct results. We’ll use Ruby’s inline document string notation to specify a test value for our message body (rather than a fixture file). Note we could put some of this in a setup method like before, but in versions of Rails before 2.1, setup was slightly broken for mail tests: <pre> def test_message # using a the setup method is not functioning properly for Rails prior to 2.1. This should # rightly be put in such a setup file @message = Message.new(:name => 'Bob', :email => 'bob@example.com', :company => 'Acme', :phone => '123.456.7890', :subject => 'test subject', :body => 'please test me') test_body = <<EOF Email from your web site From: Bob Company: Acme Phone: 123.456.7890 Message: please test me EOF created = ContactMailer.create_message(@message,@expected.date) assert_equal 2,created.parts.size assert_equal "multipart/alternative", created.content_type assert_equal "text/plain", created.parts[0].content_type assert_equal "text/html", created.parts[1].content_type assert_equal test_body, created.parts[0].body end </pre> We save and run this now, and Success! Our admittedly mostly empty unit tests are all running in the “green” state. Let’s go on to fix up some of our functional tests. <h2>Wrap up</h2> We will continue testing in the next episode where we focus on the functional tests.

19: Testing your site (Part 1)

Directory

Click for all Categories