How to stub external services in tests
Requests to external services during test runs can cause several issues:
Tests failing intermediately due to connectivity issues.
Dramatically slower test suites.
Hitting API rate limits on 3rd party sites (e.g. Twitter).
Service may not exist yet (only documentation for it).
Service doesn't have a sandbox or staging server.
When integrating with external services we want to make sure our test suite isn’t hitting any 3rd party services. Our tests should run in isolation.
Disable all remote connections
We'll use Webmock, a gem which helps to stub out external HTTP requests. In this example we’ll search the GitHub API for contributors to the FactoryGirl repository.
First, let’s make sure our test suite can't make external requests by disabling them in our spec_helper.rb:
# spec/spec_helper.rb require 'webmock/rspec' WebMock.disable_net_connect!(allow_localhost: true)
Now let's verify that any external requests will raise an exception and break the build:
# spec/features/external_request_spec.rb require 'spec_helper' feature 'External request' do it 'queries FactoryGirl contributors on Github' do uri = URI('https://api.github.com/repos/thoughtbot/factory_girl/contributors') response = Net::HTTP.get(uri) expect(response).to be_an_instance_of(String) end end
As expected we now see errors when external requests are made:
$ rspec spec/features/external_request_spec.rb F Failures: 1) External request queries FactoryGirl contributors on Github Failure/Error: response = Net::HTTP.get(uri) WebMock::NetConnectNotAllowedError: Real HTTP connections are disabled. Unregistered request: GET https://api.github.com/repos/thoughtbot/factory_girl/contributors with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Host'=>'api.github.com', 'User-Agent'=>'Ruby'} You can stub this request with the following snippet: stub_request(:get, "https://api.github.com/repos/thoughtbot/factory_girl/contributors"). with(:headers => {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Host'=>'api.github.com', 'User-Agent'=>'Ruby'}). to_return(:status => 200, :body => "", :headers => {}) ============================================================ # ./spec/features/external_request_spec.rb:8:in `block (2 levels) in <top (required)>' Finished in 0.00499 seconds 1 example, 1 failure
We can fix this by stubbing any requests to api.github.com with Webmock, and returning pre-defined content.
# spec/spec_helper.rb RSpec.configure do |config| config.before(:each) do stub_request(:get, /api.github.com/). with(headers: {'Accept'=>'*/*', 'User-Agent'=>'Ruby'}). to_return(status: 200, body: "stubbed response", headers: {}) end end
Run the test again and now it will pass.
$ rspec spec/features/external_request_spec.rb . Finished in 0.01116 seconds 1 example, 0 failures
VCR
Another approach for preventing external requests is to record a live interaction and ‘replay’ it back during tests. The VCR gem has a concept of cassettes which will record your test suites outgoing HTTP requests and then replay them for future test runs.
Considerations when using VCR:
Communication on how cassettes are shared with other developers.
Needs the external service to be available for first test run.
Difficult to simulate errors.
We'll go a different route and create a fake version of the GitHub service.
Create a Fake (Hello Sinatra!)
When your application depends heavily on a third party service, consider building a fake service inside your application with Sinatra. This will let us run full integration tests in total isolation, and control the responses to our test suite.
First we use Webmock to route all requests to our Sinatra application, FakeGitHub.
# spec/spec_helper.rb RSpec.configure do |config| config.before(:each) do stub_request(:any, /api.github.com/).to_rack(FakeGitHub) end end
Next we'll create the FakeGitHub application.
# spec/support/fake_github.rb require 'sinatra/base' class FakeGitHub < Sinatra::Base get '/repos/:organization/:project/contributors' do json_response 200, 'contributors.json' end private def json_response(response_code, file_name) content_type :json status response_code File.open(File.dirname(__FILE__) + '/fixtures/' + file_name, 'rb').read end end
Download a sample JSON response and store it in a local file.
# spec/support/fixtures/contributors.json [ { "login": "joshuaclayton", "id": 1574, "avatar_url": "https://2.gravatar.com/avatar/786f05409ca8d18bae8d59200156272c?d=https%3A%2F%2Fidenticons.github.com%2F0d4f4805c36dc6853edfa4c7e1638b48.png", "gravatar_id": "786f05409ca8d18bae8d59200156272c", "url": "https://api.github.com/users/joshuaclayton", "html_url": "https://github.com/joshuaclayton", "followers_url": "https://api.github.com/users/joshuaclayton/followers", "following_url": "https://api.github.com/users/joshuaclayton/following{/other_user}", "gists_url": "https://api.github.com/users/joshuaclayton/gists{/gist_id}", "starred_url": "https://api.github.com/users/joshuaclayton/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/joshuaclayton/subscriptions", "organizations_url": "https://api.github.com/users/joshuaclayton/orgs", "repos_url": "https://api.github.com/users/joshuaclayton/repos", "events_url": "https://api.github.com/users/joshuaclayton/events{/privacy}", "received_events_url": "https://api.github.com/users/joshuaclayton/received_events", "type": "User", "site_admin": false, "contributions": 377 } ]
Update the test, and verify the expected stub response is being returned.
require 'spec_helper' feature 'External request' do it 'queries FactoryGirl contributors on Github' do uri = URI('https://api.github.com/repos/thoughtbot/factory_girl/contributors') response = JSON.load(Net::HTTP.get(uri)) expect(response.first['login']).to eq 'joshuaclayton' end end
Run the specs.
$ rspec spec/features/external_request_spec.rb . Finished in 0.04713 seconds 1 example, 0 failures
Voilà, all green! This now allows us to run a full integration test without ever having to make an external connection.
A few things to consider when creating a fake:
A fake version of a service can lead to additional maintenance overhead.
Your fake could get out of sync with the external endpoint.
Written by Harlow Ward










