Last active
December 14, 2015 14:18
-
-
Save erwanlr/5099652 to your computer and use it in GitHub Desktop.
Revisions
-
erwanlr revised this gist
May 17, 2013 . 1 changed file with 1 addition and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,5 +1,6 @@ source "http://rubygems.org" gem 'typhoeus', '>=0.6.2' #gem 'typhoeus', :git => 'git://github.com/typhoeus/typhoeus.git' gem 'webmock', '>=1.9.3' gem 'rspec', :require => 'spec' -
erwanlr revised this gist
May 17, 2013 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -13,7 +13,7 @@ context "when concurrency = #{concurrency}" do it 'returns all the files' do foo.retrieve_files(concurrency).sort.should == foo.urls.sort end end end -
erwanlr revised this gist
Mar 6, 2013 . 1 changed file with 4 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -40,3 +40,7 @@ def self.files %w{file1.php file2.php file3.php file4.php} end end # uncomment this for real case # foo = Foo.new('http://google.com') # p foo.retrieve_files -
erwanlr created this gist
Mar 6, 2013 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,5 @@ source "http://rubygems.org" gem 'typhoeus', '>=0.6.2' gem 'webmock', '>=1.9.3' gem 'rspec', :require => 'spec' This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,4 @@ # spec/spec_helper.rb require 'webmock/rspec' require File.expand_path(File.dirname(__FILE__) + '/../test') This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,42 @@ require "rubygems" require "bundler/setup" require 'typhoeus' class Foo attr_reader :target_url def initialize(target_url) @target_url = target_url end def retrieve_files(max_concurrency = 2) hydra = Typhoeus::Hydra.new(:max_concurrency => max_concurrency) found = [] urls.each do |url| request = Typhoeus::Request.new(url) request.on_complete do |response| found << response.request.base_url end hydra.queue(request) end hydra.run found end def urls urls = [] Foo.files.each {|file| urls << "#@target_url/#{file}"} urls end def self.files %w{file1.php file2.php file3.php file4.php} end end This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,21 @@ # spec/test_spec.rb require 'spec_helper' describe Foo do let(:url) { 'http://google.com' } subject(:foo) { Foo.new(url) } before { stub_request(:get, /.*/).to_return(status: 200) } describe '#retrieve_files' do (1..4).each do |concurrency| context "when concurrency = #{concurrency}" do it 'returns all the files' do foo.retrieve_files(concurrency).should == foo.urls end end end end end