Created
April 12, 2013 19:28
-
-
Save eprothro/5374472 to your computer and use it in GitHub Desktop.
Revisions
-
eprothro revised this gist
Apr 12, 2013 . 1 changed file with 1 addition and 52 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -82,55 +82,4 @@ octopus: <% end %> <% else %> - none <% end %> -
Apr 12, 2013 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,136 @@ <% require 'cgi' require 'uri' def attribute(name, value, force_string = false) if value value_string = if force_string '"' + value + '"' else value end "#{name}: #{value_string}" else "" end end configs = case Rails.env when 'development', 'test' # use dev and test DB as feaux 'follower' Array.new(2){YAML::load_file(File.open("config/database.yml"))[Rails.env]} else # staging, production, etc with Heroku config vars for follower DBs master_url = ENV['DATABASE_URL'] slave_keys = ENV.keys.select{|k| k =~ /HEROKU_POSTGRESQL_.*_URL/} slave_keys.delete_if{ |k| ENV[k] == master_url } slave_keys.map do |env_key| config = {} begin uri = URI.parse(ENV["#{env_key}"]) rescue URI::InvalidURIError raise "Invalid DATABASE_URL" end raise "No RACK_ENV or RAILS_ENV found" unless ENV["RAILS_ENV"] || ENV["RACK_ENV"] config['color'] = env_key.match(/HEROKU_POSTGRESQL_(.*)_URL/)[1].downcase config['adapter'] = uri.scheme config['adapter'] = "postgresql" if config['adapter'] == "postgres" config['database'] = (uri.path || "").split("/")[1] config['username'] = uri.user config['password'] = uri.password config['host'] = uri.host config['port'] = uri.port config['params'] = CGI.parse(uri.query || "") config end end whitelist = ENV['SLAVE_ENABLED_FOLLOWERS'].downcase.split(', ') rescue nil blacklist = ENV['SLAVE_DISABLED_FOLLOWERS'].downcase.split(', ') rescue nil configs.delete_if do |c| ( whitelist && !c['color'].in?(whitelist) ) || ( blacklist && c['color'].in?(blacklist) ) end %> octopus: replicated: true fully_replicated: false environments: <% if configs.present? %> <%= "- #{ENV["RAILS_ENV"] || ENV["RACK_ENV"] || Rails.env}" %> <%= ENV["RAILS_ENV"] || ENV["RACK_ENV"] || Rails.env %>: followers: <% configs.each_with_index do |c, i| %> <%= c.has_key?('color') ? "#{c['color']}_follower" : "follower_#{i + 1}" %>: <%= attribute "adapter", c['adapter'] %> <%= attribute "database", c['database'] %> <%= attribute "username", c['username'] %> <%= attribute "password", c['password'], true %> <%= attribute "host", c['host'] %> <%= attribute "port", c['port'] %> <% (c['params'] || {}).each do |key, value| %> <%= key %>: <%= value.first %> <% end %> <% end %> <% else %> - none <% end %> ``` This configuration uses the environmental variables that Heroku sets up when you create your heroku-postgresql add-on databases to automatically set up any slaves that are present. This assumes that you desire all non-primary databases to be used as read-only slaves. If you don't, see the 'More Info and Options' section below for your options. Your primary database will be configured as usual by the configuration that [Heroku injects into database.yml](https://devcenter.heroku.com/articles/ruby-support#build-behavior). How closely your followers (slaves) follow master is application specific, so the followers are not configured to automatically send all read queries to the followers by default. In Octopus lingo, we have not configured to be 'fully replicated'. Mark the appropriate AR models by setting `replicated_model`: ``` class StaticThing < ActiveRecord::Base replicated_model end ``` This results in using your followers for read queries, and master for write queries on that model. This is appropriate for models that won't yield unexpected behavior when read queries come from a slave that may be a few seconds behind the master they follow. That's everything required to get started. You can read more about Octopus to learn how to use the `using` methods in controllers, models and AR relations in a more granular fashion, if needed. If you do, read below about the `Octopus.followers` monkey-patch to ensure your code is ready for future scaling, which is necessary until using_group functionality is more robust. ## Initializer (Recommended) Add the following to `config/initializers/octopus.rb` for: * Convenient logging of the slaves configured at app initialization * Use of `Octopus.followers` to retrieve configured followers * Example: `StaticThing.using(Octopus.followers).all` ``` module Octopus def self.shards_in(group=nil) config[Rails.env][group.to_s].keys end def self.followers shards_in(:followers) end class << self alias_method :followers_in, :shards_in alias_method :slaves_in, :shards_in end end if Octopus.enabled? count = case (Octopus.config[Rails.env].values[0].values[0] rescue nil) when Hash Octopus.config[Rails.env].map{|group, configs| configs.count}.sum rescue 0 else Octopus.config[Rails.env].keys.count rescue 0 end puts "=> #{count} #{'database'.pluralize(count)} enabled as read-only #{'slave'.pluralize(count)}" if Octopus.followers.count == count Octopus.followers.each{ |f| puts " * #{f.split('_')[0].upcase} #{f.split('_')[1]}" } end end