Forked from xpepper/Massive file upload using JQueryUpload + Nginx + Rails + Sidekiq.md
Last active
August 29, 2015 14:11
-
-
Save kudrin/a00dd0f55baf72933a33 to your computer and use it in GitHub Desktop.
Revisions
-
xpepper revised this gist
Apr 28, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -2,7 +2,7 @@ The goal: Upload several files (virtually big files) to a Rails application without compromising the user experience. ## Architecture jQuery File Upload + Nginx + Rails (Carrierwave) + Sidekiq -
xpepper revised this gist
Apr 28, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,4 +1,4 @@ # Massive file upload using JQueryUpload + Nginx + Rails + Sidekiq The goal: Upload several files (virtually big files) to a Rails application without compromising the user experience. -
xpepper renamed this gist
Apr 28, 2014 . 1 changed file with 0 additions and 0 deletions.There are no files selected for viewing
File renamed without changes. -
xpepper created this gist
Apr 28, 2014 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,122 @@ Massive file upload using JQueryUpload + Nginx + Rails + Sidekiq The goal: Upload several files (virtually big files) to a Rails application without compromising the user experience. Architecture jQuery File Upload + Nginx + Rails (Carrierwave) + Sidekiq 1) jQuery File Upload (http://blueimp.github.io/jQuery-File-Upload/) 2) Nginx 3) Rails (Carrierwave) Add an asset_upload_controller.rb with a base upload action: class AssetsUploadController < ApplicationController def upload end end Let's now modify `routes.rb` to route each request for `assets_upload/upload` to our controller: post "assets_upload/upload" => 'assets_upload#upload' Once the controller and the route have been created, we can create a factory that will instantiate a new asset processor given the content type. module AssetProcessors class Factory SUPPORTED_CONTENT_TYPES = %w{ audio/mp3 video/mp4 } PROCESSORS = { 'audio/mp3' => AudioAssetProcessor, 'video/mp4' => VideoAssetProcessor } def self.supports?(content_type) SUPPORTED_CONTENT_TYPES.include?(content_type) end def self.create(content_type, file_name, tmp_path) processor = PROCESSORS.fetch(content_type, NullAssetProcessor) processor.new(file_name, tmp_path) end end end Where VideoAssetProcessor is: module AssetProcessors class VideoAssetProcessor < AssetProcessor def process resource = File.open(src_path) video = VideoAsset.create(resource: resource) end end end class VideoUploader < CarrierWave::Uploader::Base storage :file def store_dir upload_dir_root = Rails.env.test? ? "test" : "uploads" "#{upload_dir_root}/#{model.class.to_s.underscore}/video/#{model.id}" end def extension_white_list %w( mp4 ) end end 4) Sidekiq What is Sidekiq? Sidekiq is a background processing framework for Ruby (in alternative you can pick Delayed::Job or Resque). Sidekiq is easily to be integrated into your Rails application and was build with a particular aim at performances. In order to integrate Sidekiq into our Rails application, we have to add it to our Gemfile: gem 'sidekiq' And install the new dependency: bundle install Once Sidekiq is integrated, we can start implementing the worker that will carry out the job in background. The first step to take in this sense consists in creating the directory `workers` in our Rails `app` directory. In such folder we will create the file upload_worker.rb with the actual implementation of the job. class UploadWorker include Sidekiq::Worker def perform(content_type, file_name, tmp_path) processor = AssetProcessors::Factory.create(content_type, file_name, tmp_path) processor.execute end end Where `AssetProcessors::Factory` will instantiate a new asset processor according to file content type calling execute on it; so for example for a file with content type mp4 VideoUploader will be called. Let's now update `assets_upload_controller.rb` so that it will use our Sidekiq worker. class AssetsUploadController < ApplicationController def upload assets_params.each do |params| UploadWorker.perform_async(params[:content_type], params[:file_name], params[:tmp_path]) end end private def assets_params params.require(:assets) end end Run Sidekiq: bundle exec sidekiq