Abstract

This article solves 2 problems in Rails application development:

1. Uploading multiple files into Rails app
2. Uploading large files into Rails app

Backgrounds

What's the problem of uploading multiple files in Rails?

The tricky thing is how rails convert the names of form fields into the hash variable "params". Without properly naming, it will be very difficult to process uncertain number of files.

What makes rails cannot handle uploading of large files?

In the default rails implementation, it will read the whole file uploaded from the browser into memory, then write to disk. Which is very time consuming and inefficient. I'm not going into this as there is a very comprehensive blog on JEDI.

Solutions

To upload multiple files into rails, Brain has a nice post here. I just followed his steps with the following changes:

1. Change the element name to "attachment[]" in the MultiSelection js function:

   this.addElement = function( element ) {
     if( element.tagName == 'INPUT' && element.type == 'file' ) {
       element.parentNode.id = 'attachments_container';
   //  element.name = 'attachment[file_' + (this.id++) + ']';
       element.name = 'attachment[]';
   ......

2. Now we can process the attachments as array

   def process_file_uploads
     return if params[:attachment].nil?
     
     params[:attachment].each do |file|
       @attachment = Attachment.new( { "uploaded_data" => file } )
       @picture.attachments << @attachment
     end
   end
   

By reading JEDI's blog, I decide to go to nginx + upload module. Download source code of nginx and upload module, configure nginx with command:

./configure --add-module=../upload_module_dir

then make and install it.

nginx.conf as following

worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    # back-end server
    upstream mongrel {
      server 127.0.0.1:3000;
    }
    server {
        listen       80;
        server_name  localhost;
        client_max_body_size 0; # don't limit the upload file size
        location /documents/upload { # use upload module on this URL
          upload_pass /;
          upload_store /tmp; # save uploaded files here
          upload_set_form_field "attachment[]name" "$upload_file_name";
          upload_set_form_field "attachment[]content_type" "$upload_content_type";
          upload_set_form_field "attachment[]path" "$upload_tmp_path";
          upload_store_access user:rw group:rw all:rw;

          # pass authenticity_token and all html form fields start with document to http://mongrel/documents/upload
          upload_pass_form_field "^authenticity_token$|^document.*"; 
        }
        
        location / {
          proxy_pass http://mongrel;
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header Host $http_host;
          proxy_redirect off;
        }
    }
}

In the erb template:

<% form_for @document, :url=>{ :action => "upload" }, :html => { :multipart => true } do |f| %>
  <%= f.label :title %>
  <%= f.text_field :title %>
  <% fields_for(:attachment) do |a| %>
    <%= a.label :files %>
    <%= a.file_field :files %>
  <% end %>
  <script type="text/javascript">
    var multi_selector = new MultiSelector($('pending_files'));
    multi_selector.addElement($('attachment_files'));
  </script>
  <%= f.submit 'Create' %>
<% end %>

As we are using form_for helper, the generated field name will have 'document' prefix. So the upload_pass_form_field config in nginx.conf will pass all fields start with document to back-end. And the uploading multipart data will be processed by the nginx upload module, then create fields of attachment[]name, attachment[]content_type, attachment[]path for each file. The rails will process the request using DocumentController's upload method. The params is

{
  "attachment"=>[
    {"name"=>"Firefox_by_IQEye.jpg", "content_type"=>"image/jpeg", "path"=>"/tmp/0000370374"}, 
    {"name"=>"f616d49511f4a9d1.jpg", "content_type"=>"image/jpeg", "path"=>"/tmp/0000493833"}
  ], 
  "authenticity_token"=>"OQayTGSPwCjStPgYrYGnob4G8S1Z53qD9olwyL8TE0k=", 
  "document"=>{
    "author"=>"b", 
    "title"=>"a"
  }
}