About Me

My Photo
Kevin Compton
Entreprenuer and professional software developer. Husband, Father and Friend.
View my complete profile

Tuesday, April 21, 2009

Idea behind #rollup method clarified

My co-worker/manager left the following comment to my previous post:

Using Enumerable methods (esp. Rail's group_by / index_by) is more idiomatic, I think:

people.group_by(&:name).each do |name, people|
puts "Details: User #{name} has residencies at: #{people.map(&:address).join('; ')"
end

eee.c

My initial code example didn't express this clearly enough but our original use case was to process thousands of studies worth of data and import them into our database. In this case we did not wish to load all of that data into memory but instead deal with each line at a time and "build up" our Study object as we went. Finally, upon starting a new study or reaching the end-of-file, we would want to save our Study to the DB.

Here is another code example that better illustrates the issue at hand:




 1 class Study < ActiveRecord::Base
 2    # assume fields like: external_id, title
 3    # also a collection of study_team members for the following roles:
 4    #      'Billing Coordinator' & 'Admin Assistant'
 5 end
 6
 7 class StudyProcessor
 8   # process csv files in the following format:
 9   #    external_id, title, billing_coord, admin_asst.
10   #    1, "Oncology Study 1", "Emerson, Ralph", "Sanders, Colonel"
11   #    1, "Oncology Study 1",, "Roberts, Julia"
12   #    1, "Oncology Study 1", "Manners, Miss",
13   #    2, "Test Drug Study 2", "Pusher, Imyour", "Spears, Courtenay"
14   #    .... On for 20000 rows ...
15   #    
16   def initialize(file)
17     @file = file # CSV file with thousands of rows
18   end
19
20   def process
21     csv_reader = CSV.open('csvfile.csv', 'r')
22     # add #rollup method to our reader
23     class << csv_reader; include Patterns; end;
24     
25     # let us assume that we can either pass a proc or a symbol representing a locally scoped method to rollup
26     # Also it is a prereq that the study data rows already be grouped together!
27     csv_reader.rollup( :study_data_csv_key, :initialize_new_study, :merge_study_team_into_study, :save_study )
28   ensure
29     csv_reader.close rescue nil
30   end
31
32
33
34   def initialize_new_study(data_line)
35     @study = Study.new(data_line[0], data_line[1])  
36   end
37
38   def merge_study_team_data_into_study(data_line)
39     unless @study.billing_coordinators.include?(data_line[2])
40       @study.billing_coordinators << data_line[2])
41     end
42     unless @study.admin_assistants.include?(data_line[3])
43       @study.admin_assistants << data_line[3])
44     end
45   end
46
47   def study_data_csv_key(data_line)
48     data_line[0]  # let us use external id
49   end
50
51   def save_study
52     @study.save!
53   end
54
55 end

0 comments: