Matt's Blog

A blogging framework for hackers.

Rails Time Zones

Rails Time Zones I’ve recently been working on some freelance projects, one of which is building an online reservation system for a chain of soon to launch hair salons. One of the deeper rabbit holes I ended up in was dealing with time zones. What happens when the hair salons are located in different time zones? What happens when users are in different time zones and make reservations? What happens when a user is in one time zone and makes a reservation for a location in another time zone?

I barely realized I had any issue with time zones until I needed to list all remaining reservations for the day, using

reservation.to_date == Time.now.to_date && reservation.time >= Time.now  

from the method called on a location:

def todays_remaining_resevations  
    res_array = Array.new  
    self.reservations.each do |reservation|  
        res_array << "#{reservation.time.strftime("%I:%M %p")} (#{reservation.user.full_name})" if reservation.time.to_date == Time.zone.now.to_date && reservation.time > Time.zone.now  
    end  
    res_array.count != 0 ? res_array.sort! : "There are no remaining reservations for today"  
end  

This was functioning properly, except that the reservations were being saved in a different time zone than Time.now was spitting out.

Rails does a lot to help, but there are still a number of things that are very confusing until you really dive deep into how it works.

Default Time Zone

First thing you should know is that all DateTime objects are stored in the database as UTC (Coordinated Universal Time). By configuring the default time zone, Rails will use ActiveRecord to convert the data to the time zone you’ve designated. To set the default time zone, in the config/application.rb file, set config.time_zone to the name of the time zone you’d like to be default.

config.time_zone = 'Eastern Time (US & Canada)'  

To get a list of all the time zone names, run:

rake time:zones:all  

in terminal. Now, whenever something is saved, it will automatically be saved in UTC, but will be converted to UTC from your designated default time zone. And since I had set the default time zone, I then changed the above line of code to:

reservation.to_date == Time.zone.now.to_date && reservation.time >= Time.zone.now  

Multiple User Time Zones

So what happens if users make reservations from different time zones? The user model should have an attribute for time zone (string). That way, you can easily call current_user.time_zone when trying to determine what time zone a user is making a reservation from. And with the knowledge of what time zone a location is in, you can easily make the appropriate adjustments!

Rails Code

So the way DateTime objects are created through a rails form, is that for each part of a date (ie month, day, year, hour, minute, etc), its given its own key in params. For example, when a location is created an hours set in this project, params[“location”][“monday_open(1i)”] equals the month value, params[“location”][“monday_open(2i)”] equals the day value, params[“location”][“monday_open(3i)”] equals the year value, and so on. Then, those pairs will instantiate params[“location”][“monday_open”] and passed into Date.new(parameter). Its actually pretty cool checking out the Rails code and see how this is done at the granular level! Here’s a link to the relevant code :)

Other great resources

Working with Time Zones in Ruby on Rails
About UTC
RailsCast #106: Time Zones (revised)

Date Standardizing Algorithm: A CSV Date Parser

So this interview process has had its up and downs, having graduated from the Flatiron School about three weeks ago. And those ups can sway to downs within moments. But the highlights have definitely been solving the problems that are thrown my way, notably parsing differently formatted dates in CSV’s and creating DateTime objects out of them. This was the prompt emailed to me by the interviewer:

The Prompt

We allow customers to upload files (CSV, or otherwise delimited) directly to COMPANY REDACTED through a batch process. During initial setup, we ask our customers to define the structure of these files. That tends to work pretty well, save for one aspect: date formats. The problem is, we can’t rely on Ruby’s date parser (it’s too slow, and more importantly it can often be wrong when looking at individual dates), so we need to know the date format up front.

Customers tend to not fill out the date format (and leave the default), or just select the wrong format altogether. For the purposes of this exercise, let’s assume that no UI solution is possible (we just want to look at the string, and figure out the format).
I’ve attached 8 sample date files with varying date formats. What we want to do is for each file figure out the format of the date field (you can take the column as an input argument — no need to guess which column has dates). We should then be able to supply the value in each row to Ruby’s Date strptime method, along with the format we computed, and get back a Date object with the correct date. Ideally, we’d also figure out the time format if it exists (and thus use with DateTime instead), but we don’t necessarily need to start there.

You’ll notice all kinds of different varieties of dates. Some have no separators between tokens (e.g. November 11 2012 might be 20121111), others have periods separating things. Month / year / day can be in all kinds of different places and can take many shapes, e.g. with or without leading 0s.

In many cases you’ll need to scan multiple records before you can find a definitive answer, but we should be focused on being performant — meaning, it’s ok to parse many records if needed, but we want to do this in real time so we want to read as few records as possible to find the format conclusive, and then stop reading from the file.

We can make obvious assumptions to make things easier (e.g., since e-commerce didn’t exist in 1922, if you encounter a 22 you can assume it’s a day), and we can assume years are either 2 digits or 4 digits (since there is no date format string for a single digit year).

The Code

Before walking you through what I wrote, I figured it would be most logical to show you the final product, and then explain the steps the program takes to parse the dates, standardize the format, and create DateTime objects out of them. Here’s the code (also found at this repo):

require 'csv'
require 'debugger'

class CSVParser

  MONTHS_ARRAY = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]

  def initialize(file_path, column)
    @file = file_path
    @column = column
    @date_column = get_date_column
  end

###############
# MAIN METHODS #
###############
  def parse_date_format
    date_indexes = learn_date_format
    learned_date_indexes = fill_in_last_value(date_indexes)
    datetime_creation(learned_date_indexes)
  end

  def format #RETURNS THE FORMAT OF THE COLUMN'S DATES
    date_indexes = learn_date_format
    learned_date_indexes = fill_in_last_value(date_indexes)

    format = Array.new
    format[learned_date_indexes[:year]] = "YYYY"
    format[learned_date_indexes[:month]] = "MM"
    format[learned_date_indexes[:day]] = "DD"
    date_format = "#{format[0]}/#{format[1]}/#{format[2]}"
  end

private
#########################
# RUN ON INITIALIZATION #
#########################
  def get_date_column
    date_column = []
    CSV.foreach(@file) do |row|
      date_column << row[@column - 1]
    end
    standardize_format(date_column)
  end

  def standardize_format(date_column) #DOESN'T WORK FOR DATE1.CSV
    date_column.each do |date|
      #FORMATS PUNCTUATION CORRECTLY
      date.gsub!(",", "")
      date.gsub!("-", "")
      date.gsub!(".", "/")

      #TRANSLATES SPELLED OUT MONTHS INTO REPRESENTATIVE INTEGERS
      MONTHS_ARRAY.each_with_index do |month, index|
        date.gsub!(/(#{month})[a-z]{0,}/i, " #{index + 1} ")
      end

      #REMOVES TIME INFO
      date.gsub!(/\d{1,}:\d{0,}((AM)|(PM))/i, "")
      date.gsub!(/\d{1,}:\d{0,}/, "")
      date.gsub!(/\d{1,}((AM)|(PM))/i, "")

      #ADDS/REMOVES NECESSARY/UNNECESSARY /'S
      date.gsub!(" ", "/")
      date.gsub!(/\/{1,}$/, "")
      date.gsub!(/^\/{1,}/, "")
      date.gsub!(/\/{1,}/, "/")
    end
    date_column.shift if date_column.first != /\d{1,}(\/)\d{1,}(\/)\d{1,}$/ #REMOVES TOP ROW IN DATE IF ITS A HEADER
    date_column
  end

######################
# SUPPORTING METHODS #
######################
  def learn_date_format
  #ITERATES THROUGH EACH ROW AND LEARNS THE DATE FORMAT
    #IE: MM/DD/YYYY, DD/MM/YYYY, YYYY/MM/DD, ETC
    date_indexes = Hash.new
    @date_column.each do |date|
      split_date = date.split("/")
      split_date.each_with_index do |x, i|
        # study_format_learning(split_date, date_indexes) #UNCOMMENT ONLY TO STUDY ITERATION AND DATE FORMAT LEARNING
        next if i == date_indexes[:day] || i == date_indexes[:month] || i == date_indexes[:year]

        if year_indexed?(date_indexes)
          date_indexes[:day] = i and break if x.to_i > 12
        end
        if month_indexed?(date_indexes)
          date_indexes[:year] = i and break if x.to_i > 31
        end
        if day_indexed?(date_indexes)
          date_indexes[:year] = i and break if x.to_i > 12
          date_indexes[:month] = i and break if x.size == 1
        end
        if nothing_indexed?(date_indexes)
          date_indexes[:year] = i and break if x.size == 4 || x.to_i > 31
          date_indexes[:day] = i and break if x.to_i > Time.now.year%100 && x.to_i <= 31
        end
      end
      break if two_indexed?(date_indexes) #STOPS ITERATION ONCE FORMAT IS LEARNED
    end
    date_indexes
  end

  def nothing_indexed?(date_indexes)
    !year_indexed?(date_indexes) && !month_indexed?(date_indexes) && !day_indexed?(date_indexes)
  end

  def two_indexed?(date_indexes)
    year_and_month_indexed?(date_indexes) || month_and_day_indexed?(date_indexes) || day_and_year_indexed?(date_indexes)
  end

  def year_indexed?(date_indexes)
    date_indexes[:year] ? true : false
  end

  def month_indexed?(date_indexes)
    date_indexes[:month] ? true : false
  end

  def day_indexed?(date_indexes)
    date_indexes[:day] ? true : false
  end

  def year_and_month_indexed?(date_indexes)
    year_indexed?(date_indexes) && month_indexed?(date_indexes)
  end

  def day_and_year_indexed?(date_indexes)
    day_indexed?(date_indexes) && year_indexed?(date_indexes)
  end

  def month_and_day_indexed?(date_indexes)
    month_indexed?(date_indexes) && day_indexed?(date_indexes)
  end

  def fill_in_last_value(date_indexes)
  #FIGURES OUT WHICH VALUE HAS NOT BEEN DETERMINED AND ASSIGNS IT THE REMAINING INDEX
    learned_date_indexes = date_indexes
    indexes = [0, 1, 2]

    #DELETES INDEXES WHICH HAVE ALREADY BEEN ASSIGNED
    indexes.delete(learned_date_indexes[:day]) if learned_date_indexes[:day]
    indexes.delete(learned_date_indexes[:month]) if learned_date_indexes[:month]
    indexes.delete(learned_date_indexes[:year]) if learned_date_indexes[:year]

    #ASSIGNS REMAINING INDEX TO VALUE THAT HASN'T BEEN DETERMINED
    learned_date_indexes[:year] = indexes.first if !learned_date_indexes[:year]
    learned_date_indexes[:month] = indexes.first if !learned_date_indexes[:month]
    learned_date_indexes[:day] = indexes.first if !learned_date_indexes[:day]

    learned_date_indexes
  end

  def datetime_creation(date_indexes)
    @date_column.collect do |date|
      if date.split("/")[date_indexes[:year]].size == 2
        DateTime.strptime("20#{date.split("/")[date_indexes[:year]]}-#{date.split("/")[date_indexes[:month]]}-#{date.split("/")[date_indexes[:day]]}", "%Y-%m-%d")
      else
        DateTime.strptime("#{date.split("/")[date_indexes[:year]]}-#{date.split("/")[date_indexes[:month]]}-#{date.split("/")[date_indexes[:day]]}", "%Y-%m-%d")
      end
    end
  end

#######################
# DEVELOPMENT METHODS #
#######################
  def study_format_learning(split_date, date_indexes)
    puts "#{split_date} day:(#{date_indexes[:day]}) month:(#{date_indexes[:month]}) year:(#{date_indexes[:year]})"
  end
end

csvparser = CSVParser.new("#{Dir.pwd}/csvs/date7.csv", 3)
puts csvparser.parse_date_format
puts csvparser.format

The Walkthrough

For the purpose of this exercise, you’ll be dealing with one CSV at a time, each of which have a column of dates formatted in a certain way. Each CSV will have dates formatted in a different manner, but a column of dates within a given CSV will have one uniform format. When you create an new object of the CSVParser class, you must include the file path and name, as well as the column number which contains the dates. An instantiation of a CSVParser object should look like this:

csvparser = CSVParser.new("#{Dir.pwd}/csvs/date7.csv", 3)  

The initialize method takes in the file/file path and assigns that to the instance variable of @file, and the column number is assigned to @column. Using my get_date_column and standardize_format methods, the @date_column instance variable is assigned an array of dates formatted in the format, “xx/xx/xx”. The get_date_column simply feeds the standardize_format method an array of dates from the column number identified. standardize_format then uses regular expressions to shape the dates, written in nearly any logical format (emphasis on logical), into the “xx/xx/xx” format.

Given the year, month, and day values can be written in many different ways (ie with or without leading zeros, months can be spelled out or using representative numbers, etc) and orders (ie “06 Mar 2013” vs. “Mar 06 2013” vs. “2013 Mar 06”, etc), the tricky part is figuring out which order the format is in (MM/DD/YYYY vs. YYYY/MM/DD, etc). That’s where the learn_date_format, and the gist of the program come in.

For this method, you take one date string at a time from the @date_column, split it on “/”, and iterate through each index. The algorithm, whose logic follows, determines which order this date is written in. My logic for this algorithm was:

1) Nothing has been indexed yet
If the size of the element is 4, or greater than 31, the element has to be a year
If the element is greater than the last two digits of the current year and is also less than or equal to 31, the element has to be a day
2) The year is indexed
If the element is greater than 12, it has to be a day
3) The month is indexed
If the element is greater than 31, it has to be a year
4) The day is indexed
If the element is greater than 12, it has to be a year
If the size of the element is 1, it has to be a month

As soon as one of these conditionals is true, it assigns the index of the current element to a hash, with the key of what value it represents. For example, if nothing is indexed and it comes across an element who’s size is 4, date_indexes[:year] is set equal to the index of that element. Then, as the method continues to iterate through more dates, it will skip that index, knowing that all values in that index represent years.

Additionally, once a second value is indexed, you logically know what the third value is. As soon as this is true, the method breaks out of the iteration. There’s no point in continuing to iterate through the dates once the format is discovered. It then fills in the remaining value with the unassigned index, using the fill_in_last_value method.

The final step is to create DateTime objects out of the @date_column array, using the discovered format. Within the datetime_creation method, it will prepend the year with “20” if a year is written as “13” rather than “2013”.

All methods except for the parse_date_format and format methods are private (the initialize method is private by default). The parse_date_format returns an array of DateTime objects, and the format method returns the format of the standardized date as a string (ie “MM/DD/YYYY”). Lastly, I also included a method, study_format_learning, which is commented out. But once you comment this in (does that make sense?), you’ll be able to see the iteration through dates, watching it break (as in break out of the iteration) as soon as two of the values are indexed. This is just to ensure that unnecessary iterations are not taking place.

Improvements

There is, and always will be, continued improvement to be made. The first step for someone to take to improve on this would be to beef up the standardize_format method’s regular expressions to account for a larger variety of dates. Additionally, the next step would be to account for times as well, as in its current state, this program only creates dates.

Again, here’s the repo to take a look at the code more thoroughly!

Method_missing: Do You Know Where Your Methods Are?

This final week at the Flatiron School we had someone come so we can partake in mock interviews. What an experience. This was the first time I had gone through a technical interview, but I learned a lot more coming out of it than I thought I would! Best lesson was learning about method_missing in Ruby.

NoMethodError

Naturally, when you call an undefined method on an object, we’re all used to receiving this error:

NoMethodError: undefined method “#{method_name}” for "#{receiving_object_name}":“#{receiving_object_class}”  

But what exactly happens between the method being called and this error being raised? If the current class does not contain the instance method being called, it checks if its parent class does. If that class does not, it moves on up to that class’ parent class, all the way up until it reaches the outermost class: BasicObject. BasicObject is the parent class for all classes in Ruby. For more info on BasicObject, check out the Ruby docs here. If that class does not contain the instance method called (which it most likely won’t because, by default, it’s a blank class), it will check the Kernel module. If nothing comes up in the Kernel module, it will call method_missing. By default, this is what raises NoMethodError, but of course Ruby graciously provides you the ability to override method_missing!

method_missing

In a way, method_missing is a safety net. It provides you with a way to gracefully handle calling a method that doesn’t exist!

The prompt for my mock interview can be found here. I was asked to import a YAML file, and make the following string of methods on data return the relevant value in the YAML file:

parse_object.product.first.sku  

This string of methods should be the equivalent of calling:

parse_object[“product”].first[“sku”]  

This is where method missing kicks in! Rather than having to define a method for every possible key in the parse_object hash, you can abstract it all to the method_missing method. By passing in as an argument the key you want to retrieve you can set the method as follows:

def method_missing(key)  
    ShippingHash.new(@data[“#{key}”]  
end  

where ShippingHash is the class and @data is the instance variable (a hash) initialized upon creation of a new ShippingHash object. Below is the final code for the parsing program:

require 'yaml'  

file = File.open("file_path/shipping.yaml")  
data = YAML::load(file)  

class ShippingHash  
    def initialize(hash)  
        @data = hash  
    end  

    def method_missing(key)  
        ShippingHash.new(@data["#{key}"])  
    end  

    def first  
        ShippingHash.new(@data[0])  
    end  
end  

parse_object = ShippingHash.new(data)
puts parse_object.product.first.sku.inspect  

Overwriting and using method_missing, which is considered metaprogramming, may seem like a great way to make your program more abstract, and use less code. But it is naturally slow as it must first check the current class, and all parent classes for the method name called before running method_missing.

Lets Set Up ActionMailer

To begin setting up ActionMailer, first step is to run rails g mailer gg_mailer (I’m using ‘gg_mailer’ as the name for my mailer class in this walk-through) in the terminal, and then bundle install. This will create a two files for you:
    1. setup_mail.rb in config/initializers directory
    2. gg_mailer.rb in app/mailers directory

setup_mail.rb

The first file to begin working with is the setup_mail.rb file where you will connect your email account to your rails app. I’ll be using GMail for this example. The first step to set up this file is to establish the delivery method, SMTP, or simple mail transfer protocol. SMTP is a protocol for sending emails between servers, and a large majority or mail servers use this protocol. Next is to set up the SMTP settings. First are the address, port, and domain for the mail server; for GMail you can use smtp.gmail.com, port 587, and gmail.com, respectively. Then you include your username and password. Next, you must establish what type of authentication is required – plain, login, or cram_md5. You can use plain for GMail. Using login will encode your password using Base64, and I don’t fully understand the mechanism of cram_md5, but according to the ActionMailer documentation, it “combines a Challenge/Response mechanism to exchange information and a cryptographic Message Digest 5 algorithm to hash important information.” Finally, you have to set the enable_starttls_auto value, which accepts a boolean value; for GMail set it to true. This detects if your mail server uses STARTTLS (only available if your server uses SMTP), which encrypts the connection to an SSL or TSL connection rather than just a plain text connection.
This is what it should look like by the time you’re done with this step.

ActionMailer::Base.delivery_method = :smtp
ActionMailer::Base.smtp_settings = {
    :address              => "smtp.gmail.com",
    :port                 => 587,
    :domain               => "gmail.com",
    :user_name            => "gitgallery",
    :password             => "you_wish",
    :authentication       => "plain",
    :enable_starttls_auto => true
} 

Just make sure to include this file in .gitignore! Otherwise you’ll be exposing your password to the world…

gg_mailer.rb

Next file you have to work with is gg_mailer.rb in the app/mailers directory. Here, you can set your default address for emails to be sent to, and define a method for each type of email you’d like to be sent. Below you’ll see a method which sends off a registration confirmation email when called. The reason for passing user in as an argument and assigning it to the local variable @user is to allow it to be in scope when you create the email in the views/gg_mailer directory. In here you can include an attachment with the following syntax:

attachments["rails.png"] = File.read("#{Rails.root}/public/images/rails.png")  

Finally, the last line is what actually sends off the email (make sure it’s the last line of code!):

mail(:to => @user.email, :subject => "Registration Confirmation")  

Here is an example of what the file should look like.

class GgMailer < ActionMailer::Base
    default from: "gitgallery@gmail.com"

    def registration_confirmation(user)
        @user = user
        # attachments["rails.png"] = File.read("#{Rails.root}/public/images/rails.png") #USE THIS IF YOU WANT TO INCLUDE ATTACHMENT
        mail(:to => @user.email, :subject => "Registration Confirmation")
    end

    def new_project(project, contributor)
        @project = project
        @contributor = contributor
        mail(:to => @contributor["email"], :subject => "A repo you contributed to was just made into a project on Git Gallery.")
    end
end

The final step to get ActionMailer working is simply to call the method you just wrote wherever you’d like in your program! For the example I gave, I’m calling it in the user#update controller/action for various reasons, but essentially this method will only trigger when a user signs in for the first time and confirms their account info!

development_mail_interceptor

One additional step you can take is to include the development_mail_interceptor, using the gem “mail”. The mail interceptor allows you to ensure that all outgoing emails from the designated account direct to an assigned email address, usually the same one that the messages are being sent from. This is so that whenever you’re in development, testing your site/seeding your database/etc, you are not constantly sending emails to addresses that may be included in your testing environment.

So, for the mail interceptor, you want to create the file development_mail_interceptor.rb in the lib director. In it, you want to have a class of DevelopmentMailInterceptor, and the class method of delivering_email(message). You can set the email subject and to address by declaring message.subject and message.to, respectively. For the subject, it is recommended to include in there the email address in which the message is supposed to be sent to, and the message.to address should be sent to where you are sending from.

The final step for the mail interceptor is to require it in the setup_mail.rb file, and to call

ActionMailer::Base.register_interceptor(DevelopmentMailInterceptor) if Rails.env.development?  

This is what the file should look like by the time you’re done including the interceptor:

require 'development_mail_interceptor'

ActionMailer::Base.delivery_method = :smtp
ActionMailer::Base.smtp_settings = {
    :address              => "smtp.gmail.com",
    :port                 => 587,
    :domain               => "gmail.com",
    :user_name            => "gitgallery",
    :password             => "you_wish",
    :authentication       => "plain",
    :enable_starttls_auto => true
}

ActionMailer::Base.register_interceptor(DevelopmentMailInterceptor) if Rails.env.development?

View Files

Last but not least is to actually create the view file for the email itself. These files should be saved in the views/gg_mailer directory, with the same name as the method that is being called. For example, the email file for a registration confirmation email (the method triggered for this is registration_confirmation) will be registration_confirmation.html.erb. You should also create a text file for those users who prefer text emails over html emails.
Best of luck setting up ActionMailer in your own Rails app! Resources I found incredibly helpful were the railscast as well as the documentation.

Until next time!

PhoneGap: Intro (Part 1)

This is the first of two posts relating to PhoneGap. This post will be my introduction to PhoneGap, and the second post will be a walkthrough of building my first PhoneGap-based app.

What is PhoneGap?

PhoneGap is an open source framework used to allow you to easily build cross-platform mobile apps using HTML, CSS, and JavaScript. More so, it allows you to natively interact with the phone, meaning it gives you access to a phone’s features including the accelerometer, GPS, compass, notification center, etc. Without PhoneGap, a developer would most likely use Java and Objective-C to develop natively for Android and iOS, respectively.

History

To start off, its fascinating to go back into PhoneGap’s archives and read their blogs from back in the day when they first announced and released information on their framework! I definitely recommend taking a look here.

PhoneGap was first announced in 2008 for the purpose of allowing native mobile app features available to mobile web apps, taking the logic of Adobe AIR, which allows web developers to build Windows and OSX applications.

Set Up

PhoneGap is built on open source software called Apache Cordova. To build something with PhoneGap, you’ll be using the cordova command-line interface.

First step is to install Node.js

Then, you must install cordova using: sudo npm install -g cordova in the terminal. “npm” stands for node package manager, which, if you could guess, is the packet manager for the Node JavaScript platform. This is installed when you download Node.js.

To create a new project directory, you need to run cordova create DirectoryName. This will create three subdirectories, including: merges, plugins, and www.

platforms, the additional directory pictured above is for the target platform(s) you’d like to develop for – ie, iOS, Android, etc.

All subdirectories are empty except for the www directory, of which you can see the content in the image above. The www directory is where the application’s home page lives. And for each platform you create (I’ll get into this in Part 2 of the post, but the command to add a platform (iOS for exampke) is cordova platform add ios), another www directory is created within the platform/ios/ directory path.

For my follow up post, I’ll install the iOS SDK and create my first PhoneGap app! Stay tuned.

Rails Scales

Although I just began learning about the magic and wonders of Ruby on Rails, I’ve heard for some time that Rails has issues with scalability once you reach a certain point. But in reality, that’s all I’ve really ever heard – I’ve never been offered any details as to how or why this is the case.

Popular Sites on Rails

Before getting into the details, there are a number of very popular sites that are built using Ruby on Rails, including: GitHub, Groupon, Hulu, YellowPages, Basecamp, Shopify, and Airbnb.

Twitter

Twitter was launched using Rails back in 2006. Beginning in 2008, rumors of Twitter dropping Rails surfaced in the news, but this didn’t actually occur until 2011 – switching to Lucene, a “full-featured text search engine library written entirely in Java.” This transition took place the week after the tsunami hit in Japan, which led to nearly a doubling in the usage of Twitter. The switch cut latency to a third. BUT, that doesn’t mean that the scalability issues were related to Rails!

The Problem with Rails

So most of the issues that arise with Rails don’t directly have to do with the Rails framework itself, rather the interaction with the database. For example, Rails encourages programmers to develop in a local environment in which you generally work with smaller data sizes, but once deployed, you may find that working with larger data sizes can introduce significant performance problems. Additionally, because Rails separates you a good amount from SQL, many Rails developers get into this “SQL is bad” mindset, instead relying on ActiveRecord. Of course, ActiveRecord makes so many things so easy, but it does have its own performance issues.

Turns out that ActiveRecord is the main culprit in scalability issues within Rails, specifically relating to the N+1 query issues! Just for review, here’s an example of what the N+1 query issue is:

Lets say you have 100 cars, each with four tires. In order to query each tire position (lets say depending on where the tire is, it has a position number of 1-4) with ActiveRecord, you’d ultimately be querying all 100 cars, and then for each car, you query the tires. That ultimately comes out to 101 queries – incredibly inefficient! Ultimately, it is faster to issue one query with 100 results rather than issue 100 queries with one result.

Avoiding N+1

Luckily, this isn’t something you have to be stuck with. ActiveRecord uses something called lazy loading, in which it only performs a certain task when necessary. On the other side of the spectrum is eager, and over-eager loading. For example (based on a post here):

Imagine a page with rollover images like for menu items or navigation. There are three ways the image loading could work on this page:
    1. Eager – load every single image required before you render the page
    2. Lazy – load only the displayed images on page load and load the others if/when they are required
    3. Over-eager – load only the displayed images on page load. After the page has loaded preload the other images in the background in case you need them

Using eager loading helps minimize the number of queries. Ideally, the above example with cars and tires would only come out to two queries, no matter how many cars there were. Here is how one would use eager loading:

N+1 query

<% @cars = Car.all(@cars).each do |car| %>
    <p><%= car.tire.position %> </p>
<% end %>  

Eager loading

<% @cars = Car.find(:all, :include=>[:tire] %>
<% @cars.each do |car|%>
    <p><%= car.tire.position %></p>
<% end %>  

As mentioned before, this will generate at most two queries, no matter how many rows you have in the posts table.

Detecting Performance Problems

One of the most powerful plugins is the “query_reviewer” which automatically analyzes SQL code that ActiveRecord generates for potential problems. It rates a page’s SQL usage into one of three categories – Ok, Warning, and Critical – attaches warnings to any queries in which one is needed, and displays an interactive summary to help you determine which queries are not absolutely necessary. Here’s a link to the repo for the gem.

Conclusion

All in all, the scalability issues of Ruby on Rails can be overcome. The framework allows you to spend less time building and more time planning the architecture and how to scale. Don’t let is scare you off from the elegance of the framework.

Hello World

Hello World

My name is Matt and this is my first blog post. I find understanding why people are where they are today is incredibly valuable, so before I begin with the more technical posts, I’d just like to introduce myself…

Background

My First Computers

Born in NYC and raised in New Jersey, my first interaction with computers began with the family’s Windows 95-powered Gateway 2000. I absolutely loved playing games on that thing – Skyroads (still play with that one when I’m feeling nostalgic), Oregon Trail, Roller Coaster Tycoon, and Myst (I never really figured out what to do in that). It took a few years of just playing on PC’s to really appreciate what they can do. It wasn’t until we got two Dell towers for the house (one for the parents and one for the kids) that I really became obsessed.

Within a few months of getting these, newer and faster models had come out. But unlike this new Mac I got a few months ago, all the parts were easily replaceable. Of course my parents wouldn’t let me touch their computer, so against my siblings’ will, I opened ours up and began tinkering around. I ordered a few extra RAM sticks, a new hard drive, and “borrowed” a friend’s copy of Windows 2000. And luckily, my closest friends were all into computers and technology (and of course video games) the same way I was.

Separate from computers, my dad instilled in me this curiosity and desire to build and fix things. I needed (and still do) to understand how things actually work. From building outdoor furniture (and refusing to read the instructions), to watching him and my grandfather work on their cars, to watching documentaries on TV on how things were made, I just couldn’t imagine myself using something without understanding how it actually worked. And if something broke, I would sit there for HOURS and get pissed off to no end until I figured out how to fix it. More than anything else, that mindset applies to technology.

One of my favorites quotes about programming (can’t remember where I read it) is “developers are nothing if not compulsive problem solvers.” By the time high school came around I knew exactly what I wanted to focus on.

High School

I was lucky to attend a magnet high school which offered a focus in computer science. The school was divided into numerous vocational academies, and I enrolled in the Academy for Telecommunication and Computer Science. Over the four years, I received a significant amount of exposure to Java, HTML, Cisco networking fundamentals (through their CCNA program), and a small amount of Oracle database design.

Unfortunately, as I’ve found with most forms of formal education, computers didn’t appear as exciting to me as they used to. So as I finished high school my academic focus was directed towards business instead. I still did always stay fascinated with technology, its evolution, and its revolutionary capabilities, but learning how it worked was put on the backburner (until now).

College

For college, I escaped the northeast and attended Washington University in St. Louis where I studied Economics and Entrepreneurship. I found the former absolutely useless, but the allure of entrepreneurship had its’ hold on me.

At the end of freshman year, two of my roommates and I purchased Bears’ Bikes, the campus bicycle rental business – what an experience! For many things I learned in class, I was able to apply to the business. But interestingly enough, most things I learned in the business I was able to apply to class. From managing customers and our employees, to accounting, to marketing, what I appreciated most was being able to see direct results in the business based on our actions. And as we grew the company, so did my enthusiasm for the company.

While working on various parts of the business, I would picture it in my mind as some object that I’m taking apart, improving, and putting back together. If you were wondering, we successfully sold Bears’ Bikes towards the end of our senior year. As much as I loved the experience, maintaining a fleet of 150+ bikes definitely turned me off from cycling for a little while.

Post Graduation

By senior year, I had developed another business idea and discussed working on it with a close high school friend (he went on to Stanford to continue studying computer science). I don’t plan to say much about the concept itself (because I haven’t completely shut the door on revisiting some modification of the idea), but we decided to start the business that winter. Long story short, after about a year and a half, we weren’t able to keep up with our own deadlines we set, and other companies out in the Valley received millions in funding for VERY similar concepts. We realized there was no way to keep up with them given the stage in development we were at.

Of course we were both incredibly disappointed, but we were also encouraged by the fact that companies with similar plans had received large amounts of funding. It told us that we were heading down the right path, and successfully identified a need and opportunity.

Revisiting My Roots

Lesson learned for me though is the importance of being more self reliant. I’m now revisiting my roots and (re)learn programming. I know that I will start another business of my own one day, and the way this world is going, computers will no doubt play an integral role. So why rely on someone else to manage what I can learn myself? That’s why I am a student at Flatiron School today. And less than three weeks in, I can already tell this is one of the best decisions of my life.

My friend had explained to me that he wasn’t a fan of one of his summer internships because he already knew everything he needed to do well. At the time I didn’t really understand why he said that. I couldn’t imagine the downside to actually being fully qualified for your job! But it wasn’t until studying here did I really understand what he was talking about.

More than ever, I now see how programming fits my need to understand how things work. Working on econ problem sets in college, I would have no problem passing over a problem after a while if there was no hope in me figuring it out. But with programming, I’ve spent countless hours staying up at night trying to work out issues in a program, not willing to give up. I feel like a kid again because this reminds me of my younger self, and I couldn’t have asked for more.

Until next time!


Arduino the Ruby Way

Arduino the Ruby Way

Speakerdeck Slideshow

Ruby in Our World

Over the past two weeks, I’ve heard a lot about the origin of the Ruby language and Matz’s reasoning. (Paraphrasing) “I didn’t want to create a computer language that humans can understand, I wanted to create a human language that computers can understand.” And since beginning work with Ruby, my minimal exposure has opened my eyes to the world of possibilities with this language. But not just the TRON-like world of software. I’m talking about our world as well.

There’s this little thing out there called an Arduino. What is that you ask? I got my buddies at WIRED to right up a little post about Arduinos (Arduinoes? Arduini?) for you. But long story short, its a microcontroller that can be connected with a number of sensors) – including ultrasound, humidity, microphone, etc – that opens the gates and connects our world with the world of TRON.

So, I’ve got a pop quiz for you. What’s better than the capabilities of an Arduino? Hint: It starts with an ‘r’ and rhymes with “doobie.” Its an Arduino powered by Ruby, the “computer language” of our world.

Arduino Capabilities

What can can’t you do with an Arduino? Here are a few Arduino projects I’ve found incredible creative/useful:

  1. Using a sensor to detect the moisture of the soil for your plant, a tweet will be sent out as soon as your plant needs to be watered.
  2. Make a motion detector for various purposes (including automatically turning on and off lights)
  3. If you don’t have a thermostat, you can set it to turn on your A/C or heat depending on the indoor and outdoor temperature (indoor using a temperature sensor, and outdoor pulling the info from the internet)
  4. Control an RC car using your phone (my personal favorite).
Using Ruby with an Arduino

To begin using Ruby to program an Arduino, there will be a few gems to download that will help you out. There’s a link here which explains a number of gems.

  • To start off, “Dino” will help you get up and running very quickly. And of course, its an open source gem so its constantly being updated and improved. “Dino” even creates the class Board to make working with your Arduino even easier!
  • “SerialPort” will allow you to read from the Arduino’s serial port. The serial port is the port that allows the Arduino to interact with other devices.
  • My personal favorite is the “Arduino” gem which allows you to prototype programs for your Arduino without having to burn them to the device. That way, you can see on your computer how the program would work without having to deploy it!
  • This isn’t a gem, but a class which I find fascinating. The TxRx class allows your Arduino to connect to another Arduino and its serial port (and therefore all the info coming from any attached sensors). I can’t even begin to think the type of network you can build with a few of these things and what you can do with them…
Using Ruby with an Arduino

So I’ve just recently become familiar with the capabilities of an Arduino after another student in my class, Kristen Curtis, posted about it on her blog. As I begin to learn more about Ruby, I absolutely plan to implement my new found skills on an Arduino and definitely will keep you posted here!