Archive for the ‘Ruby’ Category

Zip Code Distance Searching in Ruby on Rails

Tuesday, June 23rd, 2009

In my last entry we looked at using scopes to dynamically build scopes for searching in your Ruby on Rails application.  In Tryst, we needed to let our uses search for other singles that were xxx miles from their location.  We require that new users enter their zip code, so that is what we had to go off of.

When I was researching the best way to do this, I did a quick Twitter poll – most of my friend thought I should just use Google’s GeoCoding API, which will let you preform these types of searches.  I really didn’t want to go this route for a couple reasons.

  1. It’s an external dependency.  Yes, Google’s uptime is impeccable, but I don’t want to rely on them
  2. Google’s TOS requires that the site/app/service be free.  Tryst may end up with paid features down the road and I didn’t want to have to rewrite a substantial amount of code 1 year from now.
  3. It’s a challenge.

So, we decided to do our own thing.

Our workflow for a search would work like this – we get a distance and a zip code as search parameters, feed both of these into something that returns all the zip codes within the specified distance, and then find all the uses in those zip codes.  Easy enough.

PS. I’m using MySQL here.

First, we need a reference for zip codes.  I found this great free CSV that has all zip codes, major cities, longitude, latitude, area codes, and even metropolitan codes. Import it into your database and remove any rows that were not in the US.  I actually had a ‘Location’ model in my Rails application that I imported this into.

After some digging around the webs, I found some crazy algorithms for generating distance between different locations based on their longitude and latitude.  I’m not going to pretend to understand everything that is happening here, but after some playing around I ended up with this:

SELECT o.zip_code
FROM locations z, locations o, locations a

WHERE z.zip_code = #{zip_code}
AND	z.zip_code = a.zip_code
AND	(3956 * (2 * ASIN(SQRT(
		POWER(SIN(((z.latitude-o.latitude)*0.017453293)/2),2) +
		COS(z.latitude*0.017453293) *
		COS(o.latitude*0.017453293) *
		POWER(SIN(((z.longitude-o.longitude)*0.017453293)/2),2)
	)))) <= #{distance}

I’ll let you guess which part does the calculation. :)
zip_code and distance you’ll need to pipe into the SQL, but you’ll get back all the zip codes with in your specification. We just thew the returned array of zip odes into a scope like this:

# models/users.rb
def search(params)
    scope = User.scoped({})
    ...
    scope = scope.conditions "users.zip_code in (?)", zip_codes unless zip_codes.blank?
    ...
end

Now, this query is fairly slow. By slow, I mean about 25-30ms correctly indexed on my development box. For us it’s just fine.

I hope that helped someone out – I had a hell of a time finding a solution I liked and I really think this is the best one.

Smart Searching Using Anonymous Scopes

Sunday, June 21st, 2009

I really like how Lovetastic’s search works.  Instead of a plethora of select boxes, users can simply use expressions to create searches.  We did something similar with Tryst’s searching.  One text field and you enter stuff like age ranges (30-45), things like smoke-yes or smoke-no, cities, zipcodes, etc.

Tryst.com Search

To make this whole thing more Rails like, I did not want to just use Regex to parse out search terms and plug them into a complicated SQL query.  That would be soooooo PHP.

Named Scopes are awesome.  You can use them to built decently formatted and fairly complicated SQL queries in a nice, no bullshit, manner. You can do cool stuff like this:

@users = User.active.males.in_zipcode(params[:zip_code]).nonsmokers.top_ten

Just keep stacking scopes on.  Rails generates the query and it usually doesn’t suck.

Go read the API docs for the basics on Named Scopes. We’re gonna do some not basic stuff. Check out Ryan Bate’s Anonymous Scopes Railscast, which introduced me to this method of generating Scopes.

We wanted to add a Class Method called search and have it define the Scopes. We pass in the params hash, anything else we need (like the current user), and let it handle the Scoping and determining what we’re looking for from the search params.

To start, here is our search class method.

# app/models/user.rb
 def self.search(params, current_user)
    page = params[:page] || 1
    search_terms = params[:search]
end

Yeah, using Will Paginate and assigning my search terms to a local variable. I’ve also written another Class Method that does the parsing of the params.

# app/models/user.r.b
def self.search(params, current_user)
    page = params[:page] || 1
    search_terms = params[:search]
   #parse params
    max_age, min_age, zip_codes, gender, smoke, drink = self.get_parameters(search_terms.to_s, current_user)
  end

Now, we’re going to define some dynamic scopes. This is awesome. For it to work, we’ll add an initializer.

#config/initializers/scopes.rb

class ActiveRecord::Base
  named_scope :conditions, lambda { |*args| {:conditions => args} }
end

Now, lets make the scopes.

def self.search(params, current_user)
    page = params[:page] || 1
    search_terms = params[:search]

    max_age, min_age, zip_codes, gender, smoke, drink = self.get_parameters(search_terms.to_s, current_user)

    scope = User.scoped({})
    scope = scope.conditions "users.age >= ? and users.age <= ?", min_age, max_age unless min_age.blank?
    scope = scope.conditions "users.zip_code in (?)", zip_codes unless zip_codes.blank?
    scope = scope.conditions "users.smoker = ?", smoke unless smoke.blank?
    scope = scope.conditions "users.drink = ?", drink unless drink.blank?
    scope = scope.conditions "users.gender = ?", gender unless gender.blank?
    scope.paginate :page => page, :order => "created_at DESC"
  end

Pretty slick, huh? So now, in my controller I just need

# app/controllers/search.rb
def users
    @users = User.search(params, current_user)
end

Assuming your class method for parsing out scope parameters doesn’t suck, you should have very clean, cuddly, and concise searching. No more worrying about breaking some huge crappy SQL query.

You’ll notice I used

users.zip_codes in (?)

This is because I can pass an array of zipcodes and return all the users in those zipcodes. This is because I’ve implemented radial distance searching base on zipcodes. Users can search like ” 60606 25miles” and return all users who are within 25 miles of 60606. We’ll go into how this works in the next post.

Yes, I started a dating site. I’m sorry.

Sunday, June 14th, 2009

Ok, so five years ago I never would have though that I’d have started a ‘Web 2.0′ dating site.  Hell, I didn’t think I’d be doing a lot of things, but here I am.   There is actually a small story how I got involved with (shamless self promotion) trystme.com.  Wanna hear it?  Good.

Originally, I wanted to become an elementary school teacher and was halfway though getting my elementary eduction BA… until I started to volunteer at a school library.  I hated it.  Plus I realised I had too many tattoos to be a school teacher… never should have been in all those punk bands growing up.

So there went that idea – I quit school and took a year off.

I decided to go for something that is completely different and the opposite of little crazy kids – computer science.  I had always like computers, had some experience with Dreamweaver and the idea of not having my work talk back to me all the time sounded great.  I went back to school and loved CS.

About 3 years ago I graduated and I got a contract job with a small local development shop in Duluth, MN.  After a few months it went under and the guy screwed me out of a pay check or two, but I did walk away with something really cool – I was exposed to Ruby on Rails.  I freakin loved it.  LOVED IT.  I quit wasting my time with PHP and drank the Ruby juice.  I even wrote an article for a Rails magazine.

Eventually I got a job working programming for this educational coop in Norther Minnesota.  It was pretty slick.  I enjoyed the work, had tons of freedom, and actually got paid (unlike the contract gig).  Worked there for a little over a year… until… I got an iPhone January 2nd of 2009.

I had a couple week lapse in my old phone contact, so my old phone was still on but it would get left at home.  One day it rang and my wife answered it.  It was some lady from Chicago who said she worked for some company that was trying to find Rails developers and somewhere they found my resume or something crazy like that.  She even said that she had called before and left messages and that I never called back.  This is true – I rarley call back if I don’t know the caller.  Thanks telemarketers.

So, Jen tells me I should call this lady, Andrea The HR Lady From Chicago.  Jen and I had been talking about moving from Norther MN for sometime now and almost took a job in Milwaukee (thank God we didn’t).  The more I talked to Andrea The HR Lady From Chicago the more it sounded good.

So I got this job, moved to the Chicago Burbs, and met a guy at work named Mike.  Mike had lots of crazy ideas and had this one to start a dating site.  The original idea was to only spend two weeks developing it and whatever we had at the end of two weeks was it.  This sounded great at first, but since two weeks really meant about 40 hours between the two of us (since we’re doing this at night after our real jobs) it got stretched to about a month and a half.

There you have it.  We didn’t exactly build it to make tons of money. That’d be nice – we have ads on it, but really we just wanna see what happens.  I guess we want what every programmer with an entrepenurial spirit wants – to wake up one morning and find that your one lowely server is dying under the load and we have to split into an app server and a database server.  Then add Memcached, another app server, shard the database, another app server, hire a real DBA, and on and on.

Check it out – we launched it today, 14/6/2009 at 7:30pm, and are super proud of it.  If you’re looking for a date sign up and give it a try.  Let us know what you think and have fun.

Deployment Nightmare: Godaddy

Tuesday, November 4th, 2008

Against the advice of, well, the entire Rails community, I attempted to deploy a project to Godaddy.

Godaddy proudly displays the little Ruby logo next to Php when browsing through their hosting plans, and seeing as you can get Rails hosting for $7/month through them, it seems like a decent deal, even if you know it’s gonna get slow.

Baisically, what ends up happening when deploying to Godaddy, is you upload your application to a special directory you create through their control pannel, modify your dispatch.fcgi file, .htaccess files, turn off Java and pray.  Its all supposed to run as FCGI in the end, which is not the speediest way to deploy a Rails app, but is acceptable for smaller, non-demanding applications.

The application I was attempting to deploy was a Rails 2.1 app, and I found that Godaddy is rocking Rails 1.1.6 (Release Aug 2006), so I froze Rails (standard practice), change the configs to point to the manually created database (no rake support here folks) and tried to hit it.  Nothing.  First, every time you modify the .htaccess file (and you will need to – many times) it takes 10-30 minutes for Apache to notice.  So, you’re stuck making a change, waiting, testing, washing and repeating.  If you’re using FCGI, there is literally NO output into the ‘errorlog’ Godaddy gives you, so you have to deploy with regular CGI to get any idea why your application isn’t working and then switch back to FCGI and pray.

Finally, after a couple hours of searching the interwebs and poking and prodding, I was getting somewhere – the error log started to get populated, but FastCGI wasn’t starting correctly.  After some snooping, I saw that Rails 2.1 assumes that you’re not running whatever version of Gem was around in 2006.  Gem was failing with a method not found error when Rails tries to talk to Godaddy’s almost 3 year old version.

I putzed around a bit looking for ways to skip that step of Rails’ boot process, swaping out boot.rb files from older version of Rails, hacking, etc.. and to no avail.

This is where I stopped.  I figured that even if I got past this problem, the version of Rmagick was from mid 2006 and would likely not work with my app, and I wasn’t about to try and get Godaddy to update it.  If they haven’t updated any of the Rails stuff yet, I don’t think they are going to any time soon.

My message for Godaddy is this:  I want the entire day I spend on this back.  You should stop advertising that you support Ruby/Rails when you obviously don’t.  I’d bet 90% of Rails apps in development today would not be able to run on their hosting plans. Eh.

rQuote – Ruby on Rails Stock Quote Plugin

Friday, August 29th, 2008

I merged some stock quoting stuff I had into a Rails plugin today. If you’d like to be able to simply grab real time stock quotes in your Rails app, this will do the job. Pretty much any stock symbol will work and you can enter as many as you’d like, you’ll get a hash array of hashes containing each symbol’s current value, change since open, and volume.

http://github.com/johnyerhot/rquote/tree/master

Rquote
======

Gets realtime stock quotes from Yahoo Finance. 

Its super simple to use.

Example
=======

quote = Rquote.new
quote.find("aapl", "msft") 

=> [{:change=>"-4.02", :price=>"169.72", :volume=>"16105013", :symbol=>"aapl"}, {:change=>"-0.42", :price=>"27.52",
:volume=>"27024456", :symbol=>"msft"}]

Copyright (c) 2008 John Yerhot, released under the MIT license

Testing Your Apps.

Saturday, August 23rd, 2008

I’ve been really putting off using Rspec. Unfortunately ( and like the vast majority or programmers out there) I have not been writing enough tests for my apps. When I did write them, they were using Rail’s built in Test::Unit library, which, many will say… sucks. Plus, honestly, they were half assed.

So, I’ve taken it upon myself to dive into Rspec. Most everyone agrees it is the way to go, and it gets you into BDD, which gives me another bussword I can throw on the resume (/sarcasm).

The part about writing tons and tons of tests to cover all your app that didn’t make sense was that you would end up writing so much more code. In reality, I think it boiled down to being lazy, which all programmers are. Since the Rails community prides itself on keeping things DRY and keeping the amount of code written to a minimum, why the hell would I want to write tons of tests for stuff I don’t think are going to break?

In the long run, for any programmer, it is really not about how much code you actually have to write, its more about how much time you send making the app work. If you’re like me, there have been many nights spent debugging some controller that totally broke and you’re stuck refreshing your browser, creating a user, deleting a user, and checking the log over and over until you solve it. I hate it and have probably wasted days doing this. If you had written tests for your app, starting with that first controller, and continued writing quality tests, used autotest, you would avoided this. You would be confident that you code was rock solid.

What about if you’re absolutely sure you’re code won’t break? Well, it will. Maybe not all of it, but you’re gonna have problems eventually. I don’t think that writing tests for trivia stuff (like validating that validates_presence_of is working) is really worth while, but other stuff is fair game.

The list point I’d like to make is that I think since really writing tests and using Rspec, my thinking has changed. I know think writing code, not just as how do I make this work, but also, how to I ensure this WILL work. You’ll find yourself trying look for ways that your code will break, looking for more flaws, and consequently, leaning from those flaws. I’ve found myself refactoring much more code and that end up making my code more flexible and cleaner.

I don’t want to pretend to be a captain awesome when it comes to writing tests yet, but I really think I’ve been coming along very well and am so glad I’ve been investing so much time in it. Give it a try ya’ll.

rDigg – Ruby on Rails Digg API plugin

Thursday, August 14th, 2008

I’ve sort of finished up rDigg today. It’s to the point I’m comfortable letting other people start to play with it anyways.

As you’d expect, rDigg is a Digg API wrapper in the form of a Ruby on Rails plugin. It still needs some work, but works pretty well.

For example:

#create new Rdigg object
digg = Rdigg.new

# find the 3 newest submissions from Kevin Rose
stories = digg.user.find_submissions("kevinrose", :count => 3)

# stories is now an array with a hash for each story
stories.first[:story] #=> the story's text
stories.first[:href] #=> the story's url
stories.first[:diggs] #=> number of diggs the story has

Grab it at: http://github.com/johnyerhot/rdigg/tree/master. I’ll have the Rdoc up at rdigg.yerhot.org later tonight You can peruse the documentation at rdigg.yerhot.org. I highly recommend you check out all methods that are available to you.

If you want to really dig in (sorry couldn’t resist) I’d go over the Digg API wiki to see what arguments you can pass.

I hope you enjoy the plugin!

Simple Meta Programming in Ruby

Sunday, August 10th, 2008

Metaprogramming.  What a lovely buzz word.  I guess I’ve heard it enough and knew what the short definition is. Metaprogramming is code that writes code.  I think it is one of those things I just never thought about, even though I had used concepts and even written some before without realizing it until recently.

Here is a short and simple example.

class Something

@my_hash = {"foo" => "1234", "bar" => "5678"}

def initialize
  @my_hash.each do |a, b|
       self.instance_eval do
             define_method(a.to_s) {b.to_s}
       end
  end
end
end

And now we can play with it.

a = Something.new
a.foo  # => 1234
a.bar # => 5678

Now, what happened here is pretty neat in my opinion. We took our hash, my_hash, and in our initialize method, created two instance methods from its values. define_method is what did all the magic. You need to pass it a proc or, like what I did, just give it a simple block (just the string value from my hash).

Pretty neat, eh?