Software and Other Stuff


29 Feb 2012

Net::HTTP not setting cookies... or so it looked like

I was retrieving a web page from a site, but I found out it needed a cookie to be set in order to work. Otherwise, it was redirecting me to set up my language of preference. I thought no problem... let's set up that cookie.

require 'net/http'

url = "http://www.realtor.com/realestateandhomes-detail/3429-W-Lone-Mountain-Rd_North-Las-Vegas_NV_89031_M12163-28068"
uri = URI(url)
http = Net::HTTP.new(uri.host, 80)
request = Net::HTTP::Get.new(uri.request_uri)
request['Cookie'] = "preference=USA"
r = http.request(request)

Awesome! it worked! Sometimes...

After a bit of banging my head against the wall I decided to try with a different version of ruby, 1.9.3.

Locally I was using ree-1.8.7-2011.12, and on the c.i. server ruby-1.8.7-p352.

So what was the difference? Check it out.

Turns out 1.9.3 sets a default user agent. Even though it should be optional, looks like some servers do not look at the cookies header if the user agent is not set.

The takeaway: if you are setting cookies on your request, make sure the user agent is being set too:

request['User-Agent'] = 'MyCoolUserAgentString'

Related Posts

28 Mar 2014
» Fun with Google's Apps Script! Batch Processing Gmail emails By Host
07 Oct 2013
» Graphs are Everywhere! An overview of GraphConnect San Francisco 2013
23 Jul 2013
» Using AngularJS with a Rails backend


The opinions expressed in this personal blog represent my own and not those of my employer. All data and information provided is for informational purposes only. This blog makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information on this site and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis.
adapt.960.gs  |  icons