FAQ Database Discussion Community


Unable to modify active record inside of net/http callback

ruby-on-rails,ruby,http,net-http
I'm unable to set an attribute in the callback of a Net::HTTP GET request. Why is this happening? def validate(params) ... net http setup ... res = Net::HTTP.get_response(URI(path)) print res.body print res.code << prints 200 if res.code == 404 puts "404" elsif res.code == 200 user.validated = Time.now user.save valid...

Net/HTTPS not getting all the content

ruby,web-crawler,nokogiri,net-http,mechanize-ruby
I need to login into Jenkins through a crawler to collect some data, but Net/HTTPS gets an incomplete page in comparison to Jenkins' source, here are both sources: Net/HTTPS' HTML <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta http-equiv="refresh" content="1;url=/login?from=%2F"> <script> window.location.replace('/login?from=%2F'); </script>...

Rails ping website and evaluate Net::HTTP response

ruby-on-rails,ruby,ping,net-http
I'm making a simple way to check is a site is up or not, this is my Ping model that holds a few adresses I want to check require 'net/http' def self.check pings = Ping.all pings.each do |p| http = Net::HTTP.new(p.address,80) response = http.request_get('/') if response.message == ( 'OK' or...

Authentication Issue with WP-API and Net::Http

ruby,wordpress,oauth,wordpress-plugin,net-http
I am new to Wordpress. I want to access WP-API to create a post using REST API. I am unable to perform POST, PUT and Delete request. Though I can retrieve data only. I wrote a ruby script for this. Below is my code snippet. require 'net/http' require 'net/http/oauth' require...

HTTP Post Rails 4 Issue

rest,ruby-on-rails-4,xmlhttprequest,http-post,net-http
Thank you in advance for any and all help. I've been having a ridiculous issue with trying to get my rails 4 app to take in data from an email parsing API that sends an XML response to a specified url within my application. For whatever reason I can't seem...

OpenSSL::SSL::SSLError while making post request with Net::HTTP

ruby,http,openssl,ssl-certificate,net-http
I tried different approaches(from google) to solve this error, but failed. I am unable to resolve this problem. My code looks like this: require 'net/http' require 'uri' require 'json' uri = URI.parse "https://dev.ramble.com/v1/user/login" uri.user = "user1" uri.password = "mypass" connection = Net::HTTP::new uri.host, uri.port connection.use_ssl = true connection.ca_file = "/usr/local/etc/openssl/cert.pem"...

Wait for selector to present

ruby,nokogiri,net-http
When doing web scraping with Nokogiri I occasionally get the following error message undefined method `at_css' for nil:NilClass (NoMethodError) I know that the selected element is present at some time, but the site is sometimes a bit slow to respond, and I guess this is the reason why I'm getting...

Ruby - WebCrawler how to visit the links of the found links?

ruby,url,hyperlink,web-crawler,net-http
I try to make a WebCrawler which find links from a homepage and visit the found links again and again.. Now i have written a code w9ith a parser which shows me the found links and print there statistics of some tags of this homepage but i dont get it...

net/http automatically redirects webpage to another language

ruby,web-scraping,nokogiri,net-http,open-uri
I'm trying to scrape the data of webpage "https://www.zomato.com/grande-lisboa/fu-hao-massamá" using open-uri. But, the website is automatically redirecting it to "https://www.zomato.com/pt/grande-lisboa/fu-hao-massamá". I don't want the spanish version. I want the english one. How do I tell ruby to stop doing that. Please help...