FAQ Database Discussion Community

Mechanize: Unable to redirect to final destination

I am using mechanize to try to login to a site. However when logging in with mechanize, I cant seem to get to final destination. Below is the example. Mechanize go to (1) https://app.abc.com/users/login If login successful, webpage will momentarily post to page link below (2) https://paid.abc.com/Login.aspx?CID=123&XRALID=321&LanguageID=1&ExecuteLogin=1 From there automatically...

Net/HTTPS not getting all the content

I need to login into Jenkins through a crawler to collect some data, but Net/HTTPS gets an incomplete page in comparison to Jenkins' source, here are both sources: Net/HTTPS' HTML <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd"> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <meta http-equiv="refresh" content="1;url=/login?from=%2F"> <script> window.location.replace('/login?from=%2F'); </script>...

How do I ignore the nil values in the loop with parsed values from Mechanize?

In my text file are a list of URLs. Using Mechanize I'm using that list to parse out the title and meta description. However, some of those URL pages don't have a meta description which stops my script with a nil error: undefined method `[]' for nil:NilClass (NoMethodError) I've read...