proxy,yql,head , No head section when requesting page via YQL API

No head section when requesting page via YQL API


Tag: proxy,yql,head

I'm trying to get the full page via yahoo YQL API, in order to access metadata fields defined in the target page head section (specifically open graph metadata).

My example URL:*%20from%20html%20where%20url%3D%22

But it does not contain any metadata there another way to call this Yahoo API to include the head section or perhaps a similar service that does include the head section?


You cannot use YQL to parse directly, use the Community Tables

Here's an example to get the meta tags of via YQL.

SELECT * FROM html WHERE url = '' AND xpath='descendant-or-self::meta'

Partial Query Output:

        <meta charset="utf-8"/>
        <meta content="width=device-width, initial-scale=1.0" name="viewport"/>


I need to do a redirect on a proxied path in Nginx

I'm working with a poorly architected app and I have a need to proxy / and redirect it to a sub-directory (unless someone can comeup with a better solution). Here's the layout of the paths the app uses: var securePaths = [ "/someaction", "/anotheraction", "/athirdaction", "/client/interfaceishere/index.html" //path to client interface...

Proxy awareness with pip

I have tried to set up a python development environment at my work place to create some basic applications. I can't seem to get pip to work, for example I type in pip install numpy and get the following error message: Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by...

Can't install Composer on Ubuntu behind proxy

I'm trying to install Composer, in order to use Laravel, but I'm behind the company proxy. The proxy is already configured in the system, so wget --proxy-user=<my_user_name> --proxy-password=<my_password> works (curl doesn't!), and I get the 270kB "installer" file. Next, I'm trying to run php installer as the manual says,...

what is “root:'data' ” extjs store config

I am very new to ExtJs and I have the following code for a treeView,,which has a default root in the beginning and then an Ajax call to a servlet .But i don't understand the 'root' field with value 'data' .Is data an alias or xType .Here is the code:...

How to connect to a SOCKS proxy in a java applet

I've made a Java applet that accesses a webpage. I need it to connect to a website via a SOCKS proxy I've already tried putting this code where my program initialises: System.setProperty("socksProxyHost", ""); System.setProperty("socksProxyPort", "1080"); But nothing seems to happen and it just uses my normal IP address?...

How to set up a reverse proxy in nodejs for multiple targets?

I am using nodejs, expressjs and request to create a reverse proxy. What I want is that when a request is sent to http://localhost:PORT/NAME/ that the corresponding page is loaded. For example, if I enter http://localhost:1234/google/ the google website would be loaded and if I enter http://localhost:1234/stack/ the stackoverflow page...

Web Service Client - Construction Failing Due To Timeout (Proxy Issue)

I have a webservice in Java 1.6 that extends The WSDL URL is located at an HTTPS endpoint and I am behind a corporate proxy (NTLM I believe). I have the proxy host, port, username, and password. I have verified that I can access the WSDL using curl if...

Ember CLI with Multiple Proxies

The Problem: I have an Ember CLI Application that will consume multiple APIs, which I need to proxy to in development mode. Background: I have a legacy api which exposes services at /api running on my local development machine at localhost:3000 I have a new api which exposes services at...

How to pass nginx proxy url for socket

I am using socket in my application,i want to pass socket connection url from nginx as proxy url.I am doing it like that My socket code var socket = io.connect('/explorer/socket',{ 'reconnect': true, 'reconnection delay': 500 }); My nginx conf location /explorer/socket { proxy_pass; } but it is not working,it...

WSO2 DOCTYPE not allowed calling REST Api

i m working with this proxy: <?xml version="1.0" encoding="UTF-8"?> <proxy xmlns="" name="PubMedDataProxy" transports="https,http" statistics="disable" trace="disable" startOnLoad="true"> <target> <inSequence> <call> <endpoint> <http method="get" uri-template=";Mdb=pubmed&amp;id=230900"/> </endpoint> </call> <log level="custom"> <property name="MyLOg"...

Apache proxy - HTTP to HTTPS

I work with application which comunicate with server over HTTP, but this server runs on HTTPS in my case and there is also basic authentication. There is no way to change this settings. My idea is configure proxy on Apache server and then website wil be accessible at http://localhost/ and...

ENOTFOUND error with express.js (nodejs)

whenever I try to do some redirecting to another web page (e.g. with my nodejs express server it will fail returning the following error: throw er; // Unhandled stream error in pipe. ^ Error: getaddrinfo ENOTFOUND at errnoException (dns.js:44:10) at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:94:26) I tried many differend...

npm ERR! Windows_NT 6.3.9600, unable to install cordova in windows8

I tried to install cordova on my windows8 machine by using the below command in command prompt npm install -g cordova but i'm not able to install because of the following error can any one help me out please. Error: npm ERR! Windows_NT 6.3.9600 npm ERR! argv "C:\Program Files\nodejs\\node.exe" "C:\Program...

ZMQ: No subscription message on XPUB socket for multiple subscribers (Last Value Caching pattern)

I implemented the Last Value Caching (LVC) example of ZMQ (, but can't get a 2nd subscriber to register at the backend. The first time a subscriber comes on board, the event[0] == b'\x01' condition is met and the cached value is sent, but the second subscriber (same topic) doesn't...

HTTP to HTTPS mapping using proxy servers

I have a java application which is trying to call a HTTPS endpoint which is setup in my internal network. Also this request go through a corporate proxy. Having said that, I don't want to implement a HTTPS client at my application level. Instead I will just trigger a plain...

Using phantomjs print proxy it used to access website

can someone point me in the right direction. Just need some documentation. I manually input a proxy, but I think it might be by passing it. I want to test my script to see if its actually going through my proxy with phantom. It looks like I successfully went through...

Powershell Proxy setting

With this cmdlet can I check my proxy setting in powershell: $regKey="HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings" $proxyServer = Get-ItemProperty -Path $regKey $proxyServer | fl *proxy* Or with this cmdlet: Get-ItemProperty Registry::HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings' | Select-Object *Proxy* My question is, how can I get the proxy settings for another network user?...

Can the HTTP method “PATCH” be safely used across proxies etc.?

Suppose my server exposes an HTTP-based API that uses the PATCH method introduced by RFC 5789. Is it possible that clients (browsers or otherwise) behind corporate firewalls, proxies, caches, parental controls filters and the like will encounter any problems using this method? If so, how likely is this? Given that...

How Can I Run a Local PHP Simple HTML DOM Parser with a Proxy?

I have a PHP Simple HTML DOM Parser locally in MAMP that pulls information and works well with the Japan version of a website, since I'm located in Japan. However, I would like to pull information from the UK version of the site. What is the simplest way to do...

Node JS TCP Proxy: Set up HTTP Tunnel using Node JS

I'm trying to implement a TCP proxy in Node JS. I only have some experience with Javascript so I met a lot of problems along the way. I've done a lot of searching for this one but had no luck. I want to implement a two-hop TCP connection for HTTPS...

Proxying casperjs / phantomjs through Charles

I'd like to use casperjs / phantomjs to autofill some web forms. In order to debug my casperjs scripts, I'd like to use the (wonderful) Charles Web Proxy as a man-in-the-middle SSL Proxy, but haven't figured out how to make that work in casperjs / phantomjs. What works I've enabled...

Apache2 mod_auth_form too many redirects under Location

I am trying to build a Proxy HTTP Authorisation page using mod_auth_form My goal is to have a single Auth page in the DocumentRoot directory, then once the user is connected, just Proxy all routes to the "real" application, running on localhost with another port. I set up my vhost...

How to add http proxy for apachebench(ab)

I want to grab http packets which ab sends or receives. So I add a http_proxy in shell command: $ export http_proxy= Then I execute ab command: $ ab -c 1 -n 1 http://localhost/ Finally, my proxy( failed to get http packets from ab. Is there any way to let...

Camel as an HTTP Proxy to a REST service - How to route paths and parameters?

I am trying to use Apache Camel as an HTTP proxy to a REST service, performing some auth in the middle, and have a route which looks like this (auth removed for testing purposes): from("servlet:apiwrapper?matchOnUriPrefix=true") .to("http://HOST/BASEPATH?bridgeEndpoint=true&amp;throwExceptionOnFailure=false"); When accessing the Camel Servlet path like so: http://CAMELHOST/apiwrapper/node http://CAMELHOST/apiwrapper/node/stuff/blah?etc=t And so on... I...

Redirect Static Request Express Nodejs

I am making a game using websockets. I own 3 Nodejs servers:,, I want game1 and game2 to only handle game logic (websocket communication) while handles all http requests. Players go to and connect with server with websocket. If players request static files (images,...

Cannot find .bowerrc file

I am trying to install Learning Locker on my Ubuntu 14.04 which runs apache2 and php5, I have installed all the dependencies listed here. How ever when I run composer (actually its composer.phar, I dont know why) it shows that its downloading but bower shows ECONNRESET errors all along. So...

How can I make a special URL like "http://appname/

I know some applications run on companies' internal networks with URLS like "https:/statsd/", so I was wondering how to go about doing this. I was thinking maybe I would use a reverse proxy like Nginx to proxy all urls that match a particular regex to the correct server, but I...

Node JS TCP Proxy: Reuse socket in callback function

I'm trying to implement a TCP proxy in Node JS. I only have some experience with Javascript so I met a lot of problems along the way. I've done a lot of searching for this one but had no luck. The problem occurs when browser sends a CONNECT request for...

I dont know how to add Proxy to my Phantomjs script

I've been having issues accessing websites and just realized that since the websites are controlled by my company I need to add a step in my script that shows the webpage I have my companies proxy IP or Im on their server. I hope that makes sense. For example if...

How to configure AppJS to work with node-http-proxy server

I'm trying to get node-http-proxy working with AppJS. Unfortanly it crashes on launch of the app. What I did: Download & extracted AppJS from; Installed node-http-proxy with npm install http-proxy; Edited the app.js window.on(create) function: window.on('create', function(){ console.log("Window Created");;; window.frame.setMenuBar(menubar); var http = require('http'), httpProxy = require('http-proxy');...

HTTP Proxy Servlet Error

Let me start off by saying this was working yesterday, so I'm totally confused as to why it stopped working all of a sudden. And Java / Plugins are really not my specialty. I'm working on an application that makes POST and GET calls to a server on another domain....

Create proxy according URL path

I need to create proxy according to some URL which is coming from the browser, since I fairly new to this topic Im not sure how to test it...:( I need some way to test it and see that this is actually working I use this following code from this...

Request from different Country IP

I would like to view the movie content from a different country on the Playstation store. However, the playstation store blocks the IP based on location of the request, so a movie in Canada cannot be viewed from the US:!/en-ca/movies/the-house/cid=UV0130-NPVA92773_CN-0000000000236063 Is there a way to do something like the...

Nginx proxy_pass doesn't work on port 80

I am using a DigitalOcean VPS hosting a meteor app. I don't have a domain name yet, so just use the plain IP address. When I set below config and use myipaddress:3000 and myipaddress:8080, both of them worked well; but if I change the 8080 to 80, only myipaddress:3000 works....

How to add custom cookies in browsermob proxy on java

I am writing automation test and to capture the network call made in background, I am using browsermob-proxy. In browsermob-proxy, I want to set cookie before making requests. How can i do it? Below is my code:- String strFilePath = "data.har"; // start the proxy ProxyServer server = new ProxyServer(4444);...

Is it possible to make two SOCKSv4a session connections over one TCP connection?

I'm trying to create multiple tunnels out of a single TCP connection to a SOCKSv4a proxy server in order to keep from closing and reopening a bunch of sockets to the SOCKS proxy. Something like "Connection: keep-alive". How can I do that? Does the protocol allow it?...

Not able to access Server-Sent-Events over Mobile 3g Network

I am having an issue with Server Sent events. My endpoint is not available on mobile 3G network. One observation I have is that a https endpoint like the one below which is available on my mobile network. But the same endpoint when proxy passed using an nginx and...

how do you download files via ant though proxy [duplicate]

This question already has an answer here: Ant GET task and proxy 1 answer I am using ant which uses an build.xml file. This is the command: java version : 1.8.0 ant version: 1.7.1 CLASSPATH=/app/hbase-0.94.27/lib/hadoop-core-1.0.4.jar CFLAGS=-m64 CXXFLAGS=-m64 ant compile-native tar it hangs here: Buildfile: build.xml ivy-download: [get] Getting:

Why is my real IP address still visible even when using a proxy?

I am trying to understand how it is that our real IP address is discovered even after going through a private exclusive (I'm the only user) high security proxy. PROXY_IP = "<private proxy IP>:<port>" options = webdriver.ChromeOptions() options.add_argument("--proxy-server=%s" % PROXY_IP) driver = webdriver.Chrome(executable_path=".\\driver\\chromedriver.exe", chrome_options=options) driver.get("") This website displays a bunch...

DocumentDB Java SDK behind a proxy

I'm trying to create a simple web application from the todo tutorial. As I'm behind a corporate proxy, I used the following common java system properties : -Dhttp.proxyHost=myproxy Dhttp.proxyPort=3128 -Dhttps.proxyHost=myproxy -Dhttps.proxyPort=3128 I also try the following parameter: I still have a client exception java.lang.IllegalStateException: Http client execution failed. at...

Persistent session in glassfish behind nginx proxy

I am running a glassfish webserver behind a nginx proxy (which resolves domains). I want to store @SessionScoped data on the Webserver. My problem is, that nginx opens a new session everytime it connects to glassfish and that session is in no way connected to the client. The client itself...

How to specify proxy authentication credentials in Apache httpclient in addition to the target site credentials?

I need to connect to a remote web site via proxy using Apache httpclient. Both the proxy and the web site are using basic authentication with different logins/passwords. I have the below code that connects to the remote site via proxy. But I can't figure out how to add proxy...

actionHero.js Set Http Header

I want to know if there is a way to Set the Http.Header in actionHero.js: in our server we are working with actionHero.js and Golang I need to send on the "Header" two domain name "client domain & mine(actionHero Service)" to the Golang service. or if there is any other...

How to configure proxy using Squid

I have a connection and I want change the connection behavior of the process to connect through that proxy. I would also like it to use a new IP in the event that it detects that it has been throttled. The new connection process that I want is : connect...

Domino Xpages with Reverse Proxy

I have a problem with Domino Web Server with an XPages Application which is placed behind some reverse proxy. The problem is, the proxy forwards all requests from the URL like h2tps://organization/test_server/ to the Domino Web Server. This makes all links in the application brocken. And I don't know how...

My workplace's proxy is preventing me from using Maven

I work in the public sector so my workplace controls internet access pretty strictly. I've tried configuring the settings.xml file that M2Eclipse reads to no avail. It may be simple user error in my case, but I'd like to explore other options anyways just in case that it isn't. I...

NGINX Serve all static files and forward others to proxy without / giving 403?

I'm trying to configure an nginx server to act as a proxy to a node application. What I would like is to have any static files served by nginx and all other paths forwarded on to the Node.js application (inluding the / path). I've seen the following question/answers: How to...

Reverse Proxy in Python using WSGI

app, built with Flask. I want to have the following routing : "/" -> my Flask app "/foo/" -> Reverse proxy toward http://bar/ So far, I don't have any reverse proxy, so my application looks like : import app [...] if __name__ == '__main__': = 'XXX', use_reloader=False) I...

Parse Returned Text into JSON Object in Swift?

Solved: I was able to solve the problem in question with this: if let returnedData = NSJSONSerialization.JSONObjectWithData(data, options: nil, error: nil) as? NSString? { let dataString = NSString(data: data!, encoding: NSUTF8StringEncoding)! let trimmedDataString = ((dataString.stringByReplacingOccurrencesOfString("YAHOO.Finance.SymbolSuggest.ssCallback", withString: "") as NSString).stringByReplacingOccurrencesOfString("(", withString: "") as NSString).stringByReplacingOccurrencesOfString(")", withString: "") let dataFromDataString =...

How to check Proxy headers to check anonymity?

I'm trying to determine high anonymity proxies. Also called private/elite proxies. From a forum I've read this: High anonymity Servers don't send HTTP_X_FORWARDED_FOR, HTTP_VIA and HTTP_PROXY_CONNECTION variables. Host doesn't even know you are using proxy server and of course it doesn't know your IP address. A highly anonymous proxy will...