FAQ Database Discussion Community


Can I stop casper.fetchText from removing
automatically?

javascript,parsing,text,casperjs
casper.fetchText seems to automatically remove <br> in the fetched text. e.g. if the text is abc<br>de, then casper.fetchText will return abcde instead of abc<br>de. Is there any way to keep it as abc<br>de?...

Scraping only when a change is detected?

javascript,phantomjs,screen-scraping,casperjs
My bank has a really simple login system. Using casperjs I have been able to pull my latest account balance and my last transaction. There are many ways to scrape data off the Internet, I just used Casperjs to test out its capabilities.I checked with the bank and they say-...

How to determine if a specific input exists using CasperJS

javascript,css-selectors,casperjs
My Html <input name="SUBMIT-chn:$INTERNAL_password.pss" title="Select" class="image selectIcon" type="image" alt="Select" src="docs/pics/select.png"> I am trying to use CasperJS to know if this exists and if it does then "Click" it? var casper = require('casper').create(); var site = 'http://internalsite/username=abc'; var exists; casper.start(site, function() { exists = this.evaluate(function() { return __utils__.exists('image selectIcon'); }); });...

Not possible to trigger GWT listbox ChangeHandler using CasperJS

javascript,gwt,phantomjs,casperjs
I'm writing tests using CasperJS for legacy GWT (2.3) code. I can change the selected value of a ListBox. document.querySelector('#id_of_select').selectedIndex = 1; But this does not trigger the onChange() method of a change handler on the ListBox. I have tried manually dispatching a change event on the select element (jquery...

Get child elements within casper.each

javascript,casperjs
Using CasperJS 1.1 with the following codes, I'm able to fetch useful DOM html from web page. casper.each(c.getElementsInfo(xpath), function(casper, element, j) { var html = element["html"].trim(); if(html.indexOf('Phone') > -1) { // what should I put here? } }); However, I want to access & obtain the child elements of the...

Accessing inner scope variable values on the page with CasperJS

javascript,variables,casperjs
I'm new to CasperJS and trying to get some JS code from a website. Let's say I got this code: <html> ... <script type="text/javascript"> var a = "true"; function abc(){ var b = "someValue"; b = b + " and other value"; } </script> ... ... </html> I'm able to...

How to apply :contains pseudo class for PhantomJS/CasperJS?

jquery-selectors,casperjs
Is there a way to select a tag containing specified text like the :contains selector does with jQuery in CasperJS?

Not able to use sendKeys in dialogue box

javascript,casperjs,slimerjs
i am using casperJS to import files, and below is the HTML script and picture of the dialogue box <div class="container"> <p>Import notebooks from another GitHub instance.</p> <p>Currently import does not preserve history.</p> <p> source repo api url: <input id="import-source" class="form-control-ext" type="text" value="https://api.github.com" style="width:100%;"> </p> <p> notebooks: <br> <textarea id="import-gists"...

Count DOM elements with CasperJS' evaluate failed

javascript,css-selectors,casperjs
I've just got trouble with CasperJS. I need to count table rows, since there are many tables that contain same information structure (lets say user table) So I do some casperjs evaluate code like this var table_rows1 = casper.evaluate(function(it){ return jQuery("#dResult > div:nth-child("+1+") > div > div:nth-child(4) > div:nth-child("+it+") >...

CasperJS: How to fill a asp form

javascript,asp.net,xpath,phantomjs,casperjs
I'm trying to automate a few steps of a asp written website through casperjs. I want to acheive following. Navigate to the login page Fill the username and password fields click the login button capture a screenshot of the success page But so far I'm only able to do the...

How to use HTTP basic auth in SpookyJS

javascript,node.js,casperjs,basic-authentication,spookyjs
I wrote a little bit code to go to my Wordpress page with CasperJS with a htaccess protection. The script should login an then later update the plugins. At this time I can login and create a screenshot from the plugins who should updated. (also the CasperJS works fine) Now...

Simplify mapping multiple children elements in CasperJS to use only one function

javascript,casperjs
I have this code, which works, but would like to simplify it. In particular, I believe there is a shorthand method for referring to children elements so that I do not need to write multiple functions. Please let me know if there is a shorthand method to simplify this code!...

How to fill a form with CasperJS or PhantomJS? [closed]

javascript,forms,phantomjs,casperjs
I want to fill a form by using id, cssSelector, className, name, XPath etc... Is it possible by using CasperJS or PhantomJS?

I dont know how to add Proxy to my Phantomjs script

python,proxy,phantomjs,casperjs
I've been having issues accessing websites and just realized that since the websites are controlled by my company I need to add a step in my script that shows the webpage I have my companies proxy IP or Im on their server. I hope that makes sense. For example if...

How to retrieve browser web console log and write it into a file?

javascript,automation,casperjs
I am trying to process the log from a specific webpage. Are there any suggestions how to retrieve the logs and put into text file from any web page using CasperJS?

Catch errors caused by evaluate() in CasperJS / PhantomJS?

javascript,phantomjs,casperjs
I'm wondering why the following code isn't picking up errors output in the console by javascript code executed using the evaluate() method. casper.on 'resource.error', -> @echo 'Resource error: ' + "Error code: "+resourceError.errorCode+" ErrorString: "+resourceError.errorString+" url: "+resourceError.url+" id: "+resourceError.id, "ERROR" Since evaluate() executes the code in the context of the...

CasperJS waitForResource again

javascript,casperjs
Im using .waitForResource('/auth') .then( .... ) .waitForResource('/auth') .then( .... ) To wait for an ajax call to /auth to complete before then validating the UI response. However, the subsequent call to /auth appear to continue immediately as it has already loaded that resource. Is there some way to tell casperJS...

CasperJS - Click button based on value stored in variable

javascript,casperjs
Based on the input variable "feeling", I would like to click a button (good,bad or ok). How do i specify the input variable in casper.click? var feeling = 'bad'; //User can choose good, bad or ok . . logit_feeling(feeling); //fucntion call . . // fucntion defnition var logit_feeling = function...

How to fill a select element which is not embedded in a form with CasperJS?

javascript,casperjs
My html code is: <div class="setting-control"> <select class="on-off" id="custom-cache-pref"> <option value="">Default</option> <option value="byc">Bypass cache</option> <option value="basic">Basic caching</option> <option value="iqs">Ignore query string</option> <option value="agg">Aggressive caching</option> <option value="all">Cache everything</option> </select> </div> Usually with casperjs I would use...

How to download a file through a custom POST request with CasperJS

javascript,encoding,casperjs,binaryfiles
I am writing a crawler and needs to download file generated after a form request using POST. I have successfully used this.download(url,'POST',Params) for regular forms. One of the sites has many fields using the same name, thus preventing me from using the regular download method. After trying a lot of...

Using document.querySelectorAll to get textContent with CasperJS

javascript,phantomjs,casperjs,selectors-api
I am automating filling out an online form with CasperJS. Whenever you try to submit the form where you've filled out any of the required fields with invalid data, an error message appears by each of the fields that has invalid data to alert you to what the problem is....

How do I set a variable from casper.evaluate()?

javascript,phantomjs,casperjs
I'm trying to set a value from within casper.evaluate() that I will check later to run a test on, but it doesn't seem to be working. isArticleOrReview = false; casper.waitFor(function check() { //here I'm just waiting for jQuery to load return this.evaluate(function() { return jQuery.fn.jquery == '1.2.6'; }); }, function...

Can´t load casperjs module placed at the same parent node_modules directory as spookyjs

javascript,node.js,casperjs,spookyjs
I only have spookyjs installed in the node_modules folder of my project, but I am only able to successfully execute my scripts (or the example one included) when casperjs is installed with the global (-g) flag, not when I have it in the same nodes_modules as spookyjs. Is there any...

CasperJS - How to open up in parallel all links from an array of links

phantomjs,casperjs
I need to open all the links of the array in parallel. How to make it? In my code, all links will be open one by one, instead of parallel. Here is my code: casper.then(function(){ links = this.evaluate(function(){ var links = document.getElementsByTagName('a'); links = Array.prototype.map.call(links,function(link){ return link.getAttribute('href'); }); return links;...

CasperJS equivalent to RSelenium for filling a form

casperjs,rselenium
I have an Rselenium script to fill in a form, but am trying to use CasperJS as I am finding Rselenium too slow. The following code will navigate the form as I expect. remote.driver$navigate("http://news.ceek.jp/search.cgi?kind=sports") search.form <- remote.driver$findElement(using = "xpath", "//*[@id='query']") search.form$sendKeysToElement(list("SearchTerm",key = "enter")) The equivalent CasperJS code I have tried...

Scrape links, store in array and then run another process in CasperJS

javascript,casperjs
I currently have two CasperJS scripts that I want to combine into one for usability purposes. test1.js scrapes a webpage for links ( <a> elements). All the resulting links scraped are stored in an array urls. Script test2.js takes a link and extracts youtube src link if present from iframe....

phantomjs --web-security=no

security,phantomjs,casperjs
In the documentation for phantomjs, there is an option to turn off web security and allow XHR (cross domain requests). Does this present a security risk to a user if you are filling forms with credentials such as usernames and passwords and then downloading screenshots with casper/phantom?...

Unable to click in CasperJS

javascript,web-crawler,phantomjs,casperjs
I want to crawl the HTML data. And, I tried headless browser in CasperJS. But, Can't able to click. - The following is tried code in CapserJS. var casper = require('casper').create(); var mouse = require('mouse').create(casper); casper.start('http://sts.kma.go.kr/jsp/home/contents/climateData/smart/smartStatisticsSearch.do', function() { this.echo('START'); }); casper.then(function() { this.capture("1.png"); this.mouse.click('li[class="item1"]'); casper.wait(5000, function() { this.capture("2.png"); }); });...

Loop for casper.start only executed once

javascript,casperjs
I got the following script to loop through a page's options to fetch subsequent values: var casper = require('casper').create(); casper.on('remote.message', function (message) { this.echo(message); }); casper.on( 'page.error', function (msg, trace) { this.echo( 'Error: ' + msg, 'ERROR' ); }); casper.start(url, function() { this.evaluate(function() { // nothing }); this.then(function() { ddlArea_options...

PhantomCSS not working with this website

phantomjs,casperjs,phantomcss
I'm trying to make this test work but it seems PhantomCSS can't take a screenshot of this particular website. /* Require and initialise PhantomCSS module Paths are relative to CasperJs directory */; var phantomcss = require('./phantomcss.js'); casper.test.begin('Test', 5, function(test) { phantomcss.init({ screenshotRoot: './screenshots', failedComparisonsRoot: './screenshots', libraryRoot: '.', }); casper.on("resource.error", function(msg,...

Wait for a web page alert in CasperJS

javascript,automation,alert,casperjs,headless
I'm a newcomer to CasperJS and after a couple hours I can login and navigate a few webpages with it, but I'm stumped by the alert message on this website: https://www.macysliquidation.com/ I need to get rid of the alert so I can login. My simple (non-working) code is: var casper...

Finding elements using __utils__.getElementsByXPath in CasperJS

javascript,xpath,phantomjs,casperjs
A snippet from my casper.js test script is given below; var refObject = undefined; casper.then(function() { refObject = this.evaluate(function() { return __utils__.getElementsByXPath('//div[contains(concat(" ", @class, " "), " refObject ")]//a[contains(normalize-space(text()), normalize-space("SampleTest"))]//../../../..'); }); require('utils').dump(refObject); this.echo(refObject); test.assertTruthy(refObject); }); Here inside evaluate function I am trying to get reference to a html element which...

How to verify the sorting(ascending and descending) functionality in casperjs

javascript,testing,order,casperjs,columnsorting
I am new to this framework and also not getting any idea how to test the sorting functionality with CasperJS. The scenario is, I need to verify the search results with sorting functionality. When i search with any keyword it displays all the possible matching results. ...

CasperJS tester module output to csv instead of command line

javascript,csv,automated-tests,casperjs
Using a CasperJS test module, is there a way to output the assert outputs to a CSV instead of a command line?

SlimerJS window is not opening instead it is showing message in alertbox

casperjs,xvfb,slimerjs
We have installed slimerjs,phantomjs,casperjs on VM(we are accessing it through 'putty').When we tried to execute the casperjs test suite by following command: casperjs test --engine=slimerjs 'testsuite name' --username --password we are getting an alert message saying, slimerjs is already running,but is not responding.To open new window, you must first close...

How to use keyboard commands using sendKeys in CasperJS

javascript,casperjs
I want to use left arrow key (keyboard) in CasperJS. Please tell me how to use this one. The scenario is I am adding one line in the cell and I want to place the cursor between the cells so that I can split the cell in to two halves.

How to import other js files in casperjs

javascript,testing,phantomjs,casperjs,file-import
Basically,I have extracted the re-usable functions into a file 'utility.js'.I want to import this file into every CasperJS test script.I'm following the documentation here When i try to get it to work,I'm getting the following error. 'undefined' is not a function Folder structure is like this utility.js test1.js test2.js Test...

Casperjs not extracting value of the class

javascript,jquery,casperjs
I am using CasperJS to log into hasjob.co and scrape some information from the homepage. I did most of the work fine, like getting things clicked and logining in with jquery, but I am stuck at actually extracting the value of the tag that I want. I use the below...

How show time for each test with casperjs

javascript,unit-testing,testing,casperjs,e2e-testing
I have a "launcher" script with one array, with all names and routes of many tests. When I run all tests, casper only shows the total time of run all tests. I need show how much time needs each test script, does casperjs have any method for do this? When...

How to tell CasperJS to loop through a series of pages

javascript,while-loop,casperjs
I try to make CasperJS achieve the following: Go through a series of pages that are named sequentially by date. On each page, locate a PDF link. Download the PDF. I got some working code, but I don't understand how CasperJS is going through the sequence of events. For instance,...

How to download a csv file after login by using Casperjs

javascript,csv,casperjs
I want to donwload a csv file by using Caperjs. This is what I wrote: var login_id = "my_user_id"; var login_password = "my_password"; var casper = require('casper').create(); casper.userAgent('Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36 '); casper.start("http://eoddata.com/symbols.aspx",function(){ this.evaluate(function(id,password) { document.getElementById('tl00_cph1_ls1_txtEmail').value = id; document.getElementById('ctl00_cph1_ls1_txtPassword').value = password;...

Differences when using functions for casper.evaluate

javascript,function,casperjs
I'm using PhantomJS v2.0 and CasperJS 1.1.0-beta3. I want to query a specific part inside the page DOM. Here the code that did not work: function myfunc() { return document.querySelector('span[style="color:#50aa50;"]').innerText; } var del=this.evaluate(myfunc()); this.echo("value: " + del); And here the code that did work: var del=this.evaluate(function() { return document.querySelector('span[style="color:#50aa50;"]').innerText; });...

Hover over element with CasperJS

javascript,hover,casperjs
How to perform click event on the element which becomes visible after hovering on an element. below is the HTML code <div class="jqtree-element jqtree_common"> <span class="jqtree-title jqtree_common" contenteditable="true">Notebook 1</span> <span class="notebook-right"> <span class="notebook-date disappear" style="visibility: visible;"> <span class="notebook-commands-right"> <span class="notebook-commands appear-wrapper"> <span class="notebook-commands appear" style="display: none;"> <span...

Black part of page during capture with CasperJS

javascript,phantomjs,screenshot,casperjs,capture
I have a strange problem. I have to take screenshots from vk.com in album with comments. In this case everything is ok: But when i click to load more comments, the screenshot has a black lower part: The code is pretty simple: // click on image casper.waitForSelector(".photo_row_img", function() { this.click('.photo_row_img');...

How to call python script from CasperJS

javascript,python,phantomjs,casperjs
I'm trying to call a python script from within CasperJS and grab the output from python. casp = require('casper').create({ verbose: true, logLevel: 'debug' }); casp.start().then(function() { var cp = require('child_process'); cp.execFile('/usr/bin/python','test.py', {},function(_,stdout,stderr){ console.log(stdout); console.log(stderr); }); }); casp.run(); test.py is just print "hello world" atm for testing, but this script when...

Phantomjs does not open web page

javascript,testing,phantomjs,casperjs
I have started testing campus2020 site with casperjs (1.1.0-beta3) + phantomjs (1.9.8). And faced with the problem that site is not opening but instead tests just freeze. I have taken script example from phantomjs site: var page = require('webpage').create(); page.open('http://informatik.uni-leipzig.de/campus2020', function(status) { console.log("Status: " + status); if(status === "success") {...

CasperJS form not submitting in AngularJS

javascript,angularjs,css-selectors,casperjs
I am going to use CasperJS for for testing, form submitting and UI testing, and I am new to CasperJS and AngularJS. While doing testing with CasperJS my form is not submitting, but it gives errors, that is, form not found. Here is my form <form role="form" name="login-form"> <div class="list...

Heroku - Redirect console output to Logentries

bash,postgresql,heroku,casperjs
How do I redirect console output of Heroku's bash to Logentries / other file log? I use the following commands: heroku run bash use the following command to run a long-running CasperJS script: casperjs myscript.js Then a lot of output appears in the console. How do I redirect the console...

Using non-ASCII character as Javascript object key [duplicate]

javascript,casperjs
This question already has an answer here: JavaScript property access: dot notation vs. brackets? 7 answers I have the following object: var obj = { 'ア' : 'testing', 'ダ' : '2015-5-15', 'ル' : 123, 'ト' : 'Good' }; How do I access the value by its non-ASCII key (it's...

Unsafe JavaScript attempt to access in capserjs [duplicate]

phantomjs,casperjs
This question already has an answer here: Using PhantomJS to embed all images of a webpage produces warnings but works 1 answer On each test run of casperjs test, the output closes with the following: Unsafe JavaScript attempt to access frame with URL about:blank from frame with URL file:///usr/lib/node_modules/casperjs/bin/bootstrap.js....

CasperJS is not working filling the google search field

javascript,firefox,phantomjs,casperjs,slimerjs
I'm using CasperJS (with the slimerjs engine) to fill a form. This is however not working. I tried it with a simple google.com script: var casper = require("casper").create({ verbose: true, logLevel: "debug" }); var fs = require("fs"); phantom.cookiesEnabled = true; casper.options.viewportSize = { width: 1024, height: 768 }; casper.start(); casper.userAgent("Mozilla/4.0...

CasperJS can't find element using CSS selector or xPath

javascript,xpath,css-selectors,casperjs
I'm trying to click on the 'Next' button on the Google search using CasperJS and I'm getting the following error: CasperError: Cannot dispatch mousedown event on nonexistent selector: #pnnext The button is conveniently tagged with the id #pnnext. I tried using both CSS selectors as well as xPath using several...

Sinon catch xmlhttprequest

javascript,jquery,mocha,casperjs,sinon
I have a question concerning Sinon testing. I am using mochacasper and I want to catch the XMLHttpRequest that is sent after I click a button. So far I am not able to catch the request... casper.then(function () { this.fill(...); }); casper.then(function() { this.click('#myButton'); }); casper.then(function () { //I want...

How to handle browser prompts in CasperJS

javascript,phantomjs,casperjs,prompt
The scenario is, when I click a button browser would show an alert which accepts user input field with OK and cancel buttons. Now please tell me how to handle this type of alert. As we know CasperJS doesn't displays the alert windows. This is the casperJS code casper.then(function ()...

Get value from nested JavaScript object in CasperJS

javascript,arrays,object,casperjs,associative
I'm trying to dig into a nested javascript array to grab the first instance of an object. Here's the code: var utils = require('utils'); var casper = require('casper').create(); casper.start('http://en.wikipedia.org/wiki/List_of_male_tennis_players', function() { this.echo(this.getTitle()); // Get info on all elements matching this CSS selector var tennis_info_text = this.evaluate(function() { var nodes =...

Loop refresh page until selector is available

javascript,phantomjs,casperjs
I've got a script in casperjs that goes to a website checks a box if available and then click on a button to continue with the script. right now I have a cronjob that exacutes this everysecond untill the checkbox is available. What I'd like to do is to execute...

What must be wrapped in then() statements in CasperJS? How to determine execution order of sync/async functions?

javascript,asynchronous,casperjs
I'm having something of a hard time determining what is asynchronous and what is not while running CasperJS, what must be wrapped in then() statements, and what is going to be evaluated when. I'll run into a problem somewhere that has to do with a fall-through break statement, variable scope,...

Using $.ajaxStop() to determine when page is finished loading in CasperJS

javascript,jquery,ajax,phantomjs,casperjs
So far in my tests written in CasperJS, I've been using waitForSelector() on page-specific elements to determine if a page has fully loaded (including all the async ajax requests). I was hoping to come up with a more standard way of waiting for page load and was wondering if the...

Can't load json from url with Casper Js -> “Loading resource failed with status=fail”

javascript,json,get,phantomjs,casperjs
I am new to casper js and I would like to retrieve a json object from a get request. The message I get is the following : Loading resource failed with status=fail: https://api.twitter.com/1.1/search/tweets.json?q=casperjs Here is my code : var casper = require('casper').create({ verbose: true, logLevel: 'debug', pageSettings: { loadImages: false,...

Proxying casperjs / phantomjs through Charles

ssl,proxy,phantomjs,casperjs,charles
I'd like to use casperjs / phantomjs to autofill some web forms. In order to debug my casperjs scripts, I'd like to use the (wonderful) Charles Web Proxy as a man-in-the-middle SSL Proxy, but haven't figured out how to make that work in casperjs / phantomjs. What works I've enabled...

Casper JS: TypeError: 'null' is not an object

javascript,null,phantomjs,typeerror,casperjs
I am trying to grab the "David Welsh" text on this page: http://foster.uw.edu/faculty-research/directory/david-welsh/ When I do document.getElementById('Boundless').children[0].children[2].children[0].children[1].children[0].innerHTML in the browser console, I successfully select the text I want, but when I run my js file in Terminal, I get the error: TypeError: 'null' is not an object (evaluating 'document.getElementById('Boundless').children') Why...

CasperJS' sendAJAX asynchronous flag set to true failed

ajax,casperjs
I set up a CasperJS script to call my web service (running on localhost:9000). The webservice containing parameter which is needed to be filled. Let's say an amount parameter, and my webservice saves amount data from its parameter. So I wrote my CasperJS script like this: casper.then(function(){ val = this.evaluate(function(){...

Click a button to download a file using CasperJS

javascript,unit-testing,phantomjs,casperjs
I want to write unit tests to download a file by clicking a form button using casperJS. I tried using document.getElementsByTagName to get to the button and clicked it in JavaScript's way. But I got an error message: TypeError: 'undefined' is not a function (evaluating 'inputs[1].click()') The segment of the...

Issues running casperjs with phantomjs#2.0.0 on Mac (Yosemite)

phantomjs,casperjs
I'm trying to run CasperJS having PhantomJS#2.0.0 on a Mac(Yosemite). When trying to run CasperJS I´m getting the following message: "CasperJS needs PhantomJS v1.x /usr/local/Cellar/casperjs/1.1-beta3/libexec/bin/bootstrap.js:91 in __die" How can I make it run? Has anyone got it working?...

How to print CasperJS debugging results to text file

javascript,python,debugging,phantomjs,casperjs
I can't read the debugging issues in the command prompt. Can I send that to a text file in my PhantomJS or CasperJS scripts? Here is my python code; import os import subprocess #proxies = {'https': 'http://wmhproxy2:8080'} APP_ROOT = os.path.dirname(os.path.realpath(__file__)) CASPER = "C:/casperjs/bin/casperjs" #SCRIPT = os.path.join(APP_ROOT,'unicorn.js') SCRIPT = os.path.join(APP_ROOT,'unicorn.js') params...

Get option values from drop down box with CasperJS

javascript,casperjs
Just start learning CasperJS, I know I can use the following to obtain the texts of the options: var casper = require('casper').create(); casper.start('http://www.example.com/test.aspx', function() { this.evaluate(function () { this.echo("Values: " + this.fetchText('#mySelect')); }); }); casper.run(function () { this.exit(); }); where mySelect is a <select> in the page. Sample HTML is...

Scraping an infinite scroll page stops without scrolling

javascript,phantomjs,casperjs
I am currently working with PhantomJS and CasperJS to scrape for links in a website. The site uses javascript to dynamically load results. The below snippet however is not getting me all the results the page contains. What I need is to scroll down to the bottom of the page,...