What they don't tell you in the docs: CasperJS
This is the first post of my "What they don't tell you in the docs" series. In it and all future posts with that title I will explain things that are not told in the docs, but are important. Just so you can save your time and don't irritate stackoverflow with your questions.
Introductions aside. Today's guest: CasperJS
CasperJS is a navigation scripting utility for PhantomJS and SlimerJS. Essentially it's your headless js browser, but nicer.
Despite having a great documentation Casper has some untold secrets.
I started working with Casper without touching Phantom and did not know how to find out if Casper even manages cookies. It took me some time to find out if Casper uses Phantom's features. Casper in fact does manage cookies in background, so no worries about it.
Have some pretty things to help you.
casper.echo(document.cookie); //prints the cookie to console
casper.page.setCookies(''); //clears the cookies
Should have probably made it the first thing. A very common question.
We have two ways to add jQuery and use it. Docs explain it, but many people still encounter problems.
var casper = require('casper').create({
clientScripts: [
'includes/jquery-latest.min.js']})
this.page.injectJs('/jquery-latest.min.js'); //you have to do this on a specific page, while the first way works to each page
Note: If you can't use $() in your nagivation script that is because you need to use jQuery in context of the page you are navigating. To do this you use casper.evaluate(function(){ $() })
For example to click a button with class ".button"
casper.thenEvaluate(function(){ $('.button').click() })
You want your little casper to be super fast? Well then you have to disable the styles. This is very handy when you need to navigate a lot of pages. In order to disable styles we need to abort any attempts to send a .css file towards our casper. This is how we do it:
casper.options.onResourceRequested = function(C, requestData, request) {
if ((/http:\/\/.+?.css/gi).test(requestData['url']) || requestData['Content-Type'] == 'text/css') {
//console.log('Skipping CSS file: ' + requestData['url']);
request.abort();
}
}
Easy as that. This can improve the speed of your navigation twice or even more, depending on the styling.
Last, but not the least: mysterious conditional statement behaviour.
Casper is sure myserious like any ghost should be, at times.
Let's play a game. Guess what page will casper visit last if I run this
casper.start('www.google.com');
if (false){
casper.then(function(){ this.open('www.bing.com');})
}
casper.run();
If you said Bing then you just won a whole lot of nothing. But what happens? Isn't the block inside a false conditional supposed to be ignored?
Yes it is. But ghosts are not to be measured with mortal ways. Casper does add this step to its' navigation stack despite it being enclosed by a conditional statement.
What is it? A question from the audience? Shoot!
"How can I make a step only run if a conditional statement is true?"
Good question. Here is how:
casper.start('www.google.com', function(){
if(true){this.bypass(1)} });
casper.then(function(){ this.open('www.bing.com');})
casper.run();
casper.bypass() skips a number of steps, 1 in the case. This way we can manipulate the navigation stack.
