How to login to Amazon using CasperJS – Working example
Intro
In this post I want to share with you a working example of “How to login to Amazon using CasperJS”. This post is part of post series about PhantomJS and CasperJS. If you are working with PhantomJS and you want to login to Amazon website, then read my post How to login to Amazon using PhantomJS. Also, if you want to know why CasperJS is better than PhantomJS, then read this post.
Working example and explanation
When you need to handle login system with CasperJS you have to think about the following things:
- Submit login form
- Store cookies received by server (this step is automatically performed by CasperJS)
- Resend cookies upon every new request (this step is automatically performed by CasperJS)
- Save cookies to file (optional)
Sometimes, login forms are loaded by AJAX requests, which makes your life much harder. In that case, you can use other available functions in CasperJS.
Available functions
wait() – Pause steps suite execution for a given amount of time, and optionally execute a step on done
waitFor() – Waits until a function returns true to process any next step
waitForAlert() – Waits until a JavaScript alert is triggered.
waitForPopup() – Waits for a popup having its url matching the provided pattern to be opened and loaded
waitForResource() – Wait until a resource that matches a resource matching constraints defined by testFx are satisfied to process a next step
waitForUrl() – Waits for the current page url to match the provided argument
waitForSelector() – Waits until an element matching the provided selector expression exists in remote DOM to process any next step
waitWhileSelector() – Waits until an element matching the provided selector expression does not exist in remote DOM to process a next step
waitForSelectorTextChange() – Waits until the text on an element matching the provided selector expression is changed to a different value before processing the next step
waitForText() – Waits until the passed text is present in the page contents before processing the immediate next step
waitUntilVisible() – Waits until an element matching the provided selector expression is visible in the remote DOM to process a next step
waitWhileVisible() – Waits until an element matching the provided selector expression is no longer visible in remote DOM to process a next step
Working example
The following CasperJS working example will login you to Amazon, and will take a screenshot when user is logged in:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
var casper = require('casper').create({ pageSettings: { loadImages: false,//The script is much faster when this field is set to false loadPlugins: false, userAgent: 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36' } }); //First step is to open Amazon casper.start().thenOpen("https://amazon.com", function() { console.log("Amazon website opened"); }); //Second step is to click to the Sign-in button casper.then(function(){ this.evaluate(function(){ document.getElementById("nav-tools").children[0].click(); }); }); //Now we have to populate username and password, and submit the form casper.then(function(){ console.log("Login using username and password"); this.evaluate(function(){ document.getElementById("ap_email").value="AMAZON USERNAME"; document.getElementById("ap_password").value="AMAZON PASSWORD"; document.getElementById("signInSubmit").click(); }); }); //Wait to be redirected to the Home page, and then make a screenshot casper.then(function(){ console.log("Make a screenshot and save it as AfterLogin.png"); this.capture('AfterLogin.png'); }); casper.run(); |
Cookie Management
Every time when your CasperJS script ends, all cookie information are deleted, and by next script execution you have to log-in again. If you repeat this process multiple times, you may end up with Captcha screen, or some other “Confirm you are human” page.
To avoid this, it would be good that you save cookies in file before you script ends. Next time when your script is executed, check if cookie file exists, and if yes, then load the cookies from the file, and you will be automatically logged in into amazon website.
I wrote JavaScript prototype for cookie management. You can copy and past this code at the top of your CasperJS script, or you can save it into separate file, and then load it from your script. I prefer to save it in another file, and then load it from several scripts where I need cookie management functionality.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
/** * This prototype is used to manage cookies * Author: Amir Duran */ var DCookieManagement = function(cookiesFileName){ this.fileManagement = require('fs'); this.phantomCookies=null;//Original cookies from PhantomJS this.cookiesFileName=cookiesFileName;//set cookies file name DCookieManagement.prototype.loadCookies = function (cookies){ this.phantomCookies = cookies; }; DCookieManagement.prototype.saveCookies = function(){ if(this.phantomCookies != null) this.fileManagement.write(this.cookiesFileName, JSON.stringify(this.phantomCookies), "w"); }; DCookieManagement.prototype.readCookies = function () { if(this.cookieFileExists()) this.loadCookies(JSON.parse(this.fileManagement.read(this.cookiesFileName))); }; DCookieManagement.prototype.cookieFileExists = function(){ return this.fileManagement.isFile(this.cookiesFileName); }; DCookieManagement.prototype.getCookies = function(){ return this.phantomCookies; }; DCookieManagement.prototype.removePreviousCookies = function(){ this.fileManagement.remove(this.cookiesFileName); }; }; exports.create = function(cookiesFileName){ return new DCookieManagement(cookiesFileName); }; |
Here is example of usage (code above is saved into separate file called DCookieManagement.js, that is loaded/imported from my CasperJS script)
|
1 2 3 4 5 6 7 8 |
//Name of the script where code above is saved DCookieManagement.js //Create cookie manager object. Cookies will be saved in file called liveCookies.txt var cookiesManager = require('./DCookieManagement').create("liveCookies.txt"); if(cookiesManager.cookieFileExists()){//Cookie file exists, try to read it cookiesManager.readCookies();//Read cookies from cookie file phantom.cookies = cookiesManager.phantomCookies;//Set phantom cookies } |
If you want to save cookies, then you just have to call the following line:
|
1 |
cookiesManager.saveCookies(); |
and your cookies will be saved into liveCookies.txt file.
Cookies are saved as JSON string, and you can check cookie values simply by opening the .txt file, or you can use cookies in other applications.
http://code-epicenter.com/how-to-login-to-amazon-using-casperjs-working-example/How to login to Amazon using CasperJS - Working examplehttp://code-epicenter.com/wp-content/uploads/2015/08/amazon.pnghttp://code-epicenter.com/wp-content/uploads/2015/08/amazon-150x150.pngJavaScriptLibrariesProgrammingTutorialsAmazon,CasperJS,Java Script,Web scraping
great tutorial! thanks a lot !!!
Hello! I am new in javascript and i have a question. I know it is a dumb question but i would like to know how to run this script, and what additional programs do I need to run. Thank you!
Hi,
here is an official page where you can check how to install CasperJS on your machine.
http://docs.casperjs.org/en/latest/installation.html
Good morning,
I’ve used your scrypt.
I’ve just added before : //First step is to open Amazon :
// print out all the messages in the headless browser context
casper.on(‘remote.message’, function(msg) {
this.echo(‘remote message caught: ‘ + msg);
});
// print out all the messages in the headless browser context
casper.on(“page.error”, function(msg, trace) {
this.echo(“Page Error: ” + msg, “ERROR”);
});
And I get theses messages after running :
C:\>casperjs sample.js (enter)
Amazon website opened
Page Error: TypeError: ‘null’ is not an object (evaluating ‘document.getElementById(“nav-tools”).children’)
Login using username and password
Page Error: TypeError: ‘null’ is not an object (evaluating ‘document.getElementById(“ap_email”).value=”xxx.xxx@gmail.com”‘)
Make a screenshot and save it as AfterLogin.png
Where should I find the .png file ? In C: ? I guesse it did not work for me.
Thanks a lot.
Matt
Hi Matt,
The .png image file is saved where casperjs.bat file is (if you are working on windows).
I checked the script one more time, and there are slight changes in a DOM elements of the Amazone website. I sent you customized script on email.
Regards, Amir!
Thank you so much for your email Amir—-
I had done three errors I Guess :
– First : is being on Windows 7 64, as of december 2015, I have to use CasperJS 1.1-beta3 in cunjunction with PhantomJS 1.9.2 (instead of 2 version), unless it is not working
– Second : I had put my *.js scrypt in C:/, and this folder will not allow the writing in it, so the screenshot is not working
– Third : I had to add the argument “–ssl-protocol=any” si it is :
casperjs –ssl-protocol=any js.js
And turn on verboose option so I could understand how to get further.
I’m very far from what I want to do, but I’m getting further…
Thank you so much for your email Amir—-
Your welcome, cheers 😉
Hi Amir,
Awesome tutorial!! it’s very clear and the code sample works very well.
Also, could you share how to use cookie in casperjs to avoid logon to amazon frequently?
Thanks!
I already have code somewhere, I will update my post today.
Thx for suggestion!
Hi,
Now you can find a part in blog post, that explains how to handle a cookies.
Hi Amir,
Thanks for the article.
I tried to run it but it return blank image AfterLogin.png.
Am I missing something?
-Pankaj
Hi Amir,
Very helpful article.
I have one problem…
I followed the instructions to save the cookie management code in a separate file and then require it in the main code file.
In my code I login to a form and after successful login call the saveCookies function but the cookie never gets saved because ‘this.phantomCookies’ always evaluates to null.
You said Store cookies received by server (this step is automatically performed by CasperJS)
So I guess I just have to call saveCookies to save the cookies to a file. But it does not work.
Am I missing something?
Thanks.
– Amar
Hi,
thx for your comment. CasperJS will always resend received cookies as long as casperJS is running. Once the application ends, cookies are deleted and you will have to get them again. This means for example if you login on any website, the website will return you few cookies which will be resent every time until your application stops. Once it stops, cookies are gone. But, before application ends, you can save cookies in a file, and then you can reload them next time when app starts. I hope this helps.