Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the&nbsp;DevTools Protocol. Puppeteer runs&nbsp;headless&nbsp;by default, but can be configured to run full (non-headless) Chrome or Chromium. Learn more about puppeteer <a href="https://github.com/puppeteer/puppeteer/blob/main/CONTRIBUTING.md" rel="noopener noreferrer" target="_blank">here</a>.In this article, we will write a script that will automatically logs you in from the google oauth form and scrape the cookie data from the site. Below are the use cases of it:<ol><li>Test automation in modern web applications: verifying that the features we are exposing our users/customers to are actually behaving as expected.</li><li>Taking screenshots of web pages: useful for a variety of different uses going from simple archiving to automated comparison for e.g. visual testing.</li><li>Scraping web sites for data: extracting data from websites for later retrieval or analysis.</li><li>Automating interaction of web pages: speed up and scale any sort of sequence of actions we would like to perform on a website automatically.</li></ol> Let's scrape a site,shall we? Create a Project Folder and structure it as below:<pre class="ql-syntax" spellcheck="false">root-------
				----- script.js
			 ----- cookies.json
				----- config.json
</pre> In cookie,json file.Add a { } as below:<pre class="ql-syntax" spellcheck="false">{ }
</pre>In config.json. Add mail and password as below:<pre class="ql-syntax" spellcheck="false">{
"mail":"your email",
"password":"your password"
}
</pre> Now lets install puppeteer:<pre class="ql-syntax" spellcheck="false">
npm i puppeteer
</pre> Start writing the script in script.js file.copy the below code: <pre class="ql-syntax" spellcheck="false">const&nbsp;puppeteer&nbsp;=&nbsp;require("puppeteer");
const&nbsp;fs&nbsp;=&nbsp;require("fs");
const&nbsp;cookies&nbsp;=&nbsp;require("./cookies.json");
const&nbsp;config&nbsp;=&nbsp;require("./config.json");


(async&nbsp;()&nbsp;=&gt;&nbsp;{
&nbsp;&nbsp;try&nbsp;{
&nbsp;&nbsp;&nbsp;&nbsp;//*&nbsp;start&nbsp;puppeteer&nbsp;and&nbsp;creates&nbsp;a&nbsp;new&nbsp;page
&nbsp;&nbsp;&nbsp;&nbsp;let&nbsp;browser&nbsp;=&nbsp;await&nbsp;puppeteer.launch({&nbsp;headless:&nbsp;false&nbsp;});
&nbsp;&nbsp;&nbsp;&nbsp;let&nbsp;page&nbsp;=&nbsp;await&nbsp;browser.newPage();


&nbsp;&nbsp;&nbsp;&nbsp;//*&nbsp;check&nbsp;for&nbsp;any&nbsp;saved&nbsp;session
&nbsp;&nbsp;&nbsp;&nbsp;if&nbsp;(Object.keys(cookies).length)&nbsp;{
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;//*&nbsp;set&nbsp;the&nbsp;saved&nbsp;cookies&nbsp;in&nbsp;the&nbsp;page
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.setCookie(...cookies);


&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;//*&nbsp;go&nbsp;to&nbsp;sentinal
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.goto("https://sentinel.zerodha.com/",&nbsp;{
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;waitUntil:&nbsp;"networkidle2",
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;});
&nbsp;&nbsp;&nbsp;&nbsp;}&nbsp;else&nbsp;{
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.goto("https://sentinel.zerodha.com/login",&nbsp;{
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;waitUntil:&nbsp;"networkidle0",
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;});


&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;//*&nbsp;brings&nbsp;login&nbsp;page
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.click('a[href="/api/accounts/google/login/"]',&nbsp;{&nbsp;delay:&nbsp;30&nbsp;});


&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;//*&nbsp;adds&nbsp;email
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.waitForSelector('input[id="identifierId"]');
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.type('input[id="identifierId"]',&nbsp;config.mail,&nbsp;{
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;delay:&nbsp;30,
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;});


&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.click("#identifierNext");


&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;//*&nbsp;adds&nbsp;password
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.waitForSelector('input[type="password"]',&nbsp;{&nbsp;visible:&nbsp;true&nbsp;});
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.type('input[type="password"]',&nbsp;config.password);


&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.waitForSelector("#passwordNext",&nbsp;{&nbsp;visible:&nbsp;true&nbsp;});
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;page.click("#passwordNext");


&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;//*&nbsp;get&nbsp;the&nbsp;current&nbsp;browser&nbsp;page&nbsp;session
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;let&nbsp;currentCookies&nbsp;=&nbsp;await&nbsp;page._client.send("Network.getAllCookies");


&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;//*&nbsp;creates&nbsp;a&nbsp;cookie&nbsp;file&nbsp;to&nbsp;store&nbsp;the&nbsp;session
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;fs.writeFileSync("./cookies.json",&nbsp;JSON.stringify(currentCookies));


&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;await&nbsp;browser.close();
&nbsp;&nbsp;&nbsp;&nbsp;}
&nbsp;&nbsp;}&nbsp;catch&nbsp;(e)&nbsp;{
&nbsp;&nbsp;&nbsp;&nbsp;console.log(e);
&nbsp;&nbsp;}
})();
</pre> To run the script:<pre class="ql-syntax" spellcheck="false">node script.js
</pre>Check the cookie.json file. The file gets the cookie from the site. There are more stuffs that you can do.Hope this helps. Happy coding...

How to extract Cookie from a site with Puppeteer by logging in?

Related blogs

About us: