How to Run Nightmare JS On Remote Linux Server

  Last Updated July 23, 2021  |  1 minutes  |  133 words  |  Talha Awan

Nightmare JS is a browser automation tool. It works perfectly on local machine and even has an option to view the browser in action by passing { show: true } option when initializing Nightmare. On remote linux machine, however, it will not work without installing and including an extra package xvfb. [Read More]
nodejs  nightmarejs  script 

How to Deal With Nightmare JS Zombie Electron Processes

Nightmare JS leaves in its wake zombie electron processes and memory leaks

  Last Updated July 10, 2021  |  3 minutes  |  551 words  |  Talha Awan

Nightmare JS is a great browser automation library, especially effective for scrapping websites. It has two faults though: memory leaks and zombie electron processes. But with a duct-tape hacks we can easily overcome both. For memory leak, which is minor but builds up over time, daily restart of the node script or app is pretty effective and avoids solutions that are overkill. [Read More]
nodejs  nightmarejs  script 

Restart Pm2 Process Every Day Using A Node Script

A brute force yet effective way to restart scripts that need it

  Last Updated July 23, 2021  |  3 minutes  |  624 words  |  Talha Awan

To run NodeJS scripts and apps constantly, we usually use pm2 (some prefer forever). However, some scripts require a brute force restart from time to time as they do not fare better in uninterrupted longer runs. Mostly, the reason is memory leaks of packages you are bound to use but have no control over. [Read More]
nodejs  pm2  script 

Trace Redirect Urls NPM Request

  Last Updated July 23, 2021  |  2 minutes  |  229 words  |  Talha Awan

With NPM request module it is quite easy to observe and trace all the intermediate urls before the final url is reached. We can do that using followRedirect option, which takes either true or false, or a function that should return a boolean value. [Read More]
nodejs  request  script 

Clean Up MongoDB Of Old Inactive Users And Their Data With Node Js Script

Reduce Expensive Database Space By Pruning Never-To-Be-Used-Again Data

  Last Updated July 10, 2021  |  3 minutes  |  509 words  |  Talha Awan

Databases are not cheap, especially when your application is just taking off and you are low on budget. It is then that freeing up retrievable and precious space from the database is a good way to save some dollars for a while. [Read More]
nodejs  mongodb  mongoose  script 

Build Queue For Processing Webhooks With Node Js And MongoDB

Manage The Flood Of Webhooks With A Database Queue

  Last Updated July 23, 2021  |  4 minutes  |  831 words  |  Talha Awan

Many of the third party services we integrate in our applications use webhooks to communicate events to us. Responding to the incoming webhook with success response (status 200) is all that is required to acknowledge the receipt of the webhook. To process that webhook immediately upon receiving or entering it in a queue to be processed later is up to you.

[Read More]
nodejs  mongodb  script 

Intercept Node JS Console Log and Insert in MongoDB

Easily Manageable and Searchable Application Logs Stored in Database.

  Last Updated September 8, 2022  |  4 minutes  |  838 words  |  Talha Awan

Your node js application is running on production, leaving hard-to-search ever-increasing log files in its wake. You want all your logs in a centralized, easily searchable place. For that, you have two options: 1 - Buy a subscription to some cloud log management service, such as papertrail. [Read More]
nodejs  mongodb  script 

Scrape Website Data Without Headless Browser, Using Node Js

Not Every Website Scraping Needs Headless Browsers; Request And Cheerio Js Do Just Fine

  Last Updated July 23, 2021  |  5 minutes  |  885 words  |  Talha Awan

During the course of web development, we occasionally need to scrape a website. Not just one page but many. On such requirement, mostly because the host site does not allow otherwise, we naturally tend to favor the use of headless browsers such as phantomjs or casperjs. [Read More]
nodejs  scraping  request  cheerio  script 

Export MongoDB Collection Data In CSV From Node Express Mongoose Application

  Last Updated August 4, 2021  |  2 minutes  |  371 words  |  Talha Awan

Suppose a student collection in mongoDB have some data that we want to export in a CSV file from a Mongoose, Node, and Express app. For simplicity, let us assume only two fields in Mongoose schema: lastName and firstName. To export a CSV file, install moment, json2csv, and mongo-date-query (optional) npm packages: [Read More]
nodejs  expressjs  mongodb  mongoose  csv 
When you purchase through links on techighness.com, I may earn an affiliate commission.