Let's execute shell commands with Node.js

Native way

Let's begin with a basic solution. Node.js is very powerful, and gives us a good variety of tools to do this out of the box. And I am going to talk about exec and spawn methods from build-in module child_process, and execute shell commands in child processes.

const { exec, spawn } = require('child_process');

// or if you are on ESM already
import { exec, spawn } from 'child_process';

In most cases you would prefer to use spawn, because output data is streamed, not just passed as an argument in callback in exec use case. You can read in details about these commands in great book by Dr. Axel Rauschmayer. But there is one big problem with native solutions: you are going to work with event-, and callback-based solutions. You need to define event handlers for all use cases. Not so handy.

const { spawn } = require('child_process');
const command = spawn('cat', ['file.json']);

// console log will output log on new line 
command.stdout.on('data', (data) => {
	console.log(`${output}`.replace('\n', ''));
});

// handle errors during execution
command.stderr.on('data', (data) => {
	console.error(`${data}`.replace('\n', ''));
});

// handle end of execution
command.on('close', (code) => {
	console.log(`was closed with code ${code}`);
});

It does not look so sweet, especially when you have short of time. Of course, you can create your small, tiny library and reuse it between projects, but there are already several players on the table that can help you with that.

Library way

We have looked into opportunities that are given by Node.js from scratch. On the other hand, there are a bunch of libraries that can help you to have more elegant and easy to use solutions. These libraries act as bridges, allowing seamless communication between your Node.js code and the underlying command-line environment. Whether you're automating system tasks, interacting with external programs, or managing complex deployments, the right library can significantly streamline your development process and enhance the functionality.

And you can find some of them:

package	github stars*	downloads*
zx	37,551	299,024
execa	5,715	65,589,649
shellJS	13,838	8,384,179
* stats by Jun 4th 2023

These libraries give you as a developer a lot of freedom by hiding internal usage of native modules:

1. promised based APIs
2. usual usage of output values and errors handling
3. better errors messages
4. escaping arguments
5. handling force exit of the parent project for you
6. Increased buffer
7. and more other ...

zx

⚠️ ESM library (from v5)

zx is a library from Google, and it is quite fresh (from 2021), and without big audience. Nevertheless, zx is quite straightforward and does its job.

This library exports function $ for executing scripts, which you can configurate. For example, use custom method to execute scripts instead of spawn. In addition to a main function, zx also includes several nice functions to have better communication with the user, like spinner, question and retry. Those are quite common types of interactions.

Code example:

import { $ } from 'zx';

/* To prevent `zx` to prints all executed commands 
alongside with their outputs */
$.verbose = false;

try {
	const output = await $`cat file1.json`;
	// You can easily chain commands
	const outputWithReplacePipe = await $`cat file1.json`
		.pipe(
      		 $`sed -e "s/hello\ world/my\ name/g"`
	    );
} catch (p) {
	console.log('Can not open file');
	console.log(`Code: ${p.exitCode}`);
	console.log(`Error: ${p.stderr}`);
}

Example for CommonJS (Node.js >=14):

(async () => {
	const { $ } = await import('zx');
	try {
  		const output = await $`cat file1.json`; 
	} catch (err) {
    	console.log(`Code: ${err.exitCode}`);
    	console.log(`Error: ${err.stderr}`);
  	}
})();

execa

⚠️ ESM library (from v6)

It seems execa is the most popular library from our small list. Main script function is simple and similar to zx. On the other hand, as for me, execa is closer to low-level built-in methods, based on configuration that you can adjust. Anyway API still hides all internal logic, and it is nice to have control.

import { $ } from 'execa';

try {
	const output = await $`cat file1.json`;
	/*
	  If you want to use pipes -
	  you need to use `execa` directly;
	  `execa` syntax is close to built-in `spawn` method
	*/
	const outputWithReplaceExaca = await execa(
		'cat',
		['file1.json']
    ).pipeStdout(
		execa(
			'sed',
			['-e', 's/hello world/my name/g']
		)
	);
} catch (err) {
    console.log(`Code: ${err.exitCode}`);
    console.log(`Error: ${err.stderr}`);
}

Example for CommonJS (Node.js >=14):

(async () => {
  const { $ } = await import('execa');
  try {
    const output = await $`cat file1.json`;
    console.log(`Output: ${output.stdout}`);
  } catch (err) {
    console.log(`Code: ${err.exitCode}`);
    console.log(`Error: ${err.stderr}`);
  }
})();

shellJS

⚠️ CommonJS library

shellJS is the oldest from the list, and it was not got any updates for long period of time. Nevertheless, it still does a job. This library gives you a list of script commands which you can execute with your arguments with nicely opportunity to print output into file. Also you can pipe commands.

💡 shellJS does not extend methods from child_process module and it does not use terminal commands. Instead its methods are based mostly on fs module and just do the same things as terminal commands.

const shell = require('shelljs');
const output = shell.cat('file1.json');

if (output.code === 0) {
	// success
	console.log(output.toString());
	// or
	console.log(output.stdout);
} else {
	// error
	console.log(output.stderr);
}

const outputWithReplace = shell
	.cat('file1.json')
	// you can use chaining 😱 
	.sed(/hello world/g, 'hello json').toString();

Summary

Build-in features in Node.js provides developers with the freedom to build everything using low-level functions. It also reduces compatibility during migration to newer versions. And, anyway, understanding of basic is always good. However, when it comes to developer experience, the execution of terminal scripts in Node.js scripts, relying solely on these built-in features, may not be the most convenient option.

By leveraging the power of helper libraries, like execa or zx, you can focus on building the core functionalities of their applications without having to reinvent the wheel when it comes to executing terminal scripts.