# CodeMod Tutorial - Part 3 - Paths, Moving Nodes

In [part 2](https://blog.royston.dev/codemod-tutorial-part-2-how-ckd7v8uyi00ovses15sv4hkmc) we walked through the process
of developing a codemod, and built one from scratch.  This time we're
going to build a more complex one, introduce the concept of *Path*s, node
factory functions, and look at why we should "use it in a sentence".

# Our Codemod

This time, we're going to write a codemod that rewrites array construction.

Let's say you've just inherited a large old JS codebase that contains
lots of code like this:

```js
var myArray = new Array(1, 2, 3);
var myOtherArray = new Array('x', 'y', 'z');
```

Note the explicit array constructor: `new Array()`.  That's generally a pretty unnecessary thing
to be doing (in those forms), and can actually be very confusing.  We're going to write a codemod that changes those array construction calls into plain array syntax

i.e. to convert the above into:

```js
var myArray = [1, 2, 3];
var myOtherArray = ['x', 'y', 'z'];
```

_Aside_

**Array construction**

A brief interlude for a quick pop quiz...

Do you know what `new Array(5, 'b')` does?
You probably know, or can guess, that it creates a two-element array containing `5` and `b`.  It's more simply written as `[5, 'b']`.

What about `new Array(5)`?  Some readers may be surprised to hear that it _doesn't_ create a one-element array; it creates a _five_-element array, initialized all as empty. Its equivalent is `[,,,,,]`.

There are many JS developers who don't know about that one-argument special case, and it's easy to
fall into it accidentally even if you do know (e.g. by removing the `b` element from the first example).

So, it's usually better to use the array syntax `[5, 6]` instead of explicit Array construction (`new Array(5, 6)`).  The codemod
we develop on this page will perform that change automatically across an entire codebase.


# Building the codemod

We'll work through [the development steps we listed last time](../codemod-tutorial-pt2/#writing-a-codemod).

## 1. Figure out what we want to change

We want to convert any calls that look like `new Array('a', 'b')` with or
without parameters, into plain array literal syntax `['a', 'b']`.
The array literal will contain the arguments from the `new Array` call
as its contents, even if they're complex expressions.

## 2. Figure out what we don't want to change

-   It's important that we don't modify any one-parameter `new Array(5)`
    calls.  `new Array(5)` is absolutely not replaceable by `[5]`.
-   We don't want to modify the construction of any other type, only `Array`

_Aside_

Actually, things are a little more nuanced than this.
The special case Array construction behaviour only happens when the
single parameter to `new Array()` is a number.

If we know that a single parameter isn't a number (e.g. `new Array('a string')`),
or if we had access to type information from, for instance, a TypeScript parser,
we _could_ be smarter, but we'll leave those as 'extra credit' exercises.


## 3. Construct a testbed

We'll develop the testbed shortly, but first, let's look at what happens if
our testbed is a little... lacking.

It's very easy, when first writing codemods, to rush into targeting a
very precise piece of syntax.  In our case, we want to target

```js
new Array('a')`
```

so let's look at the AST for exactly that.  It's:

![ASTDiagram2.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1596055678072/UDVg1kjVM.png)

It would be easy to conclude, from that code and diagram that we
should be looking for an `ExpressionStatement` containing a `NewExpression` with the appropriate `callee`.

Let's look at a very similar piece of code:

```js
const x = new Array(3, 4);
```

The AST for that is

![ASTDiagram1.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1596055619729/xhHJiZKUu.png)

Notice that there's no `ExpressionStatement` in that tree.

It turns out that the original `ExpressionStatement` was only there
because the `new Array` code was at the root of our source file. It's
actually only the `NewExpression` that we're interested in. The `ExpressionStatement`
was a red herring.

How do we avoid making this mistake?

### "Use it in a sentence"

Imagine you're in an elementary spelling bee competition and you're asked
to spell the word 'pair'. Of course, in such a competition you only
_hear_ the word spoken, so you're not sure whether they said 'pair' or 'pear' (or  ['pare' or 'pere'](https://www.homophone.com/h/pair-pare-pear-pere) )?

You can't ask them to spell the word to clarify, so what do you do? You ask the
caller to _use the word in a sentence_.
If they respond with "I'm going to _peer_ through the window.", you know you
completely mis-heard originally, but you now know what the word is.

Similarly, if we're trying to figure out the AST structure for
a piece of code, we can easily make bad assumptions if we only
look at one example with no context.

So, when collecting interesting cases to match, try to 'use them in a
sentence' to get a more realistic AST.  In fact, use them in
_multiple_ different ways that are close to how your actual code uses them.

We'll create a set of sample code that illustrates our positive and negative
cases and doesn't simply leave the `new Array` at the top of the source
file:

```js
// Empty array case
// -> const x = [];
const x = new Array();

// Complex arguments case
// -> somefunc(['a', 3 + 4]);
somefunc(new Array('a', 3 + 4));

// Single argument case; leave it alone
somefunc(new Array(5));

// Not Array case; leave it alone
somefunc(new NotArray('a', 5));
```

That includes things we _want_ to change and similar things
we _don't_ want to change. Good!

## 4. Build a target profile

Follow [this link to AST Explorer](https://astexplorer.net/#/gist/505a176a4ab8900fdefd5b103bd306d7/d4c3a7bdc55076f2c2a6f2226334a16fee27e79f) and it'll take you directly to a pre-configured AST Explorer with that code in it. 

The configuration is simply to do 'JavaScript parsing' and to use 'recast' with the 'esprima' parser, and be set up for JSCodeShift transformations.

It also includes the same skeleton codemod from last time ready for us to develop, namely:

```js
export default function transformer(file, api) {
  const j = api.jscodeshift;

  return j(file.source)
    .find(j.Identifier)
    .forEach(path => path.node.name = 'Foo')
    .toSource();
}
```

### Explore the AST

Click around the code and get a feel for the syntax tree.

You'll discover that we're looking for `NewExpression` nodes where
the `callee` is an `Identifier` with a `name` of `Array`:

![NewExpression.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1596055883590/29269rg4o.png)

## 5. Fire tracer bullets

Let's update our `.find()` call to target those nodes.

We'll also
need to tweak our `.forEach()`: we were previously targeting `Identifier`s
and could simply set their `name` property. We're now targeting
`NewExpression`s and they don't have `name` property; it's their `callee`
that does.

```js
export default function transformer(file, api) {
  const j = api.jscodeshift;
  return j(file.source)
    .find(j.NewExpression, { callee: { type: 'Identifier', name: 'Array' } })
    .forEach(path => path.node.callee.name = 'Foo')
    .toSource();
}
```

That's an excellent start. We're targeting all the `new Array` calls:

```js
// Empty array case
// -> const x = [];
const x = Foo;

// Complex arguments case
// -> somefunc(['a', 3 + 4]);
somefunc(Foo);

// Single argument case; leave it alone
somefunc(Foo);

// Not Array case; leave it alone
somefunc(new NotArray('a', 5));
```

We're still incorrectly targeting the one-argument `new Array(5)` call. Can we extend the `find` call to locate that?  Sadly not.
If we wanted to look for a `NewExpression` that had an `arguments` property with a `length` _equal to_ 1, we could do it, but we want only those with a length _not equal to_ 1.

It's time to introduce another JSCodeShift function that gives us more power: `.filter()`. But to make sense of that, we firstly need to introduce the concept of _Paths_.

## Paths

Let's take a look at a simple bit of code and its syntax tree:

```js
new Array(3, 'q')
```

![NodesTree.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1596055935898/ymrwvf-H_.png)

The `NewExpression` AST node has a `callee` property which points at an `Identifier` (with a `name` of `Array`). It also has an `arguments` property, but this doesn't point at an AST node object; it points at an _array_ object which contains some nodes.

When processing ASTs we often discover candidate target nodes and want to
'walk around the tree a bit' to confirm that we've found the correct pattern.
But navigating around trees like this can be very difficult:

-   There are no 'parent' pointers from a node to its parent.
-   For nodes such as the `NumericLiteral` and the `StringLiteral`, what _is_ their parent anyway? Is it the `NewExpression` or the containing array?
-   Even if there was a 'parent' pointer, we often want to know which property we would come through from parent to child (e.g. left or right side of a `+` operator?), and a simple 'parent' pointer wouldn't provide that.
-   It's tricky to inspect all of a node's child nodes (e.g. to recurse through a tree): you'd have to examine all the properties and deal with any arrays you came across.  Examining the properties safely (without tripping over internal implementation properties, potentially triggering infinite loops) requires knowledge of the properties available on each node type.

This is where _Paths_ come in. `Path` objects are a feature of the [`recast`](https://github.com/benjamn/recast) library, which JSCodeShift uses.

Recast builds a tree of _Path_ objects parallel to, and pointing at, the tree of AST nodes:

![PathsToNodes.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1596055969093/A9ggC8Sp_.png)

Amongst the properties on a Path object are:

-   `value`: the corresponding entry in the syntax tree. Note that Path objects don't always point at syntax \_Node_s. Path objects are also created for intermediate containers in the syntax tree, such as arrays, in which case the `value` property would point at the array.
-   `parentPath` (not shown on the above diagram, for simplicity): this
    enables us to walk _up_ the tree
-   `name`: the 'property' (or numeric index for array Path objects) that was traversed when moving from parent Path object to child Path object
-   `node`: the containing syntax tree node. That is, if a Path object's `value` property is pointing to an array for the `arguments` property of a syntax tree node, the `node` property will point at the syntax tree node itself

## Filtering

When we work with a chain of commands in JSCodeShift (e.g. `.find(j.Identifier).forEach(...)`), the chain
is actually operating on a collection of Path objects, not simply nodes.

In our particular case, we have found all the `NewExpression`
syntax tree nodes, and are manipulating the Path objects that
point at them.  We want to keep only those Path objects whose
`NewExpression`s have an argument count `!== 1`.

So we add a new `.filter` call as follows:

```js
export default function transformer(file, api) {
  const j = api.jscodeshift;

  return j(file.source)
    .find(j.NewExpression, { callee: { type: 'Identifier', name: 'Array' } })
    .filter(path => path.node.arguments.length !== 1)
    .forEach(path => path.node.callee.name = 'Foo')
    .toSource();
}
```

The result is

```js
// Convert this to
// const x = [];
const x = Foo;

// Leave this alone
somefunc(new Array(5));

// Convert this to
// somefunc(['a', 5]);
somefunc(Foo);

// Leave this alone
somefunc(new NotArray('a', 5));
```

The difference this time is that, as we'd hoped, the single-argument `new Array(5)` call is now left alone.

That's taken care of all the 'negative' cases that we don't want to change. We now need to replace the array with something more useful than merely 'Foo'.

## 6. Apply changes

Last time our codemod didn't _do_ very much: it just changed the
name of an existing `Identifier`.  This time, we want to replace,
completely, a `NewExpression` with an array.

Let's start by replacing them with an empty array, to keep things simple.

What does an array look like - what AST structure do we need to create?
We can repeat the same exercise
we went through to find the structure to _target_ (remembering to "use it
in a sentence").

We need to create an `ArrayExpression` node like this:

![ArrayExpression.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1596056002392/jiuMledbR.png)

### `.replaceWith()`

Thankfully, JSCodeShift provides a neat little function called `.replaceWith()`.
It can be called on a collection of Paths, and it uses the navigation
smarts that we outlined earlier to replace the Paths' nodes with ones
we specify.

We're going to start by using `.replaceWith()` to replace all our target
nodes with empty arrays.  But how do we actually create those nodes?

JSCodeShift provides us with a whole suite of AST node factory functions.

### AST Node Factories

By now, we're used to writing `j.Identifier` or `j.NewExpression` to indicate a type
of node to _search_ for.  JSCodeShift provides a parallel set of functions,
starting with lowercase letters, which _construct_ nodes.

e.g. `j.identifier('Foo')` will create an `Identifier` with the `name` property set to `'Foo'`.

In our case, we need to create an `ArrayExpression`, so we'll need to use `j.arrayExpression()`. Let's try it:

```js
export default function transformer(file, api) {
  const j = api.jscodeshift;

  return j(file.source)
    .find(j.NewExpression, { callee: { type: 'Identifier', name: 'Array' } })
    .filter(path => path.node.arguments.length !== 1)
    .replaceWith(path => j.arrayExpression())
    .toSource();
}
```

Our output?

```text
no value or default function given for field "elements" of ArrayExpression("elements": [Expression | SpreadElement | RestElement | null])
```

Uh oh. That's not good. We've obviously missed out a value for `elements`, whatever that is.  We can get a clue by looking at
the typing information in the `j.arrayExpression()` constructor:

![arrayExpressionCompletion.png](https://cdn.hashnode.com/res/hashnode/image/upload/v1596056037027/1bymioizJ.png)

It's expecting to be given an array of AST nodes for the array elements, but we didn't provide that.

Let's try again, providing an empty array of AST nodes:

```js
export default function transformer(file, api) {
  const j = api.jscodeshift;

  return j(file.source)
    .find(j.NewExpression, { callee: { type: 'Identifier', name: 'Array' } })
    .filter(path => path.node.arguments.length !== 1)
    .replaceWith(j.arrayExpression([]))
    .toSource();
}
```

The result is

```js
// Empty array case
// -> const x = [];
const x = [];

// Complex arguments case
// -> somefunc(['a', 3 + 4]);
somefunc([]);

// Single argument case; leave it alone
somefunc(new Array(5));

// Not Array case; leave it alone
somefunc(new NotArray('a', 5));
```

That's looking good. 3 of the 4 cases are now correct.  We just need to fix the case where there are
actually array parameters.

`.replaceWith(j.arrayExpression([]))` replaces _all_ of the found nodes with the same expression.
We want to examine each Path being replaced, and create an appropriate `ArrayExpression` for each
Path.

The good news is that `.replaceWith` also accepts a transformation function which is expected to transform
a Path object into a replacement expression.  We want to transform a `NewExpression` of a series
of array parameters into a literal array containing those same parameters.

### Moving nodes around

Of course, we don't want to replace `new Array(1, 2)` with `[]`.
We want to replace it with `[1, 2]`.  How do we construct
all the correct array parameters? They could be anything, not
just simple numbers.

Well, the good news is that we _already have them_.  The
`arguments` property to the `NewExpression` already contains
exactly the nodes that we need.

Our transformation ends up being surprisingly simple: instead of creating a new `ArrayExpression`
with an empty array of AST nodes, we reuse the AST nodes that were the `arguments` to the `NewExpression`:

```js
export default function transformer(file, api) {
  const j = api.jscodeshift;

  return j(file.source)
    .find(j.NewExpression, { callee: { type: 'Identifier', name: 'Array' } })
    .filter(path => path.node.arguments.length !== 1)
    .replaceWith(path => j.arrayExpression(path.node.arguments))
    .toSource();
}
```

The result is our final correct desired output:

    // Empty array case
    // -> const x = [];
    const x = [];

    // Complex arguments case
    // -> somefunc(['a', 3 + 4]);
    somefunc(['a', 3 + 4]);

    // Single argument case; leave it alone
    somefunc(new Array(5));

    // Not Array case; leave it alone
    somefunc(new NotArray('a', 5));

Hooray!

# Summary

We developed our understanding of codemods quite a bit this time. You should now have some familiarity with

-   the Path object tree
-   `.find()`ing and `.filter()`ing Path collections
-   creating replacement AST trees using AST node factory functions such as `j.identifier()`

We also discussed why it's a good idea to 'use it in a sentence' when trying to understand an AST tree.

Next time, we'll talk about `Scope`s.