VM development progresses! Move node_modules wherever you like and use them!

Hello.
I'm Mandai, in charge of Wild on the development team.

Even if you don't develop systems with Node.js, I think it's common to use it to reduce your workload.
For example, you might use webpack to precompile CSS and transpile JS files, or you might use gulp to create scripts for routine tasks and run them regularly. There are many use cases, and I feel like it's steadily becoming a part of my life.

There is a package management tool for Node.js called npm, and this time we have summarized what to do when you need to move the node_modules directory outside the project directory for some reason

 

What is that certain reason?!

The reason is due to the development environment

I think VMs are commonly used as development environments, but in this case, the host OS was Windows 10, and CentOS 7 was running on Virtualbox (and managed with vagrant), which is a common configuration for a VM-based development environment.
However, there was an unexpected pitfall in this common development environment.

This may seem a bit sudden, but one of my priorities this year is to always use Visual Studio Code (hereafter referred to as VSCode) when developing, and it's already August, and I've become quite familiar with it.
VSCode is developed with Node.js (TypeScript) and is a multi-platform editor that uses Electron.

It has become so!

The shortcut keys for VSCode are quite difficult to use on the Linux version, so installing VSCode on a VM and developing with it is not an option, which is just frustrating (the Windows version's incredible ease of use is probably due to the fact that it was developed by Microsoft)

Therefore, the configuration of the development environment also needs to be tweaked with VSCode as the core

In conclusion, by sharing the entire project directory using the VirtualBox shared folder function, you can share the contents edited in VSCode on Windows with the VM. That's it.
If you start the VM in headless mode, the load can be reduced surprisingly, so you can run up to about three VMs.

 

Pitfall #1: The node_modules directory cannot be shared

Assuming you understand the situation, the big problem remains:
the node_modules directory cannot be shared.

Although the files can be shared, if you think about it carefully, node_modules is created for the platform at the time of npm install, which can cause problems with modules that contain platform-dependent binaries

This problem seems to be solved if you can place node_modules outside the project directory.
In exchange, you will need to run npm install for each development platform, but I don't mind it too much because I think that's what npm install is for.

 

Pitfall #2: "Maximum call stack size exceeded"

When I run npm install in a shared folder, I often encounter the error "Maximum call stack size exceeded."
I didn't really understand it, but it seems that the cause of this error is in the shared folder. References - npm ERR! code ETXTBSY · Issue #9979 · npm/npm · GitHub

So, this problem can be solved by placing node_modules outside the project directory

 

Install node_modules wherever you like

In fact, you can change the location of node_modules using the npm config command.
You can change it with the following command.

npm config set prefix "/home/vagrant"

 

Now the node_modules directory recognized by npm is "/home/vagrant/node_modules"

However, this only makes npm recognize the location of node_modules as "/home/vagrant/node_modules", so even if you run npm install in this state, node_modules will still be at the same level as package.json. That's annoying..

Therefore, you will need to specify the prefix in the installation command and install it using the following command

npm install --prefix "/home/vagrant"

 

If you do this, you'll probably get an error.
An error like this.

npm WARN saveError ENOENT: no such file or directory, open '/home/vagrant/package.json'

 

There is no package.json! But there is!
Then I thought, "Package.json is required in the directory specified by --prefix!"

Hard copies are difficult to manage, so we will use symbolic links

cd /home/vagrant ln -s /home/vagrant/src/package.json ln -s /home/vagrant/src/package-lock.json # sometime

 

It's fine if package-lock.json exists, but in the case of the first installation, the original will be in "/home/vagrant", so after the installation is complete, you will have to do some mysterious work like copying it to the working directory.
This is also because of VSCode. It can't be helped.

Then just run npm install with the prefix to proceed with the installation

As an aside, I thought npm install was really slow, but when I ran it outside of a shared folder, it finished in an instant.
Shared folders are convenient, but they're slow.

 

Sometimes it's not okay to go anywhere

Although I said it can be installed anywhere, it's not the case that anywhere is fine.
Below is what happens when I install node_modules in a certain directory and run the task runner gulp.

gulp sass [06:46:05] Local gulp not found in ~/src [06:46:05] Try running: npm install gulp

 

I'm sure I have gulp installed both globally and locally, so I think the problem is that node_modules isn't in the same directory as package.json.
A bit of Googling didn't help, so I looked at the gulp source and it gave me some insight. → gulp-cli/index.js at master · gulpjs/gulp-cli · GitHub

The only strings in the cfgLoadOrder array are home and cwd, so it seems like it would be fine to move node_modules directly under the home directory.
That's why the command prefix in the sample is the home directory.

If you create node_modules under your home directory, you can run gulp tasks successfully even if node_modules is not in the usual location

I have a feeling that writing a configuration file would give me more freedom, but I'm not sure about that, so I'll look into it when I have time

 

summary

I've gone off on a tangent a lot and the article has become quite disorganized, so I'll summarize it here

• In some cases, the directory where npm install is installed cannot be shared in the VM's shared folder
  • If you are using a module that requires platform-dependent binaries, it will not work
  • Due to the nature of shared folders, even npm install cannot be done on shared folders
  • If the host and guest are *nix based, maybe it's OK?
• The location where node_modules is created can be changed with the npm install command option "--prefix"
• The directory specified with "--prefix" must already contain package.json (a symbolic link is also OK)
• Sometimes, such as with gulp, the location of node_modules cannot be freely determined
• Node.js is useful

In the end, I think it's best to put it directly under my home directory.
I use a different VM for each project, and even if I can manage it with a config file, I don't think it's right to set it in gitignore just for myself.

I was glad that node_modules was separated from the working directory

 
That's it.