Why we switched from Yarn to pnpm

We’re really focused on developer productivity at TakeShape. We’re a small team with limited resources, so it’s worthwhile to spend time thinking about how we can work together faster and more efficiently. While refactoring our build process recently, we made a big decision: we were going to leave behind Yarn and switch to pnpm to manage our dependencies and run our scripts. This is the story of how we came to that decision and how it’s benefited us so far.

In the beginning, TakeShape’s codebase was split across several Git repos. Every package was developed in isolation, and they depended on each other. In theory, this was the ideal setup. In practice, we found that once everything was dependent, we really wanted to be able to test and release all the packages at the same time. We’d encounter failures when we’d issue a new release for one of our packages, but forget to update the version in our other projects that depended on it.

Ultimately, we realized that a monorepo was the right tradeoff when maintaining the separation and dependence of our projects. All our packages—like our web client, frontend routing library, and CLI—exist in one testable, deployable unit. And our packages can depend on each other using the link: syntax in package.json.

This mostly worked well, but we still found managing parts of our monorepo to be tedious. This is partly because each package in our monorepo manages its own dependencies with its own package.json and its own lock file. Even though every package used the same development toolchain of eslint, Jest, Typescript, and Babel, each package declared these devDependencies individually and this toolchain had to be kept up-to-date across all our packages. We decided against using Yarn’s workspaces feature to resolve this, since that would require ditching each package’s lock files in favor of a single, workspace-wide one.

Avoiding phantom dependencies was also trickier than it needed to be. Phantom dependencies happen when your code imports a package that’s not declared in your package.json. Say you add Package A to your project, which depends on Package B. Since Yarn keeps all packages at the root of node_modules, you’d be able to import and use Package B without it being in your package.json at all. While not too common, this is an error that can really slow down your debug process unless you remember to intentionally check for it.

Our monorepo also made our CI pipeline more convoluted than it needed to be. At first, we parallelized our CircleCI builds to speed up slow Webpack builds. But, as our monorepo grew so did the overhead of installing dependencies separately for each build. Installing dependencies for each build became a bottleneck. In response, we consolidated our build process to use less jobs with a CircleCI script we wrote and maintained ourselves. Ultimately, we ended up with a fragile set of CI scripts lint, test, and build changes to any package.

What we really needed was a way to recursively install our packages, hoist any shared dependencies, and run our scripts for linting, testing, and building. We knew that Yarn wasn’t scaling for us, so we started considering alternatives:

• What if we kept Yarn and added Lerna? Adding Lerna would solve some of our issues with CI scripting, but it wouldn't solve our issues with phantom dependency or duplicated devDependencies.
• What if we added Lerna and used Yarn Workspaces? Workspaces would solve the issue of sharing developer dependencies and automatically linking dependencies. But hoisting all node_modules to the project root just exacerbates the risk of phantom dependencies and causes issues with some modules and tooling.
• What if we upgraded to Yarn 2.0 and used…something else…with it? Yarn 2.0 is really exciting. It has a lot of cool features including Plug'n'Play (PnP). PnP would solve our issues with phantom dependencies, but it was potentially incompatible with certain dependencies that require file access. Yarn 2.0 is not compatible with Lerna; instead, it  has a plugin architecture. These plugins have the potential to solve our need for CI scripting, but they just aren’t mature enough to use confidently in production.
• What if we replaced Yarn with pnpm? According to pnpm, it exists to "[use] hard links and symlinks to save one version of a module only ever once on a disk." This approach to organizing node_modules prevents phantom dependencies and avoids other problematic issues when hoisting. It solves the same issues as Yarn 2.0’s PnP, but it has wider compatibility since it’s just using links. pnpm also includes similar filtering capabilities to Lerna.

In the end, pnpm made the most sense for us. We found that pnpm’s recursive command and --filter flag eliminated our need for a separate package like Lerna. Moving our shared dev dependencies, like Babel, ESLint, and Jest to our project’s top level works seamlessly, and now these packages can be updated from a single shared source. We even found and fixed several phantom dependencies in our project in the process of switching!

After adopting pnpm, our complicated CircleCI pipeline was reduced to a single job. As a result, we cut our overall and billable CircleCI minutes by roughly half! We think that we can get further benefits by tuning the settings, but it’s a pretty good start.

Of course, every decision comes with tradeoffs and pnpm is no different. While pnpm is actively maintained by zkochan, it’s a less popular project compared to Yarn or NPM. While PNPM is used by Microsoft, it doesn’t have the same level of direct corporate sponsorship that Yarn has from Facebook. And pnpm has its own lockfile format, so it’s not directly compatible with Yarn or NPM. It’s hard to know what the future holds, but if we ever need to switch away from pnpm it could be a bigger undertaking than we’d like.

And while it sounds silly, Yarn spoiled us by not requiring a “run” command for custom scripts, so we’ve had to retrain our muscle memory. It’s a minor tradeoff, but it does add to the cost of our team switching such a fundamental piece of our daily workflow.

But, at the end of the day, these costs were more than worth the improvements pnpm has contributed to our stack. Now we’re working faster, more efficiently, and with fewer trade offs than before.

pnpm may not be the right tool for every project or every stack, but if you’re encountering any of the same issues with your monorepo that we were, take a look and consider it as an alternative.

Are you interested in joining our growing team? We’re always looking to hear from talented folks interested in making tools that help web developers be more creative. See our open positions and apply!