[rush] Deploy problems with Prisma as dependency

[Faithfinder]

Summary

I can't deploy an app that depends on Prisma because prisma client doesn't get copied over

Repro steps

• Clone https://github.com/Faithfinder/rush-reproduce
• rush update
• rush build
• rush deploy

Look into common/temp/node_modules/.pnpm/@prisma+client@2.11.0_@prisma+cli@2.11.0/node_modules/, see .prisma folder

Look into common/deploy/common/temp/node_modules/.pnpm/@prisma+client@2.11.0_@prisma+cli@2.11.0/node_modules/, don't see .prisma folder

Details

The problem is basically that Prisma guys are being smarta... guys. The prisma client is being generated in a postinstall hook of the package (or with npx prisma generate, whatever), however it's generated outside of the main package folder. So, when deploy happens, the hook doesn't run, obviously - files are just copied over, but, .prisma doesn't get copied over as well, because it's neither a dependency from anything's package.json, nor is it in a folder of any dependency.

Standard questions

Please answer these questions to help us investigate your issue more quickly:

Question	Answer
@microsoft/rush globally installed version?	5.46.1
rushVersion from rush.json?	5.46.1
useWorkspaces from rush.json?	yes (doesn't seem to matter)
Operating system?	WSL
Would you consider contributing a PR?	No
Node.js version (node -v)?	14.15.0

[apostolisms]

This sounds like an issue with Prisma. Can you open an issue there to fix this behavior?

[Faithfinder]

On their side it's not an issue, it's a "policy". https://www.prisma.io/docs/concepts/components/prisma-client/working-with-prismaclient/generating-prisma-client#why-is-prisma-client-generated-into-node_modulesprismaclient-by-default

If you can link to any rules they're breaking by doing that I can try my luck there, I guess, but I doubt anything will come out of it.

I've found my workaround by regenerating the client again after I do the deploy, but I thought this might be worth a discussion anyway. (since it feels like things shouldn't be touched after the deploy, if possible).

[clouedoc]

This issue also arises when developing.

node_modules/@organization/database/node_modules does not contains .prisma/client which causes downstream package compilation to fail.

Manual fix: bash -c 'cd node_modules/@organization/database && pnpm prisma generate'

It would be nice to have a way to execute a command after a rush update (which removes .prisma/client).

[Yaojian]

May be you can generate .prisma to a custom directory : https://www.prisma.io/docs/concepts/components/prisma-client/working-with-prismaclient/generating-prisma-client#using-a-custom-output-path

[Faithfinder]

Unfortunately even if you generate the main client in a custom directory, it still creates a stub in the usual place

[arunmenon1975]

I also had to restart my IDE (VS Code) after prisma generate in order for my shared lib package builder (microbundle) to pick up the client. The build wouldn't go through without restarting - kept complaining 'could not find prisma client' and pointing to import { PrismaClient } from '@prisma/client'. After a restart the build goes through fine. i think rush update and similar commands shouldn't be issued before the build. Not sure if this was because i was using the integrated terminal in VS Code.

[jacobclarke92]

I had this same issue. Found a solution but it's ugly and am looking for a proper solution answer.
Weird thing is I'm working with a monorepo that I previously had set up with turbo and it was working fine, so must be some magic that allowed @prisma/client to be resolved properly when using default pnpm library in turbo.

Anyway my current 'solution' is to find where the node resolver locates @prisma/client within rush's pnpm library and generate my schema there as well as in the default location.

generator client {
  provider        = "prisma-client-js"
  output          = "../node_modules/.prisma/client"
  // output          = "../../../common/temp/node_modules/.pnpm/@prisma+client@4.9.0_prisma@4.9.0/node_modules/.prisma/client"
  // output          = "../../../common/temp/node_modules/.pnpm/@prisma+client@4.9.0/node_modules/.prisma/client"
}

So just rerunning prisma generate between commenting out those different output lines.
This is obviously fragile and awful but it's the only way I've gotten it to work.

Some extra context in case it helps:
I have a bunch of packages individually bundled with rollup that list @prisma/client as a peer (external) dep.
My main app is a NextJS app importing those packages and my issues comes not from the main NextJS app not being able to resolve @prisma/client but the packages not being able to. (Even after trying all sorts of webpack config alias overrides and whatnot).

I suspect this isn't necessarily a rush issue but I could be wrong and have no idea whether the solution would reside with rush, prisma or next :')

[Faithfinder]

Well, what we ended up doing, is treating Prisma schema as source, output to custom dir and re-export the result as a separate package (@companyName/prisma-client) (running prisma generate as the build command)

At the same time we got rid of dependencies that tried to work with Prisma by doing require('@prisma/client')

[jacobclarke92]

Interesting approach... can see how that would work.

My setup is kind of similar but methodology is different.
I have a package called @company/core-schema which doesn't export prisma client but contains the prisma schema file, along with some generated runtime types courtesy of @paljs/cli.
The idea being for apps to use this packages and be able to extend the core schema (e.g. via prismix) if needed.
The interesting part is that I can do this as many times as I want with different packages as different 'features' (and their corresponding db schemas) to combine a custom featureset of schemas for each individual app.

So you can understand why exporting the generated prisma client in a package doesn't make sense from my use case, but if its purpose was to be a reusable client that doesn't need to change then your appoach makes perfect sense, and it's good to know that it's a workable option!

If you learn anything new please keep us in the loop, and I'll do the same.

[jacobclarke92]

One last thing to note -- this might have already been touched on, but I can confirm that this is only a problem when using pnpm/rush.
I've privately published all my packages from the aforementioned monorepo setup and spun up a stock standard npm (no workspaces) project for a standalone app and had it work without a hitch.

[Cmoen11]

What is the status of this? Will this be prioritized ?

[octogonz]

Over the years we have struggled with many different NPM packages that want a place to cache data, and decide that node_modules seems like a great place for temporary storage because it's fairly standard and does get occasionally deleted. Although a simpleminded and brittle practice, it's understandable why they would choose this, given the lack of standardization for Node.js development. I'll also mention that "postinstall" lifecycle scripts generally also seem to cause endless trouble for large scale development. Package installation is an already complex and operation with stringent performance/stability requirements, thus not a good stage to be trying to do anything other than unpacking tarballs. But again, the motivation is the same -- lack of standardization means no straightforward way to perform those tasks at a later stage. To perform the operations at a later stage, NPM package maintainers would need their consumers to have some kind of build process and know how to add a step, which is easy for large codebases, but painful for the hobbyists and small projects that are so attractive for OSS adoption.

What is the status of this? Will this be prioritized ?

@Cmoen11 from this GitHub issue, it's unclear what is being asked of Rush.

One idea would be for rush deploy to provide a way to include the node_module/.prisma folder in the deployment. I had assumed that folderToCopy already provides this capability. But actually it copies into the deployment root folder. So maybe we need something like:

deploy.json

/**
  * Specifies a list of additional folder paths that will be included in the "rush deploy" output folder.
  */
"additionalFoldersToInclude": [
  // How is this maintainable? The folder path can change with any "rush update"
  "common/temp/node_modules/.pnpm/@prisma+client@2.11.0_@prisma+cli@2.11.0/node_modules/.prisma"
]

I think a better solution is to redirect Prisma's output into a more manageable location.

On their side it's not an issue, it's a "policy". https://www.prisma.io/docs/concepts/components/prisma-client/working-with-prismaclient/generating-prisma-client#why-is-prisma-client-generated-into-node_modulesprismaclient-by-default

If you can link to any rules they're breaking by doing that I can try my luck there, I guess, but I doubt anything will come out of it.

The "policy" that linked from the Prisma documentation ("Keeping the query engine out of version control by default") is really describing a requirement, not an implementation. They also call it a "default" suggesting that they are open to other solutions. I think the ideal would be something like:

1. Ask for an environment variable that will disable Prisma's postinstall script entirely. Rush can force this setting for all engineers using the environmentVariables setting from pnpm-config.json.
2. Create a proper build step that generates the Prisma client for your monorepo. For example, it could be a Rush workspace project that is a devDependency of any projects that need Prisma. And the "build" script for this project would prepare the output in a professional way that can be cached by the Rush build cache.

[Cmoen11]

Thank you @octogonz for the great answer that gives us an well written explanation.

I think a way to flag certain folders as a cache-folder-thing would be a great addition, as in my case, if i ran rush update in any of the different project within the mono repo it would wipe the prisma cache.. so a way to keep this would be awesome.

I read this answer: netlify/zip-it-and-ship-it#636 (comment)
I haven't been able to test it yet, but it might be a solution for this problem to include @prisma/engines-version along with @prisma/client in package.json.

I'll test this and give a feedback.

thank you again for the quick response!

[Faithfinder]

To be fair, in 3 years I ran monorepo at that company, Prisma was the only project needing special treatment for deploy (but we did find a good way, already described above)