Skip to content
This repository has been archived by the owner on May 22, 2024. It is now read-only.

feat: use esbuild by default for es modules, if detected #625

Merged
merged 15 commits into from
Sep 14, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
11 changes: 11 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@
"del": "^6.0.0",
"elf-cam": "^0.1.1",
"end-of-stream": "^1.4.4",
"es-module-lexer": "^0.7.1",
ehmicky marked this conversation as resolved.
Show resolved Hide resolved
"execa": "^5.0.0",
"filter-obj": "^2.0.1",
"find-up": "^5.0.0",
Expand Down
22 changes: 22 additions & 0 deletions src/runtimes/node/detect_es_module.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
const fs = require('fs')

const { init, parse } = require('es-module-lexer')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have a JavaScript parser (acorn) as a dependency, which should be able to tell us whether a file uses ESM or not. If it is, I would try to use it before adding a new dependency.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure sure, I'll take a look 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering about the performance difference there considering:

  • acorn is implemented in JavaScript while es-module-lexer is in WebAssembly
  • acorn does lexing+parsing while es-module-lexer does lexing only
  • acorn parses all possible JavaScript syntax while es-module-lexer only looks for imports/exports

The author of es-module-lexer actually claims the following:

For an example of the performance, Angular 1 (720KiB) is fully parsed in 5ms, in comparison to the fastest JS parser, Acorn which takes over 100ms.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I second @ehmicky 's evaluation - we need a parser and that's just not what es-module-lexer provides.

that being said, es-module-lexer does have a very light footprint... if we want, I can look into how to accomplish this with acorn, but we can't replace acorn with this

zip-it-and-ship-it on  cg2/617/DetectEsModules [$] is 📦 v4.20.0 via  v16.7.0 
❯ du -ha node_modules/es-module-lexer
4.0K    node_modules/es-module-lexer/types/lexer.d.ts
4.0K    node_modules/es-module-lexer/types
4.0K    node_modules/es-module-lexer/LICENSE
4.0K    node_modules/es-module-lexer/CHANGELOG.md
 12K    node_modules/es-module-lexer/dist/lexer.cjs
 12K    node_modules/es-module-lexer/dist/lexer.js
 24K    node_modules/es-module-lexer/dist
8.0K    node_modules/es-module-lexer/README.md
4.0K    node_modules/es-module-lexer/package.json
 48K    node_modules/es-module-lexer


const detectEsModule = async ({ srcFile }) => {
// console.log('srcFile: ', srcFile)
charliegroll marked this conversation as resolved.
Show resolved Hide resolved
if (!srcFile) {
return false
}

await init

// Would love to just use await fs.promises.readFile, but it's not available in our version of Node.
charliegroll marked this conversation as resolved.
Show resolved Hide resolved
charliegroll marked this conversation as resolved.
Show resolved Hide resolved
// eslint-disable-next-line node/no-sync
const srcFileContents = fs.readFileSync(srcFile, { encoding: 'utf8' })
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could probably use some try/catch safety with a default to false to safely maintain the status quo


const [imports, exports] = parse(srcFileContents)

return imports.length !== 0 && exports.length !== 0
charliegroll marked this conversation as resolved.
Show resolved Hide resolved
}

module.exports = { detectEsModule }
20 changes: 18 additions & 2 deletions src/runtimes/node/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,29 @@ const cpFile = require('cp-file')

const { JS_BUNDLER_ESBUILD, JS_BUNDLER_ESBUILD_ZISI, JS_BUNDLER_ZISI, RUNTIME_JS } = require('../../utils/consts')

const { detectEsModule } = require('./detect_es_module')
const { findFunctionsInPaths } = require('./finder')
const { getSrcFiles } = require('./src_files')
const { zipEsbuild } = require('./zip_esbuild')
const { zipZisi } = require('./zip_zisi')

// We use ZISI as the default bundler until the next major release, with the
// exception of TypeScript files, for which the only option is esbuild.
const getDefaultBundler = ({ extension }) => (extension === '.ts' ? JS_BUNDLER_ESBUILD : JS_BUNDLER_ZISI)
const getDefaultBundler = ({ extension, srcFile }) => {
if (extension === '.ts') {
return JS_BUNDLER_ESBUILD
}

if (srcFile) {
charliegroll marked this conversation as resolved.
Show resolved Hide resolved
const isEsModule = detectEsModule({ srcFile })

if (isEsModule) {
return JS_BUNDLER_ESBUILD
}
}

return JS_BUNDLER_ZISI
}

// A proxy for the `getSrcFiles` function which adds a default `bundler` using
// the `getDefaultBundler` function.
Expand All @@ -35,7 +50,8 @@ const zipFunction = async function ({
srcPath,
stat,
}) {
const bundler = config.nodeBundler || getDefaultBundler({ extension })
// how do you construct the srcFile?
charliegroll marked this conversation as resolved.
Show resolved Hide resolved
const bundler = config.nodeBundler || getDefaultBundler({ extension, srcFile: join(srcPath, filename) })
charliegroll marked this conversation as resolved.
Show resolved Hide resolved

// If the file is a zip, we assume the function is bundled and ready to go.
// We simply copy it to the destination path with no further processing.
Expand Down
4 changes: 4 additions & 0 deletions tests/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,10 @@ testBundlers('Can require local files', [ESBUILD, ESBUILD_ZISI, DEFAULT], async
await zipNode(t, 'local-require', { opts: { config: { '*': { nodeBundler: bundler } } } })
})

testBundlers('Can require local esm files', [ESBUILD, ESBUILD_ZISI, DEFAULT], async (bundler, t) => {
charliegroll marked this conversation as resolved.
Show resolved Hide resolved
await zipNode(t, 'local-require-esm', { opts: { config: { '*': { nodeBundler: bundler } } } })
})

testBundlers('Can require local files deeply', [ESBUILD, ESBUILD_ZISI, DEFAULT], async (bundler, t) => {
await zipNode(t, 'local-deep-require', { opts: { config: { '*': { nodeBundler: bundler } } } })
})
Expand Down