Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using NodeJS builtins #439

Open
ctjlewis opened this issue Jan 2, 2021 · 7 comments
Open

Using NodeJS builtins #439

ctjlewis opened this issue Jan 2, 2021 · 7 comments
Labels

Comments

@ctjlewis
Copy link

ctjlewis commented Jan 2, 2021

I was extremely pleased to see my TS compile to perfect -O ADVANCED output after adding the tsconfig.json, but when I add a simple builtin import:

import { exec } from 'child_process';

I receive the following error:

TSCC: tsickle converts TypeScript modules to Closure modules via CommonJS internally."module" flag is overridden to "commonjs".
TSCC: tsickle uses a custom tslib optimized for closure compiler. importHelpers flag is set.
TSCC: Module name child_process was not provided as a closure compilation source
TSCC: The compilation has terminated with an error.

This is one of the larger pains with Closure Compiler in general - Node builtins should be automatically detected (opt-out with a flag) and supplied as externs, and left untouched in output. TSCC does a fantastic job of handling thousands of other problems, but if we can't compile programs containing Node builtins, we can't realize the full benefit.

Any thoughts on this or recommendations? The test program is below.

import path from 'path';
console.log(path.resolve('.'));
@theseanl
Copy link
Owner

theseanl commented Jan 2, 2021

I suppose you can do something like this:

{
	"external": {
		"path": "path"
	},
	"compilerFlags": {
		"output_wrapper": "(function(path){%output%})(require('path'))"
	}
}

@ctjlewis
Copy link
Author

ctjlewis commented Jan 2, 2021

Thanks so much, that's very close to a similar workaround I saw recently. This will be incredibly powerful if I can get it to work, but there are some snags here. With the simplest possible configuration:

tsconfig.json

{
    "compilerOptions": {
        "strict": true,
        "esModuleInterop": true
    }
}

tscc.spec.json

{
    "modules": {
        "bundle": "test.ts"
    },
    "prefix": {
        "rollup": "dev/",
        "cc": "dist/"
    },
    "compilerFlags": {
        "process_common_js_modules": true,
        "module_resolution": "NODE",
        "output_wrapper": "(function(path){%output%})(require('path'))"
    },
    "external": {
        "path": "path"
    }
}

We get:

TSCC: tsickle converts TypeScript modules to Closure modules via CommonJS internally."module" flag is overridden to "commonjs".
TSCC: tsickle uses a custom tslib optimized for closure compiler. importHelpers flag is set.
ClosureCompiler: test.js.tsickle:10:37: ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "path"
  10| var path_1 = tslib_1.__importDefault(require("path"));
                                           ^

1 error(s), 0 warning(s)


✖ Closure compiler error
TSCC: Closure compiler has exited with code 1
TSCC: The compilation has terminated with an error.

With my nontrivial TSCC config, the error disappears due to --dependency_mode PRUNE_LEGACY:

{
    "modules": {
        "bundle": "test.ts"
    },
    "prefix": {
        "rollup": "dev/",
        "cc": "dist/"
    },
    "compilerFlags": {
        "process_common_js_modules": true,
        "module_resolution": "NODE",
        "output_wrapper": "(function(path){%output%})(require('path'))",
        "dependency_mode": "PRUNE_LEGACY",
        "strict_mode_input": true,
        "assume_function_wrapper": true,
        "compilation_level": "ADVANCED",
        "language_in": "ES_NEXT",
        "language_out": "ECMASCRIPT5_STRICT"
    },
    "external": {
        "path": "path"
    }
}

But the output AST is empty:

dist/bundle.js

(function(path){'use strict';})(require('path'))

If we can find a way to achieve this, it becomes pretty straightforward to set up a wrapper and pass in all ~30 builtins, and handle all Node input without issue. Your help is extremely appreciated, and the work you have done here is extremely valuable.

@ctjlewis
Copy link
Author

ctjlewis commented Jan 2, 2021

Is this due to the generated externs? Does the compiler think path.dirname is undefined and then remove the whole statement?

I believe I saw a --jscomp_off checkVars in the TSCC source somewhere, so it would make sense that the compiler wouldn't warn.

.tscc_temp/.../externs_generated.js

/**
 * @externs
 * @suppress {duplicate,checkTypes}
 */
// NOTE: generated by tsickle, do not edit.

/** Generated by TSCC */
/**
 * @type{typeof path}
 * @const
 */
var path = {};

@theseanl
Copy link
Owner

theseanl commented Jan 2, 2021

As far as I know, tsickle does not support esModuleInterop flag. tslib.__importDefault is generated by this flag, so if you remove the esModuleInterop flag, it will compile well, I guess. I am not sure what --dependency_mode PRUNE_LEGACY exactly does, but the compilation should not fail without adding any additional flags to the closure compiler.

Most of time, you can replace default imports to namespace imports (import * as ns from '...') and the runtime behavior will be the same.

It'd be better to have esModuleInterop supported, but IMO a better place for the support would be from tsickle, not from here, that's why I am not currently working on it.

@mistersomebody
Copy link

@ctjlewis: with TSCC, you will only need to add the mock from the other workaround.
Adding externs are not necessary once you've added path as external in your spec.

@jscheid
Copy link

jscheid commented Jun 15, 2022

Sorry if this should be obvious, but how could I use Node.js globals? I'm getting errors such as:

warning TS0: type/symbol conflict for Buffer, using {?} for now
ERROR - [JSC_UNDEFINED_VARIABLE] variable Buffer is undeclared
ERROR - [JSC_UNDEFINED_VARIABLE] variable process is undeclared

I think I've fixed the last error using:

var process: NodeJS.Process;

... but still not sure how I can avoid the errors to do with Buffer.


Actually I'm getting better results by adding process and buffer modules similar to the path module in the comments above and using:

import { Buffer } from "buffer";
import { stdin } from "process";

I'm still getting the warning about "type/symbol" conflict but no errors. However, the definitions don't survive ADVANCED_COMPILATION, for example stdin.on(...) becomes u = process; u.s.m(...). I'm using the @types/node npm package, should I be using some closure compiler equivalent instead? Or do I need to add @types/node to the configuration?

@theseanl
Copy link
Owner

theseanl commented Jul 6, 2022

@jscheid Most likely the content of @types/node is not referenced from source files and is not specified in tsconfig.json compilerOptions.types. It would be helpful if you could provide a repro case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants