Bundle & Minify with WebPack?

[Kilowhisky]

Confirm by changing [ ] to [x] below:

• I've searched for previous similar issues and didn't find any solution

Known Issue

• I'm using ATS data type endpoint: the endpoint should look like <prefix>-ats.iot.<region>.amazonaws.com

Platform/OS/Hardware/Device
nodejs on windows / linux (attempting to deploy)

Describe the question

I'm attempting to package up my device communicator that uses aws-iot in a node app into a command line tool that can be deployed into a device linux environment and i'm having problems with WebPack. Is there anything special i have to do to get it to work?

My code completely works without running it through WebPack and WebPack builds and runs just fine if i exclude any of the AWS stuff so i think it has to do with the import of the AWS stuff.

> webpack --config webpack.config.js

asset app.min.js 149 KiB [emitted] [minimized] (name: main)
runtime modules 211 bytes 2 modules
cacheable modules 412 KiB
  modules by path ./node_modules/ 398 KiB 36 modules
  modules by path ./app/*.ts 14.4 KiB
    ./app/app.ts 3.61 KiB [built] [code generated]
    ./app/server.ts 6.24 KiB [built] [code generated]
    ./app/aws-connection.ts 4.56 KiB [built] [code generated]
11 modules

WARNING in ./node_modules/aws-crt/dist/native/binding.js 55:18-31
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/aws-crt/dist/native/io.js 11:34-54
 @ ./node_modules/aws-crt/dist/index.js 39:24-46
 @ ./node_modules/aws-iot-device-sdk-v2/dist/index.js 38:18-36
 @ ./app/server.ts 11:32-64
 @ ./app/app.ts 6:17-36

1 warnings have detailed information that is not shown.
Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.

webpack 5.30.0 compiled with 1 warnings in 2679 ms

> node .\dist\app.min.js
<huge dump of my app.min.js file itself>

Error: AWS CRT binary not present in any of the following locations:
        C:\path\bin\native\aws-crt-nodejs
        C:\path\bin\win32-x64\aws-crt-nodejs
    at Object.1221 (C:\path\dist\app.min.js:1:11199)
    at n (C:\path\dist\app.min.js:1:152697)
    at Object.7227 (C:\path\dist\app.min.js:1:11527)
    at n (C:\path\dist\app.min.js:1:152697)
    at Object.8687 (C:\path\dist\app.min.js:1:4043)
    at n (C:\path\dist\app.min.js:1:152697)
    at Object.7290 (C:\path\dist\app.min.js:1:29541)
    at n (C:\path\dist\app.min.js:1:152697)
    at Object.8316 (C:\path\dist\app.min.js:1:32467)
    at n (C:\path\dist\app.min.js:1:152697)

Is there anything i need to do so that it will be able to pass through WebPack?

[jmklix]

Can you try adding the aws-crt-nodejs to the package.json?

Using From Your Browser Application
You can either add it to package.json (if using a tool like webpack), or just import the dist.browser/ folder into your web project

[akiannillo]

It seems that the problem is in the way the binding is done... Has anybody used aws-iot-device-sdk-js-v2 with webpack?
Even adding

node: {
    __dirname: true
  }

webpack it's able to find the file but unable to load it.

[skulljoi]

It seems that the problem is in the way the binding is done... Has anybody used aws-iot-device-sdk-js-v2 with webpack?
Even adding

node: {
    __dirname: true
  }

webpack it's able to find the file but unable to load it.

I am trying without success. Did you make it work?
please help if possible

[jmklix]

Changing this to a feature request. This doesn't seem work currently so code changes, documentation, and an example need to be added.

[giantcow]

FYI: awslabs/aws-crt-nodejs#220

[revmischa]

I have this problem too
What happened with the PR?

[glemmaPaul]

Ello, same issue here :)

[gruckion]

Same here...

[Mati365]

Same here

[TwistedTwigleg]

Thank you for making this issue. We have made several Webpack improvements and changes recently that may have fixed this issue. When I tried a webpack minimizer, I was able to successfully minimize the PubSub sample and run it.

Can you please give the sample in this PR a try and let us know if it works for you? #375

[AkshitAggarwal]

Currently facing this issue, using the most latest versions on Webpack(5.8+), AWS CRT, React(18) and Node(18).

WARNING in ./node_modules/aws-crt/dist/native/binding.js 59:18-31
Critical dependency: the request of a dependency is an expression
 @ ./node_modules/aws-crt/dist/native/crypto.js 18:34-54
 @ ./node_modules/aws-crt/dist/index.js 48:28-54
 @ ./node_modules/@aws-sdk/client-lex-runtime-service/node_modules/@aws-sdk/util-user-agent-node/dist-es/is-crt-available.js 3:96-114
 @ ./node_modules/@aws-sdk/client-lex-runtime-service/node_modules/@aws-sdk/util-user-agent-node/dist-es/index.js 5:0-52 16:23-37
 @ ./node_modules/@aws-sdk/client-lex-runtime-service/dist-es/runtimeConfig.js 12:0-65 24:898-914
 @ ./node_modules/@aws-sdk/client-lex-runtime-service/dist-es/LexRuntimeServiceClient.js 11:0-73 16:24-42
 @ ./node_modules/@aws-sdk/client-lex-runtime-service/dist-es/index.js 2:0-42 2:0-42
 @ ./node_modules/@aws-amplify/interactions/lib-esm/Providers/AWSLexProvider.js 5:0-116 83:59-82 99:46-61 150:49-67
 @ ./node_modules/@aws-amplify/interactions/lib-esm/index.js 5:0-60 5:0-60
 @ ./node_modules/aws-amplify/lib-esm/index.js 11:0-57 11:0-57
 @ ./src/App.tsx 3:0-38 21:0-17
 @ ./src/index.tsx 5:0-24 8:17-20

[TwistedTwigleg]

@AkshitAggarwal do you see that issue with the sample in the PR: #375?

Edit: Also, does the project where you are seeing this issue still run, despite the warnings? There is this issue on the Javascript CRT that mentions that the application still runs, it just shows a bunch of warnings, while this original post in this issue seems to indicate that the minimized application was not able to run (from what I can gather).

[TwistedTwigleg]

Also, if you are still seeing this issue, can you provide the package.json you are using and the webpack.config.js? That way we can have a better chance of reproducing the issue being seen, which in turn will give us a better idea on what is going on.
Thanks!

[joeBotHive]

Hello I have a similar issue when using the import import { greengrasscoreipc } from 'aws-iot-device-sdk-v2'. On build the binaries are not imported which live here node_modules/aws-crt/dist/bin/darwin-arm64-cruntime/aws-crt-nodejs.node depedning on the OS. Which causes the error, I see there's a env var we can set but ideally I just want to package it all up - any support would be greatly appreciated! Cheers

import { greengrasscoreipc } from 'aws-iot-device-sdk-v2';

const CONFIG_COMPONENT = "myConfigComponent";

export async function getConfig() {
    try {
        let client = greengrasscoreipc.createClient();

        await client.connect();

        const config = await client.getConfiguration({ componentName: CONFIG_COMPONENT, keyPath: [] });
        console.log("Got initial config", JSON.stringify(config.value));
        console.log("Subscribing to config changes");

        // // Setup subscription handle
        // const subscription_handle = client.subscribeToConfigurationUpdate({ componentName: CONFIG_COMPONENT, keyPath: [] });
        // // Setup listener for config change events
        // subscription_handle.on("message", async (event) => {
        //     console.log("Config changed, will pull full new config immediately", JSON.stringify(event.configurationUpdateEvent?.keyPath));
        //
        //     const config = await client.getConfiguration({ componentName: CONFIG_COMPONENT, keyPath: [] });
        //     console.log("Got new full config", JSON.stringify(config.value));
        // });
        //
        // // Perform the subscription
        // await subscription_handle.activate();
        console.log("Subscribed to config changes");

        return config;
    } catch (err) {
        console.log("Aw shucks: ", err);
        return null;
    }
}

My webpack config :-

const path = require('path');
const Dotenv = require('dotenv-webpack');
const webpack = require('webpack');



module.exports = {
    entry: './src/server.ts',
    target: 'node',
    mode: process.env.NODE_ENV === 'production' ? 'production' : 'development',
    devtool: 'source-map',
    module: {
        rules: [
            {
                test: /\.ts$/,
                use: 'ts-loader',
                exclude: /node_modules/,
            },
        ],
    },
    resolve: {
        extensions: [".ts", ".js"],
    },
    output: {
        filename: 'server.js',
        path: path.resolve(__dirname, 'dist'),
    },
    plugins: [
        new Dotenv(),
        new webpack.IgnorePlugin({ resourceRegExp: /^fsevents$/ }),
    ],
};

the error :-

/Users/me/sandbox/project/server/dist/server.js:43106
throw new Error("AWS CRT binary not present in any of the following locations:\n\t" + search_paths.join('\n\t'));
^

Error: AWS CRT binary not present in any of the following locations:
    /Users/me/sandbox/project/dist/bin/darwin-arm64-cruntime/aws-crt-nodejs.node
at ./node_modules/aws-crt/dist/native/binding.js (/Users/me/sandbox/project/server/dist/server.js:43106:11)
at __webpack_require__ (/Users/me/sandbox/project/server/dist/server.js:294914:42)
at ./node_modules/aws-crt/dist/native/auth.js (/Users/me/sandbox/project/server/dist/server.js:41882:35)
at __webpack_require__ (/Users/me/sandbox/project/server/dist/server.js:294914:42)
at ./node_modules/aws-crt/dist/index.js (/Users/me/sandbox/project/server/dist/server.js:41830:27)
at __webpack_require__ (/Users/me/sandbox/project/server/dist/server.js:294914:42)
at ./node_modules/aws-iot-device-sdk-v2/dist/service_client_mqtt_adapter.js (/Users/me/sandbox/project/server/dist/server.js:54543:19)
at __webpack_require__ (/Users/me/sandbox/project/server/dist/server.js:294914:42)
at ./node_modules/aws-iot-device-sdk-v2/dist/iotidentity/iotidentityclient.js (/Users/me/sandbox/project/server/dist/server.js:52333:50)
at __webpack_require__ (/Users/me/sandbox/project/server/dist/server.js:294914:42)

Node.js v18.13.0

[prakashmallow]

I also facing a similar issue while installing the aws-iot-device-sdk-v2 package. After importing iot and mqtt, I attempt to run the Next.js app using npm run dev, but encounter the below errors

Error: AWS CRT binary not present in any of the following locations:
        /bin/darwin-arm64-cruntime/aws-crt-nodejs.node
    at eval (webpack-internal:///./node_modules/aws-crt/dist/native/binding.js:110:11)
    
- error node_modules/aws-crt/dist/native/binding.js (109:0) @ eval
- error Error: AWS CRT binary not present in any of the following locations:
        /bin/darwin-arm64-cruntime/aws-crt-nodejs.node
    at eval (webpack-internal:///./node_modules/aws-crt/dist/native/binding.js:110:11)