webpack and kafka-node integration

333
July 27, 2017, at 04:28 AM

I'm trying to integrate into a HTML file a Javascript written by me. Since script requires kafka-node in order to produce something like this, I'm trying to use webpack to bundle everything. I'm using this getting started guide and adapting it to my specific case. The only thing I'm changing is index.js file, putting my script in there as follows:

import _ from 'kafka-node';
document.write("Starting Javascript Kafka Consumer...");
var kafka = require('kafka-node'),
    Consumer = kafka.Consumer,
    client = new kafka.Client("127.0.0.1:2181"),
    consumer = new Consumer( ...
...

I'm using same configuration file as published in the above getting started referred link.

I did run npm install --save-dev webpack and npm install --save kafka-node. This is my output:

./node_modules/.bin/webpack --config webpack.config.js 
Hash: f2fd34c274324b3ac92a
Version: webpack 3.4.1
Time: 1475ms
    Asset     Size  Chunks                    Chunk Names
bundle.js  1.48 MB       0  [emitted]  [big]  main
   [6] (webpack)/buildin/global.js 509 bytes {0} [built]
  [39] (webpack)/buildin/module.js 517 bytes {0} [built]
  [69] ./src/index.js 890 bytes {0} [built]
 [103] ./node_modules/optional 160 bytes {0} [optional] [built]
 [112] ./node_modules/buffermaker ^.*BufferMaker$ 205 bytes {0} [built]
 [122] ./node_modules/node-zookeeper-client/lib/jute 160 bytes {0} [built]
    + 139 hidden modules
WARNING in ./node_modules/node-zookeeper-client/lib/jute/index.js
771:20-47 Critical dependency: the request of a dependency is an expression
WARNING in ./node_modules/optional/optional.js
6:11-26 Critical dependency: the request of a dependency is an expression
ERROR in ./node_modules/kafka-node/lib/client.js
Module not found: Error: Can't resolve 'net' in '/home/work-xpandit/consumerJS/node_modules/kafka-node/lib'
 @ ./node_modules/kafka-node/lib/client.js 3:10-24
 @ ./node_modules/kafka-node/kafka.js
 @ ./src/index.js
ERROR in ./node_modules/kafka-node/lib/kafkaClient.js
Module not found: Error: Can't resolve 'net' in '/home/work-xpandit/consumerJS/node_modules/kafka-node/lib'
 @ ./node_modules/kafka-node/lib/kafkaClient.js 10:12-26
 @ ./node_modules/kafka-node/kafka.js
 @ ./src/index.js
ERROR in ./node_modules/kafka-node/lib/client.js
Module not found: Error: Can't resolve 'tls' in '/home/work-xpandit/consumerJS/node_modules/kafka-node/lib'
 @ ./node_modules/kafka-node/lib/client.js 5:10-24
 @ ./node_modules/kafka-node/kafka.js
 @ ./src/index.js
ERROR in ./node_modules/kafka-node/lib/kafkaClient.js
Module not found: Error: Can't resolve 'tls' in '/home/work-xpandit/consumerJS/node_modules/kafka-node/lib'
 @ ./node_modules/kafka-node/lib/kafkaClient.js 12:12-26
 @ ./node_modules/kafka-node/kafka.js
 @ ./src/index.js
ERROR in ./node_modules/node-zookeeper-client/index.js
Module not found: Error: Can't resolve 'net' in '/home/work-xpandit/consumerJS/node_modules/node-zookeeper-client'
 @ ./node_modules/node-zookeeper-client/index.js 19:24-38
 @ ./node_modules/kafka-node/lib/consumerGroupMigrator.js
 @ ./node_modules/kafka-node/lib/consumerGroup.js
 @ ./node_modules/kafka-node/kafka.js
 @ ./src/index.js
ERROR in ./node_modules/node-zookeeper-client/lib/ConnectionManager.js
Module not found: Error: Can't resolve 'net' in '/home/work-xpandit/consumerJS/node_modules/node-zookeeper-client/lib'
 @ ./node_modules/node-zookeeper-client/lib/ConnectionManager.js 8:10-24
 @ ./node_modules/node-zookeeper-client/index.js
 @ ./node_modules/kafka-node/lib/consumerGroupMigrator.js
 @ ./node_modules/kafka-node/lib/consumerGroup.js
 @ ./node_modules/kafka-node/kafka.js
 @ ./src/index.js
ERROR in ./node_modules/node-zookeeper-client/lib/jute/index.js
Module not found: Error: Can't resolve 'fs' in '/home/work-xpandit/consumerJS/node_modules/node-zookeeper-client/lib/jute'
 @ ./node_modules/node-zookeeper-client/lib/jute/index.js 14:9-22
 @ ./node_modules/node-zookeeper-client/index.js
 @ ./node_modules/kafka-node/lib/consumerGroupMigrator.js
 @ ./node_modules/kafka-node/lib/consumerGroup.js
 @ ./node_modules/kafka-node/kafka.js
 @ ./src/index.js
ERROR in ./node_modules/buffermaker/test/BufferMaker.js
Module not found: Error: Can't resolve 'should' in '/home/work-xpandit/consumerJS/node_modules/buffermaker/test'
 @ ./node_modules/buffermaker/test/BufferMaker.js 1:0-17
 @ ./node_modules/buffermaker ^.*BufferMaker$
 @ ./node_modules/buffermaker/index.js
 @ ./node_modules/kafka-node/lib/protocol/protocol.js
 @ ./node_modules/kafka-node/lib/protocol/index.js
 @ ./node_modules/kafka-node/kafka.js
 @ ./src/index.js

Any thoughts?

Answer 1

The way this module's tests are written it depends on should as a devDependency. NPM dev-dependencies aren't installed in the parent project even if the parent is in dev mode. Just run:

npm i -D should

Even if you don't use it in your project. This resolves the issue for me. My project had its other requirements such as Mocha already installed. If you're missing any more, you may need to do this for more than one package.

READ ALSO
Postgraphql on Separate Heroku Port for Security?

Postgraphql on Separate Heroku Port for Security?

I'd like to take a stab at using postgraphql for my Heroku appI understand that I can use it either with the CLI or as middleware, but since I can't use the CLI with Heroku, I'm using the middleware to spawn an express app that acts as the graphQL server

330
How do I tokenize the arguments from a Redis command string in Node?

How do I tokenize the arguments from a Redis command string in Node?

Supposing I have a Redis command like: HMSET "my Key" greeting "\"hello world\"" is there a library to actually decompose the string into tokens that Redis comprehends and can use from the node_redis package?

394
Map of string=>any (`require`)?

Map of string=>any (`require`)?

I have a structure where I need ordered elementsPreviously using an {}

250
Error: Uncaught (in promise) TypeError: this.props.repos.map is not a function

Error: Uncaught (in promise) TypeError: this.props.repos.map is not a function

I'm new in React but I'm building a simple app to search GitHub users just put their usernamesIt must bring me information about the users and theirs respective repositories, but I have a problem whit the code:

706