nodeJs多线程 -- 学习记录

Fibers 和 Threads

Fibers 称纤程，可以理解为协同程序，类似py和lua都有这样的模型。使用Fibers可以避免对资源的互抢，减少cpu和内存的消耗，但是Fibers并不能够真正的并行执行，同一时刻只有一个Fibers在执行，如果在其中一个Fibers中执行过多的cpu操作或者写了个死循环，则整个主程序将卡死住。node中的异步事件循环模型就有点象这个。

Threads 又称线程，他可以在同一时刻并行的执行，他们共享主进程的内存，在其中某一时刻某一个threads锁死了，是不会影响主线程以及其他线程的执行。但是为了实现这个模型，我们不得不消耗更多的内存和cpu为线程切换的开销，同时也存在可能多个线程对同一内存单元进行读写而造成程序崩溃的问题。

1. cluster

创始人Ryan Dahl建议，运行多个Nodejs进程，利用某些通信机制来协调各项任务。目前，已经有不少第三方的Node.js多进程支持模块发布，而NodeJS 0.6.x 以上的版本提供了一个cluster模块 ,允许创建“共享同一个socket”的一组进程,用来分担负载压力。本篇文章就基于该cluster模块来讲述Node.js在多核CPU下的编程

A single instance of Node.js runs in a single thread. To take advantage of multi-core systems the user will sometimes want to launch a cluster of Node.js processes to handle the load.

The cluster module allows you to easily create child processes that all share server ports.

 

官网实例

const cluster = require('cluster');
const http = require('http');
const numCPUs = require('os').cpus().length;

if (cluster.isMaster) {
  // Fork workers.
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    console.log('worker' + worker.id +'died');
  });
} else {
  // Workers can share any TCP connection
  // In this case it is an HTTP server
  http.createServer((req, res) => {
    res.writeHead(200);
    res.end('hello world\n');
  }).listen(8000);
}

 这段代码是有主线程 根据cups数目创建子进程,可以根据cluster.work.id打印出是哪个进程处理该请求。

 

 

2.node-threads-a-gogo

   a.安装  

npm install threads_a_gogo
实例
A.- Here's a program that makes Node's event loop spin freely and as fast as possible: it simply prints a dot to the console in each turn:

//cat examples/quickIntro_loop.js

1 (function spinForever () {
   process.stdout.write(".");
   process.nextTick(spinForever);
})();

 

B.- Here's another program that adds to the one above a fibonacci(35) call in each turn, a CPU-bound task that takes quite a while to complete and that blocks the event loop making it spin slowly and clumsily. The point is simply to show that you can't put a job like that in the event loop because Node will stop performing properly when its event loop can't spin fast and freely due to a callback/listener/nextTick()ed function that's blocking.

//cat examples/quickIntro_blocking.js
function fibo (n) {
  return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
}

(function fiboLoop () {
  process.stdout.write(fibo(35).toString());
  process.nextTick(fiboLoop);
})();

(function spinForever () {
  process.stdout.write(".");
  process.nextTick(spinForever);
})();

C.- The program below uses threads_a_gogo to run the fibonacci(35) calls in a background thread, so Node's event loop isn't blocked at all and can spin freely again at full speed:

1 //cat examples/quickIntro_oneThread.js
 2 function fibo (n) {
 3   return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
 4 }
 5 
 6 function cb (err, data) {
 7   process.stdout.write(data);
 8   this.eval('fibo(35)', cb);
 9 }
10 
11 var thread= require('threads_a_gogo').create();
12 
13 thread.eval(fibo).eval('fibo(35)', cb);
14 
15 (function spinForever () {
16   process.stdout.write(".");
17   process.nextTick(spinForever);
18 })();

 

D.- This example is almost identical to the one above, only that it creates 5 threads instead of one, each running a fibonacci(35) in parallel and in parallel too with Node's event loop that keeps spinning happily at full speed in its own thread:

//cat examples/quickIntro_fiveThreads.js

function fibo (n) {

  return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
}

function cb (err, data) {
  process.stdout.write(" ["+ this.id+ "]"+ data);
  this.eval('fibo(35)', cb);
}

var threads_a_gogo= require('threads_a_gogo');

threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);
threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);
threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);
threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);
threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);

(function spinForever () {
  process.stdout.write(".");
  process.nextTick(spinForever);
})();

 

 E.- The next one asks threads_a_gogo to create a pool of 10 background threads, instead of creating them manually one by one:

//cat examples/multiThread.js
function fibo (n) {
  return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
}

var numThreads= 10;
var threadPool= require('threads_a_gogo').createPool(numThreads).all.eval(fibo);

threadPool.all.eval('fibo(35)', function cb (err, data) {
  process.stdout.write(" ["+ this.id+ "]"+ data);
  this.eval('fibo(35)', cb);
});

(function spinForever () {
  process.stdout.write(".");
  process.nextTick(spinForever);
})();

 

F.- This is a demo of the threads_a_gogo eventEmitter API, using one thread:

//cat examples/quickIntro_oneThreadEvented.js
var thread= require('threads_a_gogo').create();
thread.load(__dirname + '/quickIntro_evented_childThreadCode.js');

/*
  This is the code that's .load()ed into the child/background thread:

  function fibo (n) {
    return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
  }

  thread.on('giveMeTheFibo', function onGiveMeTheFibo (data) {
    this.emit('theFiboIs', fibo(+data)); //Emits 'theFiboIs' in the parent/main thread.
  });

*/

//Emit 'giveMeTheFibo' in the child/background thread.
thread.emit('giveMeTheFibo', 35);

//Listener for the 'theFiboIs' events emitted by the child/background thread.
thread.on('theFiboIs', function cb (data) {
  process.stdout.write(data);
  this.emit('giveMeTheFibo', 35);
});

(function spinForever () {
  process.stdout.write(".");
  process.nextTick(spinForever);
})();

G.- This is a demo of the threads_a_gogo eventEmitter API, using a pool of threads:

 

//cat examples/quickIntro_multiThreadEvented.js
var numThreads= 10;
var threadPool= require('threads_a_gogo').createPool(numThreads);
threadPool.load(__dirname + '/quickIntro_evented_childThreadCode.js');

/*
  This is the code that's .load()ed into the child/background threads:

  function fibo (n) {
    return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
  }

  thread.on('giveMeTheFibo', function onGiveMeTheFibo (data) {
    this.emit('theFiboIs', fibo(+data)); //Emits 'theFiboIs' in the parent/main thread.
  });

*/

//Emit 'giveMeTheFibo' in all the child/background threads.
threadPool.all.emit('giveMeTheFibo', 35);

//Listener for the 'theFiboIs' events emitted by the child/background threads.
threadPool.on('theFiboIs', function cb (data) {
  process.stdout.write(" ["+ this.id+ "]"+ data);
  this.emit('giveMeTheFibo', 35);
});

(function spinForever () {
  process.stdout.write(".");
  process.nextTick(spinForever);
})();