Last active
October 17, 2022 00:27
-
-
Save caike/ebccc95bd46f5fa1404d to your computer and use it in GitHub Desktop.
Revisions
-
caike revised this gist
Jul 17, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -7,6 +7,6 @@ readStream.on('readable', function() { } }) writeStream.on('drain', function() { canWrite = true; }); -
caike revised this gist
Jul 17, 2014 . 1 changed file with 4 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,10 +1,10 @@ var canWrite = true; readStream.on('readable', function() { while(canWrite && (null !== (chunk = readStream.read()))){ var bufferGood = writeStream.write(chunk) if (!bufferGood) canWrite = false; } }) readStream.on('drain', function() { -
caike revised this gist
Jul 17, 2014 . 2 changed files with 13 additions and 5 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -6,13 +6,9 @@ * Instead of `data` events spewing, call `read()` to pull data from source. * When there isn't any data to consume, then `read()` will return **undefined**. * Adding a `data` event listener will switch the Readable stream into "old mode", where data is emitted as soon as it is available rather than waiting for you to call `read()` to consume it. This requires you to handle backpressure problems manually. * The `pipe` method helps write less code and handles back-pressure. * If you add an `end` listener and don't ever `read()` or `pipe()`, it'll never emit `end`. ## Resources [Isaac Z. Presentation](https://dl.dropboxusercontent.com/u/3685/presentations/streams2/streams2-ko.pdf) This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,12 @@ var canWrite = true; readStream.on('readable', function() { while(canWrite && (null !== (chunk = readStream.read()))){ var bufferGood = writeStream.write(chunk) if (!bufferGood) canWrite = false; } }) readStream.on('drain', function() { canWrite = true; }); -
caike revised this gist
Jul 9, 2014 . 1 changed file with 1 addition and 14 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -9,20 +9,7 @@ * The `pipe` method helps write less code and handles backpressure. * If you add an `end` listener and don't ever `read()` or `pipe()`, it'll never emit `end`. ## Question Is there still room for back-pressure issues in streams2 ? -
caike renamed this gist
Jul 9, 2014 . 1 changed file with 0 additions and 0 deletions.There are no files selected for viewing
File renamed without changes. -
caike revised this gist
Jul 9, 2014 . 1 changed file with 27 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,27 @@ var http = require('http'); var fs = require('fs'); http.createServer(function(req, res) { var newFile = fs.createWriteStream('destination.jpg'); var fileBytes = req.headers['content-length']; var uploadedBytes = 0; req.on('readable', function() { var chunk = null; while(null !== (chunk = req.read())){ uploadedBytes += chunk.length; var progress = (uploadedBytes / fileBytes) * 100; res.write("Progress: " + parseInt(progress, 10) + "%\n"); } }); req.on('end', function() { res.end('uploaded!'); }); req.pipe(newFile); }).listen(8080); // run with `curl --upload-file <your-file.jpg> http://localhost:8080` -
caike revised this gist
Jul 9, 2014 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -28,7 +28,7 @@ Is there still room for back-pressure issues in streams2 ? ## Resources [Isaac Z. Presentation](https://dl.dropboxusercontent.com/u/3685/presentations/streams2/streams2-ko.pdf) [Tim Caswell](http://howtonode.org/streams-explained) [Stream Compatibility](http://nodejs.org/api/stream.html#stream_compatibility_with_older_node_versions) [Getting Started With Streams2](http://maxantoni.de/blog/2013/06/getting-started-with-streams-2.html) -
caike revised this gist
Jul 9, 2014 . 1 changed file with 4 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -28,7 +28,7 @@ Is there still room for back-pressure issues in streams2 ? ## Resources [Izaac S. Presentation](https://dl.dropboxusercontent.com/u/3685/presentations/streams2/streams2-ko.pdf) [Tim Caswell](http://howtonode.org/streams-explained) [Stream Compatibility](http://nodejs.org/api/stream.html#stream_compatibility_with_older_node_versions) [Getting Started With Streams2](http://maxantoni.de/blog/2013/06/getting-started-with-streams-2.html) -
caike revised this gist
Jul 9, 2014 . 1 changed file with 12 additions and 5 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -4,11 +4,10 @@ * Introduced in node v0.10. * "suck" streams instead of "spew" streams. * Instead of `data` events spewing, call `read()` to pull data from source. * When there isn't any data to consume, then `read()` will return **undefined**. * Adding a `data` event listener will switch the Readable stream into "old mode", where data is emitted as soon as it is available rather than waiting for you to call `read()` to consume it. This requires you to handle backpressure problems manually. * The `pipe` method helps write less code and handles backpressure. * If you add an `end` listener and don't ever `read()` or `pipe()`, it'll never emit `end`. From <http://nodejs.org/api/stream.html#stream_class_stream_readable>: @@ -23,5 +22,13 @@ Readable streams have two "modes": When in flowing mode (old mode), data is read from the underlying system and provided to your program as fast as possible. In non-flowing mode (a.k.a. streams2), you must explicitly call `stream.read()` to get chunks of data out. ## Questions Is there still room for back-pressure issues in streams2 ? ## Resources [Izaac S. Presentation](https://dl.dropboxusercontent.com/u/3685/presentations/streams2/streams2-ko.pdf) [Tim Caswell](http://howtonode.org/streams-explained) [Stream Compatibility](http://nodejs.org/api/stream.html#stream_compatibility_with_older_node_versions) [Getting Started With Streams2](http://maxantoni.de/blog/2013/06/getting-started-with-streams-2.html) -
caike renamed this gist
Jul 8, 2014 . 1 changed file with 0 additions and 0 deletions.There are no files selected for viewing
File renamed without changes. -
caike revised this gist
Jul 8, 2014 . 1 changed file with 27 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,27 @@ # NodeJS Streams * Current implementation is known as **streams2**. * Introduced in node v0.10. * "suck" streams instead of "spew" streams. * Instead of `data` events spewing, call `read()` to pull data from source. * When there isn't any data to consume, then `read()` will return null. * In order to tweak back pressure settings, there is a `highWaterMark` and `lowWaterMark` property. These are limits that lets us configure how different streams exert backpressure. * All Writables emit `finish` when ended and flushed. * New streams can switch into old-mode, where they spew data. Adding a `data` event listener will switch the Readable stream into "old mode", where data is emitted as soon as it is available rather than waiting for you to call `read()` to consume it. * If you add an `end` listener, but don't add a `data` listener, and don't ever `read()` or `pipe()`, it'll never emit `end`. From <http://nodejs.org/api/stream.html#stream_class_stream_readable>: The Readable stream interface is the abstraction for a source of data that you are reading from. In other words, data comes out of a Readable stream. A Readable stream will not start emitting data until you indicate that you are ready to receive it. Readable streams have two "modes": * flowing mode * non-flowing mode. When in flowing mode (old mode), data is read from the underlying system and provided to your program as fast as possible. In non-flowing mode (a.k.a. streams2), you must explicitly call `stream.read()` to get chunks of data out. ## Resources [Izaac S. Presentation](https://dl.dropboxusercontent.com/u/3685/presentations/streams2/streams2-ko.pdf) -
caike created this gist
Jul 8, 2014 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,21 @@ // streams1 // works in node v0.6.21 // works in node v0.10.29 (latest stable) // Tested with `curl -d 'akshdakjhdakjhsdkajhdjkahdsjka' http://localhost:8080` var http = require('http'); http.createServer(function(req, res) { console.log('Got Request'); res.writeHead(200); req.on('data', function(chunk){ console.log('got %d bytes of data', chunk.length); }); req.on('end', function() { res.end('over'); }); }).listen(8080); This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,23 @@ // streams2 // works in v0.10.29 // does not work in v.0.6.21 // Example taken from http://nodejs.org/api/stream.html#stream_class_stream_readable // Tested with `curl -d 'akshdakjhdakjhsdkajhdjkahdsjka' http://localhost:8080` var http = require('http'); http.createServer(function(req, res) { console.log('Got Request'); req.on('readable', function() { var chunk = null; while (null !== (chunk = req.read())) { console.log('got %d bytes of data', chunk.length); } }); req.on('end', function() { res.end('Done'); }); }).listen(8080);