Home Node Request piping data failure
Reply: 0

Node Request piping data failure

PhilG
1#
PhilG Published in 2018-02-12 09:39:04Z

I am trying to download about 800MB per file. This is the code to achieve it:

function makeRequest(url,name,callback){
                name = promObj.Name;
                requestsdone =-1;
                request(url,{auth:auth}, function (err,respnse,body) {
                })
                    .on('error', function (err) {
                        console.log(err)
                    })
                    .on('end', function () {
                        requestsdone++;
                    }).pipe(fs.createWriteStream('./app/data/' + name[requestsdone] + '.zip'));

                callback();

            }

            async.eachLimit(promObj.requestURLS, 2, makeRequest, function(err) {
                if(err) reject(promObj)
                else{
                    resolve(promObj);
                }
            });

I updated Node.js to version 9.5.0 and get this error now:

<--- Last few GCs --->

[12060:000001FED372E610]   487169 ms: Mark-sweep 28.7 (35.4) -> 28.7 (35.4) 
MB, 33.6 / 0.0 ms  allocation failure GC in old space requested
[12060:000001FED372E610]   487205 ms: Mark-sweep 28.7 (35.4) -> 28.7 (35.4) 
MB, 36.2 / 0.0 ms  last resort GC in old space requested
[12060:000001FED372E610]   487240 ms: Mark-sweep 28.7 (35.4) -> 28.7 (35.4) 
MB, 34.7 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000001311C6254D9 <JSObject>
1: stringSlice(aka stringSlice) [buffer.js:586] [bytecode=0000031E27B6DEE1 offset=11](this=000003FACAA022D1 <undefined>,buf=000000CDF7528259 <Uint8Array map = 00000392BE4C4D01>,encoding=000003FACAA022D1 <undefined>,start=0,end=801130252)
2: toString [buffer.js:664] [bytecode=0000031E27B6DB29 offset=148](this=000000CDF7528259 <Uint8Array map = 00000392BE4C4D01>,encoding=000003FACAA022D1...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

How could I solve this? Thanks in advance.

PS: This question refers to Requests with request-promise pipe result (Large data) ,which I asked some time ago. I restructured my code and thought that this problem is worth a new, sepperated question because the original topic of the other question was a different one.

You need to login account before you can post.

About| Privacy statement| Terms of Service| Advertising| Contact us| Help| Sitemap|
Processed in 0.303528 second(s) , Gzip On .

© 2016 Powered by mzan.com design MATCHINFO