Skip to content

Instantly share code, notes, and snippets.

@jeemyeong
Forked from jcouyang/README.org
Created March 2, 2021 00:55
Show Gist options
  • Select an option

  • Save jeemyeong/fbc2017a3c4965dddfd592011a35a95d to your computer and use it in GitHub Desktop.

Select an option

Save jeemyeong/fbc2017a3c4965dddfd592011a35a95d to your computer and use it in GitHub Desktop.

Revisions

  1. @jcouyang jcouyang revised this gist Apr 12, 2017. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions README.org
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,5 @@
    * The Promise All Problem

    in case of processing a very large array e.g. =Promise.all(A_VERY_LARGE_ARRAY_OF_XHR_PROMISE)=

    which would probably blow you browser memory by trying to send all requests at the same time
  2. @jcouyang jcouyang revised this gist Apr 12, 2017. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions README.org
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,6 @@
    in case of processing a very large array e.g. =Promise.all(A_VERY_LARGE_ARRAY_OF_XHR_PROMISE)=

    which would probably blow you browser memory by trying to send all request at the send time
    which would probably blow you browser memory by trying to send all requests at the same time

    solution is limit the concurrent of requests, and wrap promise in thunk

    @@ -10,4 +10,4 @@ if set concurrent to 2, it will send request BLAH1 and BLAH2 at the same time

    if BLAH1 return response and resolved, will immediatly send request to BLAH3

    in this way promise sending at the same time always keep the limit 2 we just set before
    in this way promise sending at the same time always keep the limit 2 which we've just configed before
  3. @jcouyang jcouyang revised this gist Apr 12, 2017. 1 changed file with 13 additions and 0 deletions.
    13 changes: 13 additions & 0 deletions README.org
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,13 @@
    in case of processing a very large array e.g. =Promise.all(A_VERY_LARGE_ARRAY_OF_XHR_PROMISE)=

    which would probably blow you browser memory by trying to send all request at the send time

    solution is limit the concurrent of requests, and wrap promise in thunk

    =Promise.allConcurrent(2)([()=>fetch('BLAH1'), ()=>fetch('BLAH2'),...()=>fetch('BLAHN')])=

    if set concurrent to 2, it will send request BLAH1 and BLAH2 at the same time

    if BLAH1 return response and resolved, will immediatly send request to BLAH3

    in this way promise sending at the same time always keep the limit 2 we just set before
  4. @jcouyang jcouyang revised this gist Apr 12, 2017. 2 changed files with 3 additions and 3 deletions.
    4 changes: 2 additions & 2 deletions prelude-test.js
    Original file line number Diff line number Diff line change
    @@ -15,7 +15,7 @@ describe('promiseAllStepN', function(){
    }, 1000)))
    tasks[2].returns(Promise.resolve(2))

    return Promise.allStepN(2)(tasks).then(x=>console.log(x))
    return Promise.allConcurrent(2)(tasks).then(x=>console.log(x))
    })
    })
    })
    @@ -34,7 +34,7 @@ describe('promiseAllStepN', function(){
    resolve(1)
    done()
    }, 1000)))
    return Promise.allStepN(2)(tasks).then(x=>console.log(x))
    return Promise.allConcurrent(2)(tasks).then(x=>console.log(x))
    })
    })
    })
    2 changes: 1 addition & 1 deletion prelude.js
    Original file line number Diff line number Diff line change
    @@ -25,4 +25,4 @@ function promiseAllStepN(n, list) {
    }
    })
    }
    Promise.allCucurrent = n => list => promiseAllStepN(n, list)
    Promise.allConcurrent = n => list => promiseAllStepN(n, list)
  5. @jcouyang jcouyang revised this gist Apr 12, 2017. No changes.
  6. @jcouyang jcouyang created this gist Apr 12, 2017.
    42 changes: 42 additions & 0 deletions prelude-test.js
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,42 @@
    describe('promiseAllStepN', function(){
    describe('3 tasks, and cucurrent is 2', function(){
    let tasks;
    beforeEach(function(){
    tasks = range(3).map(x=>sinon.stub())
    })
    describe('1 is finish', function(){
    it('will kickoff the third task', function(done){
    tasks[0].returns(Promise.resolve(0))
    let task2 = tasks[2]
    tasks[1].returns(new Promise(resolve=>setTimeout(()=>{
    expect(task2.called).to.be.equal(true)
    resolve(1)
    done()
    }, 1000)))
    tasks[2].returns(Promise.resolve(2))

    return Promise.allStepN(2)(tasks).then(x=>console.log(x))
    })
    })
    })

    describe('10 tasks, and cucurrent is 3', function(){
    let tasks;
    beforeEach(function(){
    tasks = range(10).map(x=>sinon.stub())
    })
    describe('1st is finish but 2nd stuck', function(){
    it.only('final task will run before 2nd', function(done){
    tasks.forEach((task,index) => task.returns(Promise.resolve(index)))
    let task10 = tasks[9]
    tasks[1].returns(new Promise(resolve=>setTimeout(()=>{
    expect(task10.called).to.be.equal(true)
    resolve(1)
    done()
    }, 1000)))
    return Promise.allStepN(2)(tasks).then(x=>console.log(x))
    })
    })
    })

    })
    28 changes: 28 additions & 0 deletions prelude.js
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,28 @@
    function promiseAllStepN(n, list) {
    let tail = list.splice(n)
    let head = list
    let resolved = []
    let processed = 0
    return new Promise(resolve=>{
    head.forEach(x=>{
    let res = x()
    resolved.push(res)
    res.then(y=>{
    runNext()
    return y
    })
    })
    function runNext(){
    if(processed == tail.length){
    resolve(Promise.all(resolved))
    }else{
    resolved.push(tail[processed]().then(x=>{
    runNext()
    return x
    }))
    processed++
    }
    }
    })
    }
    Promise.allCucurrent = n => list => promiseAllStepN(n, list)