Created
April 10, 2012 19:12
-
-
Save waylan/2353749 to your computer and use it in GitHub Desktop.
Revisions
-
Waylan Limberg revised this gist
Apr 11, 2012 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -71,7 +71,7 @@ for x in xrange(100): try: p.stdin.write(line) except IOError as e: if e.errno == errno.EPIPE or e.errno == errno.EINVAL: # Stop loop on "Invalid pipe" or "Invalid argument". # No sense in continuing with broken pipe. break -
Waylan Limberg revised this gist
Apr 11, 2012 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -71,7 +71,7 @@ for x in xrange(100): try: p.stdin.write(line) except IOError as e: if e.errno = errno.EPIPE or e.errno = errno.EINVAL: # Stop loop on "Invalid pipe" or "Invalid argument". # No sense in continuing with broken pipe. break -
Waylan Limberg revised this gist
Apr 11, 2012 . 1 changed file with 7 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -71,13 +71,16 @@ for x in xrange(100): try: p.stdin.write(line) except IOError as e: if e.errno = errno.EPIPE and e.errno = errno.EINVAL: # Stop loop on "Invalid pipe" or "Invalid argument". # No sense in continuing with broken pipe. break else: # Raise any other error. raise p.stdin.close() p.wait() print 'All done!' # This should always be printed below any output written to less. ``` -
Waylan Limberg revised this gist
Apr 11, 2012 . 1 changed file with 25 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -56,4 +56,28 @@ except IOError as e: raise ``` Note that [`errno.EPIPE`](http://docs.python.org/library/errno.html?highlight=errno#errno.EPIPE) is "Broken pipe" and [`errno.EINVAL`](http://docs.python.org/library/errno.html?highlight=errno#errno.EINVAL) is "Invalid argument". So the final code looks like this: ``` python from subprocess import Popen, PIPE import errno p = Popen('less', stdin=PIPE) for x in xrange(100): line = 'Line number %d.\n' % x try: p.stdin.write(line) except IOError as e: # Silently fail on "Invalid pipe" or "Invalid argument". # Raise any other error. if e.errno != errno.EPIPE and e.errno != errno.EINVAL: raise p.stdin.close() p.wait() print 'All done!' ``` -
Waylan Limberg revised this gist
Apr 11, 2012 . 1 changed file with 27 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -30,4 +30,30 @@ for x in xrange(100): p.communicate('\n'.join(out)) ``` This works. We only have one call to `communicate` and that calls `wait` properly. Unfortunately, we have to create the entire output in memory before writing any of it out. We can do better: ```python from subprocess import Popen, PIPE p = Popen('less', stdin=PIPE) for x in xrange(100): p.stdin.write('Line number %d.\n' % x) p.stdin.close() p.wait() ``` The key it to close stdin (flush and send EOF) before calling `wait`. This is actually what `communicate` does internally minus all the stdout and stderr stuff I don't need. If I wanted to force the buffer to remain empty, I suppose I could do `p.stdin.flush()` on each loop, but why? Note that there probably should be some error checking on write (like there is in the source of `communicate`. Perhaps something like: ``` python import errno ... try: p.stdin.write(input) except IOError as e: if e.errno != errno.EPIPE and e.errno != errno.EINVAL: raise ``` Note that [`errno.EPIPE`](http://docs.python.org/library/errno.html?highlight=errno#errno.EPIPE) is "Broken pipe" and [`errno.EINVAL`](http://docs.python.org/library/errno.html?highlight=errno#errno.EINVAL) is "Invalid argument". -
Waylan Limberg revised this gist
Apr 10, 2012 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -26,7 +26,7 @@ from subprocess import Popen, PIPE out = [] p = Popen('less', stdin=PIPE) for x in xrange(100): out.append('Line number %d.' % x) p.communicate('\n'.join(out)) ``` -
Waylan Limberg created this gist
Apr 10, 2012 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,33 @@ Here's a few things I tried to write output to a python subprocess pipe. ```python from subprocess import Popen, PIPE p = Popen('less', stdin=PIPE) for x in xrange(100): p.communicate('Line number %d.\n' % x) ``` This seemed like the most obvious solution but it fails miserably. It seems that the first call to `communicate` also closes the pipe and the second loop raises an exception. ```python from subprocess import Popen, PIPE p = Popen('less', stdin=PIPE) for x in xrange(100): p.stdin.write('Line number %d.\n' % x) ``` This is expressly stated to be a bad idea in the docs, but it works - sort of. I get some weird behavior. There's no call to `p.wait()` (which `communicate` does by default) so anything after the loop runs before the subproccess (`less` in this case) is closed. Adding a call to `wait` after the loop causes even weirder behavior. ```python from subprocess import Popen, PIPE out = [] p = Popen('less', stdin=PIPE) for x in xrange(100): out.append('Line number %d.\n' % x) p.communicate('\n'.join(out)) ``` This works __great__! We only have one call to `communicate` and that calls `wait` properly. Unfortunately, we have to create the entire output in memory before writing any of it out. At least it works.