Skip to content

Instantly share code, notes, and snippets.

@sierrezinal

sierrezinal/io.py

Forked from mfcabrera/io.py
Created May 3, 2022 18:05
Show Gist options
  • Save sierrezinal/e871b6da74769446610b0f33d3590483 to your computer and use it in GitHub Desktop.
Save sierrezinal/e871b6da74769446610b0f33d3590483 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
io.py
def delimited(filename, delimiter=' ', bufsize=4096):
'''
Creates a generator of word from a file based on a delimiter (by default white space).
'''
buf = ''
with open(filename) as file:
while True:
newbuf = file.read(bufsize)
if not newbuf:
yield buf
return
buf += newbuf
words = buf.split(delimiter)
for word in words[:-1]:
yield word
buf = words[-1]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment