Skip to content

Instantly share code, notes, and snippets.

@FreeSlaver
Forked from eligao/query.py
Created March 26, 2024 09:10
Show Gist options
  • Select an option

  • Save FreeSlaver/34777d34071ab105f4fe8f882d223b31 to your computer and use it in GitHub Desktop.

Select an option

Save FreeSlaver/34777d34071ab105f4fe8f882d223b31 to your computer and use it in GitHub Desktop.

Revisions

  1. @eligao eligao revised this gist Apr 2, 2016. 1 changed file with 4 additions and 0 deletions.
    4 changes: 4 additions & 0 deletions query.py
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,8 @@
    # !/bin/python2

    #author: @eligao
    #licence: WTFPL http://www.wtfpl.net/

    import os
    import sys, getopt
    from multiprocessing import Pool,freeze_support
  2. @eligao eligao created this gist Apr 2, 2016.
    48 changes: 48 additions & 0 deletions query.py
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,48 @@
    # !/bin/python2
    import os
    import sys, getopt
    from multiprocessing import Pool,freeze_support


    #使用方法:先编辑这里再使用,懒得写argc,argv了

    #数据文件根目录
    folderpath="./"
    #工作线程数
    #固态硬盘建议填CPU线程数,i7-4700MQ大约可以到200-300M/S的速度
    #机械硬盘不要填太大,并发IO数太多反而更慢
    maxthreads=2
    #要查找的字符串
    querystr="dingsanshi"

    #已知问题:
    #1.以当前目录为根目录时,会查询到本文件
    #2.偷懒没有写命令行参数
    #3.you tell me

    def searchinfile(file):
    ret_val=list()
    print "Searching in:"+file
    for line in open(file,'r').readlines():
    if line.find(querystr)!=-1:
    print "MATCH FOUND:"+line
    ret_val.append(line)
    return ret_val

    if __name__ == '__main__':
    #for windows compatibility
    freeze_support()
    #worker pool
    pool=Pool(maxthreads)
    #get file paths list
    filepaths=list()
    for root, _, files in os.walk(folderpath):
    for name in files:
    filepaths.append(os.path.join(root,name))
    #launch workers
    res=pool.map(searchinfile,filepaths)
    #extract results
    for ls in res:
    for ln in ls:
    print ln