Skip to content

Instantly share code, notes, and snippets.

@Rlyown

Rlyown/encoding_detector.py

Created March 24, 2023 18:41
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Rlyown/f27a94e1675aaa08f9fdd2e31e292dc7 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Rlyown/f27a94e1675aaa08f9fdd2e31e292dc7 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. Rlyown created this gist Mar 24, 2023.
    24 changes: 24 additions & 0 deletions encoding_detector.py
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,24 @@
    #!/opt/homebrew/bin/python3
    # coding: utf-8
    from chardet.universaldetector import UniversalDetector
    import sys

    def help():
    print("Usage: python encoding_detector.py <file> ...")


    if __name__ == "__main__":
    if len(sys.argv) < 2:
    help()
    exit(1)

    for file in sys.argv[1:]:
    with open(file, "rb") as f:
    detector = UniversalDetector()
    for line in f:
    detector.feed(line)
    if detector.done:
    break
    detector.close()

    print(f"{file} --> {detector.result['encoding']}")