Skip to content

Instantly share code, notes, and snippets.

@gorborukov

gorborukov/remove_jsonl_fields.rb

Created April 13, 2023 05:59
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save gorborukov/4f70d85022e329e299204261035adb89 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save gorborukov/4f70d85022e329e299204261035adb89 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. gorborukov created this gist Apr 13, 2023.
    18 changes: 18 additions & 0 deletions remove_jsonl_fields.rb
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,18 @@
    require 'json'

    # Read the JSON data from the original file line by line
    modified_data = ''
    File.foreach('original_file.jsonl') do |line|
    # Parse the line as a JSON object
    json_object = JSON.parse(line)

    # Remove the 'category' and 'context' fields from the JSON object
    json_object.delete('category')
    json_object.delete('context')

    # Convert the modified JSON object back into a JSON string and append it to the modified data variable
    modified_data << JSON.generate(json_object) << "\n"
    end

    # Write the modified JSON data to a new file
    File.write('modified_file.jsonl', modified_data)