Skip to content

Instantly share code, notes, and snippets.

@dannguyen

dannguyen/cardib-politics-talk-transcribe.md

Last active October 18, 2025 15:51
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save dannguyen/9b8c51f5bb853209f19f1a0f18f0f74c to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save dannguyen/9b8c51f5bb853209f19f1a0f18f0f74c to your computer and use it in GitHub Desktop.
Download ZIP
An example of how to use command-line tools to transcribe a viral video of Cardi B
Raw
cardib-politics-talk-transcribe.md

Transcribing Cardi B's political speech with command-line tools

Inspired by the following exchange on Twitter, in which someone captures and posts a valuable video onto Twitter, but doesn't have the resources to easily transcribe it for the hearing-impaired:

Screencap of @jordanuhl's video tweet, followed by a request for a transcript

The instructions and code below show how to use command-line tools/scripting and Amazon's Transcribe service to transcribe the audio from online video. tl;dr: AWS Transcribe is a pretty amazing service!

(note: you should obviously not do this as one big old bash script, but I wrote this up as an example of what CLI can do if you have some weird elaborate needs)

Requirements

Sign-up for Amazon Web Services: https://aws.amazon.com

Install:

  • youtube-dl - for fetching video files from social media services
  • awscli - for accessing various AWS services, specfically S3 (for storing the video and its processed transcription) and Transcribe
  • curl - for downloading from URLs
  • jq - for parsing JSON data
  • ffmpeg - for media file conversion, e.g. extracting mp3 audio from video

The steps

  • Find a tweet containing a video you like
  • Get that tweet's URL, e.g. https://twitter.com/JordanUhl/status/1085669288051175424
  • Use youtube-dl to download the video from that tweet and save it to disk, e.g. cardib.mp4
  • Because AWS Transcribe requires we send it an audio file, use ffmpeg to extract the audio from cardib.mp4 and save it to cardib.mp3
  • Because AWS Transcribe only works on audio files stored on AWS S3, we use awscli to upload cardiob.mp3 to an online S3 bucket, eg. http://data.danwin.com/tmp/cardib.mp3
  • Use awscli to access the AWS Transcribe API and start a transcription job
  • Wait a couple of minutes
  • Use awscli to get the details of the initiated (and hopefully, finished) transcription job
  • Use jq
Raw
start-transcript-job-response.json
{
"TranscriptionJob": {
"TranscriptionJobName": "cardib-politics",
"TranscriptionJobStatus": "IN_PROGRESS",
"LanguageCode": "en-US",
"MediaFormat": "mp3",
"Media": {
"MediaFileUri": "s3://data.danwin.com/tmp/cardib.mp3"
},
"CreationTime": 1547794736.719
}
}
@briankung
Copy link

briankung commented Oct 9, 2020

Wow, I really owe you a coffee sometime. I feel like I find myself in your footsteps all the time. Thanks for posting this!

@briankung
Copy link

briankung commented Oct 9, 2020

Oh wow, I wish the speaker label was embedded in each item. It'd be a lot nicer for differentiating who said what

@briankung
Copy link

briankung commented Oct 9, 2020 •
edited
Loading

So in my case, I've used the following to verify that the item start times and the speaker start times are the same:

TRANSCRIPTION_JSON=path/to/downloaded/file
cat $TRANSCRIPTION_JSON | jq '.results.items |  map([.start_time]) | flatten | del(.[] | nulls)' > item_start_times
cat $TRANSCRIPTION_JSON | jq '.results.speaker_labels.segments | map([.items]) | flatten | map([.start_time]) | flatten' > speaker_start_times

# Diff the files backwards and forwards...

diff item_start_times speaker_start_times
diff speaker_start_times item_start_times

# Oh right, I can just compare sha hashes (on macOS):

$ shasum -a 256 *_start_times
2642201229098c2cb926c7cf8999c7a4070745d94a5a47155ac983a56db8f4ca  item_start_times
2642201229098c2cb926c7cf8999c7a4070745d94a5a47155ac983a56db8f4ca  speaker_start_times

# ...and they're identical

So in my case, the speaker start times were a subset of the item start times (which had more items, some of which had no start_time key, hence the del(.[] | nulls) jq filter). Still, I'm pretty confident that they line up for the most part.

I'm thinking about using the start times to map the transcribed words to the speakers so I can organize the transcript better.

@dannguyen
Copy link
Author

dannguyen commented Oct 9, 2020

@briankung so I think this gist is pretty old, and I'm unsure of how feature-complete the Transcribe API was when I wrote it.

In any case, I just popped open the Transcribe console and did a test of the real-time streaming endpoint (couldn't find the demo interface for uploading audio files for transcription), and it appears that speaker identification is included in the JSON response:

https://gist.github.com/dannguyen/92990a177d511bdd055ec3817da85238

I can't imagine the response for the non-streaming endpoint not having speaker identification. Too lazy right now to look in the API docs but I'm sure there's a flag for it

@briankung
Copy link

briankung commented Oct 9, 2020 via email

@dannguyen
Copy link
Author

dannguyen commented Oct 10, 2020 •
edited
Loading

@briankung sorry I'm dumb. didn't even read my old gist that had the Senate example. Looks like there is speaker identification #file-transcript-senate-bennett-json, but you were asking if it could be embedded with each transcribed item instead of its own object in the JSON that you then have to process/align on your own. Yeah I'd be surprised if they've changed the output format of this transcribe-job API to include that now ¯\_(ツ)_/¯

@briankung
Copy link

briankung commented Oct 10, 2020

Yeah I tried on my own test file, a podcast, and it didn’t include the speaker identification information in the transcribed items. And no worries! Thanks for posting in the first place, helped a lot.

@rlau1115
Copy link

rlau1115 commented Oct 18, 2020 •
edited
Loading

Thanks for sharing this! Looking to adapt this for a video editing workflow. Do you happen to know if the transcribe API can handle multiple languages at once?

@briankung
Copy link

briankung commented Oct 18, 2020

After poking around with it for a bit, it doesn't seem like it supports multiple languages in a single audio file, but further research may provide more answers. I think you mostly supply a --language-code for the primary language or let it identify the primary language.

What do you mean by multiple languages at once, though?

@rlau1115
Copy link

rlau1115 commented Oct 19, 2020 via email

@briankung
Copy link

briankung commented Oct 19, 2020 via email •
edited
Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment