Skip to content

Instantly share code, notes, and snippets.

@0xdevalias
Last active October 23, 2025 01:26
Show Gist options
  • Save 0xdevalias/359f4265adf03b0142e4d0543c156a3e to your computer and use it in GitHub Desktop.
Save 0xdevalias/359f4265adf03b0142e4d0543c156a3e to your computer and use it in GitHub Desktop.

Revisions

  1. 0xdevalias revised this gist May 25, 2024. 1 changed file with 16 additions and 0 deletions.
    16 changes: 16 additions & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -84,6 +84,22 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    - > SoftVC VITS Singing Voice Conversion [Archived]
    - https://github.com/voicepaw/so-vits-svc-fork
    - > `so-vits-svc` fork with realtime support, improved interface and more features
    - https://github.com/PlayVoice/so-vits-svc-5.0
    - > so-vits-svc-5.0
    - > Core Engine of Singing Voice Conversion & Singing Voice Clone
    - > Variational Inference with adversarial learning for end-to-end Singing Voice Conversion based on VITS
    - https://github.com/PlayVoice/so-vits-svc-5.0#data-set
    - https://github.com/PlayVoice/so-vits-svc-5.0#code-sources-and-references
    - https://github.com/PlayVoice/lora-svc
    - > lora-svc
    - > Singing voice change based on whisper, and lora for singing voice clone
    - > Singing Voice Conversion based on Whisper & neural source-filter BigVGAN
    - > LoRA is not fully implemented in this project, but it can be found here: LoRA TTS & paper
    - https://github.com/PlayVoice/VI-SVS
    - > VI-SVS
    - > Singing Voice Synthesis based on VITS, different from VISinger
    - > Variational Inference with adversarial learning for end-to-end Singing Voice Synthesis
    - > Different from VISinger, It is just VITS without MAS and DurationPredictor.
    - https://github.com/quickvc/QuickVC-VoiceConversion
    - > QuickVC: Any-to-many Voice Conversion Using Inverse Short-time Fourier Transform for Faster Conversion
    - https://github.com/auspicious3000/contentvec
  2. 0xdevalias revised this gist Apr 15, 2024. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -170,6 +170,7 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    - [Music APIs and DBs (0xdevalias' gist)](https://gist.github.com/0xdevalias/eba698730024674ecae7f43f4c650096#music-apis-and-dbs)
    - [Singing Voice Synthesizers (eg. Vocaloid, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/0b64b25d72cbbc784042a9fdff713129#singing-voice-synthesizers-eg-vocaloid-etc)
    - [Audio Pitch Correction (eg. autotune, melodyne, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/7f4a5c31758e04aea5c2f5520e53accb#audio-pitch-correction-eg-autotune-melodyne-etc)
    - [Automated Audio Transcription (AAT) / Automated Music Transcription (AMT) (aka: converting audio to midi) (0xdevalias' gist)](https://gist.github.com/0xdevalias/f2c6e52824b3bbd4fb4c84c603a3f4bd#automated-audio-transcription-aat--automated-music-transcription-amt-aka-converting-audio-to-midi)
    - [Generating Synth Patches with AI (0xdevalias' gist)](https://gist.github.com/0xdevalias/5a06349b376d01b2a76ad27a86b08c1b#generating-synth-patches-with-ai)
    - [Compare/Diff Audio Files (0xdevalias' gist)](https://gist.github.com/0xdevalias/91ae33e0c9290e69b457ce5034956fb7#comparediff-audio-files)
    - [Working Around FLStudio Trial Limitations (0xdevalias' gist)](https://gist.github.com/0xdevalias/7c51aa2d862da34baaeecafb157e726c#working-around-flstudio-trial-limitations)
  3. 0xdevalias revised this gist Jan 8, 2024. 1 changed file with 9 additions and 0 deletions.
    9 changes: 9 additions & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -61,6 +61,15 @@ My random collection of notes on AI voice cloning services/models/techniques/etc

    ## Tools

    - https://github.com/myshell-ai/OpenVoice
    - > Instant voice cloning by MyShell
    - https://research.myshell.ai/open-voice
    - > OpenVoice: Versatile Instant Voice Cloning
    - https://arxiv.org/abs/2312.01479
    - > OpenVoice: Versatile Instant Voice Cloning
    - > We introduce OpenVoice, a versatile voice cloning approach that requires only a short audio clip from the reference speaker to replicate their voice and generate speech in multiple languages. OpenVoice represents a significant advancement in addressing the following open challenges in the field: 1) Flexible Voice Style Control. OpenVoice enables granular control over voice styles, including emotion, accent, rhythm, pauses, and intonation, in addition to replicating the tone color of the reference speaker. The voice styles are not directly copied from and constrained by the style of the reference speaker. Previous approaches lacked the ability to flexibly manipulate voice styles after cloning. 2) Zero-Shot Cross-Lingual Voice Cloning. OpenVoice achieves zero-shot cross-lingual voice cloning for languages not included in the massive-speaker training set. Unlike previous approaches, which typically require extensive massive-speaker multi-lingual (MSML) dataset for all languages, OpenVoice can clone voices into a new language without any massive-speaker training data for that language. OpenVoice is also computationally efficient, costing tens of times less than commercially available APIs that offer even inferior performance. To foster further research in the field, we have made the source code and trained model publicly accessible. We also provide qualitative results in our demo website. Prior to its public release, our internal version of OpenVoice was used tens of millions of times by users worldwide between May and October 2023, serving as the backend of MyShell.
    - https://github.com/camenduru/OpenVoice-colab
    - > OpenVoice-colab
    - https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI
    - > Retrieval-based-Voice-Conversion-WebUI
    - https://github.com/SociallyIneptWeeb/AICoverGen
  4. 0xdevalias revised this gist Jan 1, 2024. 1 changed file with 19 additions and 0 deletions.
    19 changes: 19 additions & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -2,6 +2,23 @@

    My random collection of notes on AI voice cloning services/models/techniques/etc. Just because something is listed here, doesn't necessarily mean I have tried it, nor endorse it. Use this as a starting point for doing your own further research.

    ## Table of Contents

    <!-- TOC start (generated with https://derlin.github.io/bitdowntoc/) -->
    - [Services](#services)
    - [Guides, Colabs, etc](#guides-colabs-etc)
    - [Tools](#tools)
    - [Character Voices](#character-voices)
    - [Bowser](#bowser)
    - [Peach](#peach)
    - [Ellie (The Last of Us, TLOU)](#ellie-the-last-of-us-tlou)
    - [Ideas to Create](#ideas-to-create)
    - [Zoe (League of Legends)](#zoe-league-of-legends)
    - [Annie (League of Legends)](#annie-league-of-legends)
    - [Vanellope von Schweetz (Wreck it Ralph)](#vanellope-von-schweetz-wreck-it-ralph)
    - [See Also](#see-also)
    <!-- TOC end -->

    ## Services

    - https://elevenlabs.io/
    @@ -141,7 +158,9 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    ## See Also

    - [Music APIs and DBs (0xdevalias' gist)](https://gist.github.com/0xdevalias/eba698730024674ecae7f43f4c650096#music-apis-and-dbs)
    - [Singing Voice Synthesizers (eg. Vocaloid, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/0b64b25d72cbbc784042a9fdff713129#singing-voice-synthesizers-eg-vocaloid-etc)
    - [Audio Pitch Correction (eg. autotune, melodyne, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/7f4a5c31758e04aea5c2f5520e53accb#audio-pitch-correction-eg-autotune-melodyne-etc)
    - [Generating Synth Patches with AI (0xdevalias' gist)](https://gist.github.com/0xdevalias/5a06349b376d01b2a76ad27a86b08c1b#generating-synth-patches-with-ai)
    - [Compare/Diff Audio Files (0xdevalias' gist)](https://gist.github.com/0xdevalias/91ae33e0c9290e69b457ce5034956fb7#comparediff-audio-files)
    - [Working Around FLStudio Trial Limitations (0xdevalias' gist)](https://gist.github.com/0xdevalias/7c51aa2d862da34baaeecafb157e726c#working-around-flstudio-trial-limitations)
  5. 0xdevalias revised this gist Oct 17, 2023. 1 changed file with 18 additions and 0 deletions.
    18 changes: 18 additions & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -4,6 +4,22 @@ My random collection of notes on AI voice cloning services/models/techniques/etc

    ## Services

    - https://elevenlabs.io/
    - https://elevenlabs.io/speech-synthesis
    - > The First Generative Speech Synthesis Platform
    > Generate lifelike speech in any language and voice with the most powerful Text to Speech and Voice Cloning software.
    - https://elevenlabs.io/voice-lab
    - > Generative Voice AI
    > Clone your voice or create entirely new synthetic voices using the most advanced Generative AI technology ever.
    - https://elevenlabs.io/voice-library
    - > Discover AI Voices Crafted by the Community
    > Get access to an ever-growing library of high quality AI voices and discover characters that perfectly fit your needs.
    - https://elevenlabs.io/professional-voice-cloning
    - > Professional Voice Cloning
    > Create the perfect digital replica of your voice using the most advanced voice cloning AI. We create AI models on your voice from the grounds up to offer the most realistic voice cloning experience ever.
    - https://elevenlabs.io/projects
    - > Your Audiobook Workshop
    > Generate, edit, and customize long-form spoken audio with precision, all within a streamlined workflow.
    - https://create.musicfy.lol/
    - https://create.musicfy.lol/parody
    - https://lalals.com/voices/
    @@ -14,6 +30,8 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    - https://app.kits.ai/community-voices
    - https://app.kits.ai/convert/royalty-free
    - https://app.kits.ai/library/trained
    - https://play.ht/voice-cloning/
    - https://speechify.com/voice-cloning/

    ## Guides, Colabs, etc

  6. 0xdevalias revised this gist Oct 11, 2023. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -124,5 +124,6 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    ## See Also

    - [Singing Voice Synthesizers (eg. Vocaloid, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/0b64b25d72cbbc784042a9fdff713129#singing-voice-synthesizers-eg-vocaloid-etc)
    - [Audio Pitch Correction (eg. autotune, melodyne, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/7f4a5c31758e04aea5c2f5520e53accb#audio-pitch-correction-eg-autotune-melodyne-etc)
    - [Generating Synth Patches with AI (0xdevalias' gist)](https://gist.github.com/0xdevalias/5a06349b376d01b2a76ad27a86b08c1b#generating-synth-patches-with-ai)
    - [Working Around FLStudio Trial Limitations (0xdevalias' gist)](https://gist.github.com/0xdevalias/7c51aa2d862da34baaeecafb157e726c#working-around-flstudio-trial-limitations)
  7. 0xdevalias revised this gist Sep 12, 2023. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -124,4 +124,5 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    ## See Also

    - [Singing Voice Synthesizers (eg. Vocaloid, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/0b64b25d72cbbc784042a9fdff713129#singing-voice-synthesizers-eg-vocaloid-etc)
    - [Generating Synth Patches with AI (0xdevalias' gist)](https://gist.github.com/0xdevalias/5a06349b376d01b2a76ad27a86b08c1b#generating-synth-patches-with-ai)
    - [Generating Synth Patches with AI (0xdevalias' gist)](https://gist.github.com/0xdevalias/5a06349b376d01b2a76ad27a86b08c1b#generating-synth-patches-with-ai)
    - [Working Around FLStudio Trial Limitations (0xdevalias' gist)](https://gist.github.com/0xdevalias/7c51aa2d862da34baaeecafb157e726c#working-around-flstudio-trial-limitations)
  8. 0xdevalias revised this gist Sep 11, 2023. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -123,4 +123,5 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    ## See Also

    - [Singing Voice Synthesizers (eg. Vocaloid, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/0b64b25d72cbbc784042a9fdff713129#singing-voice-synthesizers-eg-vocaloid-etc)
    - [Singing Voice Synthesizers (eg. Vocaloid, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/0b64b25d72cbbc784042a9fdff713129#singing-voice-synthesizers-eg-vocaloid-etc)
    - [Generating Synth Patches with AI (0xdevalias' gist)](https://gist.github.com/0xdevalias/5a06349b376d01b2a76ad27a86b08c1b#generating-synth-patches-with-ai)
  9. 0xdevalias revised this gist Sep 9, 2023. 1 changed file with 5 additions and 1 deletion.
    6 changes: 5 additions & 1 deletion ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -119,4 +119,8 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    > - https://www.sounds-resource.com/ds_dsi/wreckitralph/sound/3249/
    > - https://www.youtube.com/watch?v=jnp2ObnVt9M
    >
    > ([Ref](https://discord.com/channels/1089076875999072296/1143346872782700644/1143346872782700644))
    > ([Ref](https://discord.com/channels/1089076875999072296/1143346872782700644/1143346872782700644))
    ## See Also

    - [Singing Voice Synthesizers (eg. Vocaloid, etc) (0xdevalias' gist)](https://gist.github.com/0xdevalias/0b64b25d72cbbc784042a9fdff713129#singing-voice-synthesizers-eg-vocaloid-etc)
  10. 0xdevalias revised this gist Aug 22, 2023. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -22,6 +22,7 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    - [Colab: AICoverGen WebUI](https://colab.research.google.com/github/SociallyIneptWeeb/AICoverGen/blob/main/AICoverGen_colab.ipynb)
    - [Training RVC v2 models Guide (by kalomaze)](https://docs.google.com/document/d/13ebnzmeEBc6uzYCMt-QVFQk-whVrK4zw8k7_Lw3Bv_A/edit#heading=h.bjzhhhcn3f69)
    - [RVC Crepe v2 + persistent storage / autosave](https://colab.research.google.com/drive/1TU-kkQWVf-PLO_hSa2QCMZS1XF5xVHqs?usp=sharing)
    - [RVC Models Archive Sheet](https://docs.google.com/spreadsheets/d/1tAUaQrEHYgRsm1Lvrnj14HFHDwJWl0Bd9x0QePewNco/edit#gid=1977693859)

    ## Tools

  11. 0xdevalias revised this gist Aug 22, 2023. 1 changed file with 4 additions and 0 deletions.
    4 changes: 4 additions & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -10,6 +10,10 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    - https://www.uberduck.ai/
    - https://voice.ai/
    - https://www.myvocal.ai/
    - https://app.kits.ai/
    - https://app.kits.ai/community-voices
    - https://app.kits.ai/convert/royalty-free
    - https://app.kits.ai/library/trained

    ## Guides, Colabs, etc

  12. 0xdevalias revised this gist Aug 22, 2023. 1 changed file with 19 additions and 0 deletions.
    19 changes: 19 additions & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -46,6 +46,25 @@ My random collection of notes on AI voice cloning services/models/techniques/etc
    ## Character Voices

    ### Bowser

    > **Bowser (Jack Black) (Super Mario Bros.) (RVC V2) 950 Epochs**
    >
    > 40k, crepe: https://huggingface.co/yeey5/BowserRVCV2/resolve/main/Bowser.zip
    > 48k, mangio-crepe: https://huggingface.co/yeey5/BowserRVCV2/resolve/main/Bowser48k.zip
    >
    > ([Ref](https://discord.com/channels/1089076875999072296/1120475730527141908/1120475730527141908))
    ### Peach

    > **Princess Peach (Anya Taylor Joy) RVC v2 300 Epochs**
    >
    > I decided to create a voice model of her because there's never been a voice model of any taylor joy
    >
    > https://app.kits.ai/convert/shared/anya-taylor-joy-1
    >
    > ([Ref](https://discord.com/channels/1089076875999072296/1139700444571123873/1139700444571123873))
    ### Ellie (The Last of Us, TLOU)

    > **Ellie TLOU RVC V2 300 Epochs**
  13. 0xdevalias created this gist Aug 22, 2023.
    98 changes: 98 additions & 0 deletions ai-voice-cloning-transfer.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,98 @@
    # AI Voice Cloning / Transfer (eg. RVCv2)

    My random collection of notes on AI voice cloning services/models/techniques/etc. Just because something is listed here, doesn't necessarily mean I have tried it, nor endorse it. Use this as a starting point for doing your own further research.

    ## Services

    - https://create.musicfy.lol/
    - https://create.musicfy.lol/parody
    - https://lalals.com/voices/
    - https://www.uberduck.ai/
    - https://voice.ai/
    - https://www.myvocal.ai/

    ## Guides, Colabs, etc

    - [RVC v2 AI Cover Guide (by kalomaze)](https://docs.google.com/document/d/13_l1bd1Osgz7qlAZn-zhklCbHpVRk6bYOuAuB78qmsE/edit#heading=h.qjrl2d41vtmt)
    - [Colab: Easy GUI (for RVC v2, with crepe) (with improved downloader)](https://colab.research.google.com/drive/1Gj6UTf2gicndUW_tVheVhTXIIYpFTYc7?usp=sharing)
    - [Colab: AICoverGen WebUI](https://colab.research.google.com/github/SociallyIneptWeeb/AICoverGen/blob/main/AICoverGen_colab.ipynb)
    - [Training RVC v2 models Guide (by kalomaze)](https://docs.google.com/document/d/13ebnzmeEBc6uzYCMt-QVFQk-whVrK4zw8k7_Lw3Bv_A/edit#heading=h.bjzhhhcn3f69)
    - [RVC Crepe v2 + persistent storage / autosave](https://colab.research.google.com/drive/1TU-kkQWVf-PLO_hSa2QCMZS1XF5xVHqs?usp=sharing)

    ## Tools

    - https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI
    - > Retrieval-based-Voice-Conversion-WebUI
    - https://github.com/SociallyIneptWeeb/AICoverGen
    - > A WebUI to create song covers with any RVC v2 trained AI voice from YouTube videos or audio files.
    - https://github.com/w-okada/voice-changer
    - > Realtime Voice Changer
    - https://github.com/yxlllc/DDSP-SVC
    - > Real-time end-to-end singing voice conversion system based on DDSP (Differentiable Digital Signal Processing)
    - https://github.com/isletennos/MMVC_Trainer
    - > Real-time voice changer using AI (Trainer)
    - https://github.com/svc-develop-team/so-vits-svc
    - > SoftVC VITS Singing Voice Conversion [Archived]
    - https://github.com/voicepaw/so-vits-svc-fork
    - > `so-vits-svc` fork with realtime support, improved interface and more features
    - https://github.com/quickvc/QuickVC-VoiceConversion
    - > QuickVC: Any-to-many Voice Conversion Using Inverse Short-time Fourier Transform for Faster Conversion
    - https://github.com/auspicious3000/contentvec
    - > ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers
    - https://arxiv.org/abs/2204.09224
    - https://github.com/uberduck-ai/uberduck-ml-dev
    - > Uberduck Synthetic Speech
    - > ML models for Uberduck
    ## Character Voices

    ### Ellie (The Last of Us, TLOU)

    > **Ellie TLOU RVC V2 300 Epochs**
    >
    > Ellie from the last of us part 1
    > (RVC V2) 300 epochs trained with a 10 minute dataset , voice lines from the game
    >
    > https://huggingface.co/TJKAI/EllieLOU/resolve/main/Ellie.rar
    >
    > ([Ref](https://discord.com/channels/1089076875999072296/1128416105229189180/1128416105229189180))
    > **Ellie (The Last of Us Part 1) (RVC v2, 119e/8806s)**
    >
    > This model is based on the younger version of Ellie from The Last of Us (video game). This model only exists because I failed to appreciate how different Ellie sounds at different parts of Part 2, but I thought I'd share it anyway because it sounds pretty good. There are other young Ellie voices that I think are better though. You should probably use this one by <@554576184101306368> instead: https://discord.com/channels/1089076875999072296/1128416105229189180/1128416105229189180 Here's my model of the older Ellie from Part 2: https://discord.com/channels/1089076875999072296/1136823620618964992/1136823620618964992
    >
    > https://huggingface.co/Grimoire-VC/Voices/resolve/main/EllieP1.zip
    >
    > ([Ref](https://discord.com/channels/1089076875999072296/1136823365605261372/1136823365605261372))
    > **Ellie (The Last of Us Part 2) (RVC v2, 203e/14210s)**
    >
    > This model is based on the older version of Ellie from the Last of Us Part 2. I'm pretty happy with it. Ellie is usually pretty breathy and I wasn't able to capture that, but I do think I got her tone of voice about right. I don't think there's another Part 2 Ellie on here yet. I also made a Part 1 Ellie that I'm uploading at the same time here: https://discord.com/channels/1089076875999072296/1136823365605261372/1136823365605261372
    >
    > https://huggingface.co/Grimoire-VC/Voices/resolve/main/EllieP2.zip
    >
    > ([Ref](https://discord.com/channels/1089076875999072296/1136823620618964992/1136823620618964992))
    ### Ideas to Create

    #### Zoe (League of Legends)

    > Voice samples here: https://leagueoflegends.fandom.com/wiki/Zoe/LoL/Audio
    >
    > ([Ref](https://discord.com/channels/1089076875999072296/1143035328450539541/1143035328450539541))
    #### Annie (League of Legends)

    > Voice samples here: https://leagueoflegends.fandom.com/wiki/Annie/LoL/Audio
    >
    > ([Ref](https://discord.com/channels/1089076875999072296/1143125784240857180/1143125784240857180))
    #### Vanellope von Schweetz (Wreck it Ralph)

    > A few sources of sound clips I found from a quick google:
    > - https://www.soundboard.com/sb/VanellopeVonSchweetz
    > - https://www.sounds-resource.com/pc_computer/disneyinfinity/sound/12086/
    > - https://www.sounds-resource.com/ds_dsi/wreckitralph/sound/3249/
    > - https://www.youtube.com/watch?v=jnp2ObnVt9M
    >
    > ([Ref](https://discord.com/channels/1089076875999072296/1143346872782700644/1143346872782700644))