Skip to content

Instantly share code, notes, and snippets.

@William9923

William9923/-ta-archive-scripts.ipynb

Last active March 13, 2022 05:07
Show Gist options
  • Save William9923/e1bd645e235e00ae6d871ab62ddd7d21 to your computer and use it in GitHub Desktop.
Save William9923/e1bd645e235e00ae6d871ab62ddd7d21 to your computer and use it in GitHub Desktop.
Download ZIP
Kaggle To Drive Scripts
Display the source blob
Display the rendered blob
Raw
-ta-archive-scripts.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "[TA] Archive Scripts.ipynb",
"provenance": [],
"collapsed_sections": [],
"authorship_tag": "ABX9TyMCE1xhBtDgMUwbcXXM1Iwt",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/William9923/e1bd645e235e00ae6d871ab62ddd7d21/-ta-archive-scripts.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"source": [
"# Kaggle Output Downloader Scripts"
],
"metadata": {
"id": "hzvs0VuKngZu"
}
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "rIGhQC780R50",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "33ec3ffc-baf2-4871-aef3-3f98f4f46e3e"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
]
}
],
"source": [
"from google.colab import drive\n",
"drive.mount('/content/drive')"
]
},
{
"cell_type": "code",
"source": [
"# Getting Data -> upload kaggle api data\n",
"# Next, install the Kaggle API client.\n",
"!pip install -q kaggle\n",
"\n",
"# The Kaggle API client expects this file to be in ~/.kaggle,\n",
"# so move it there.\n",
"!mkdir -p ~/.kaggle\n",
"!cp drive/MyDrive/TA/kaggle.json ~/.kaggle/\n",
"\n",
"# This permissions change avoids a warning on Kaggle tool startup.\n",
"!chmod 600 ~/.kaggle/kaggle.json"
],
"metadata": {
"id": "GcZTS9g6bXql"
},
"execution_count": 20,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!mkdir drive/MyDrive/TA/temp"
],
"metadata": {
"id": "d0zjgfein3IF"
},
"execution_count": 21,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# TODO: modify the output target\n",
"!kaggle kernels output williamong/exp-p4-1 -p drive/MyDrive/TA/temp"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "OuLwoKzAcds6",
"outputId": "16a534b4-cbc6-4e7d-cb4d-927a6e544816"
},
"execution_count": 22,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/HEAD\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/config\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/description\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/applypatch-msg.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/commit-msg.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/fsmonitor-watchman.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/post-update.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/pre-applypatch.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/pre-commit.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/pre-merge-commit.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/pre-push.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/pre-rebase.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/pre-receive.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/prepare-commit-msg.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/hooks/update.sample\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/index\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/info/exclude\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/logs/HEAD\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/logs/refs/heads/master\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/logs/refs/remotes/origin/HEAD\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/objects/pack/pack-b6fd3da4e282bfd0144d6fddc28588d86bc642bd.idx\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/objects/pack/pack-b6fd3da4e282bfd0144d6fddc28588d86bc642bd.pack\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/packed-refs\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/refs/heads/master\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.git/refs/remotes/origin/HEAD\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.github/pull_request_template.md\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.github/workflows/codeql-analysis.yml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.gitignore\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.vscode/launch.json\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/.vscode/settings.json\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/LICENSE\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/Makefile\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/README.md\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/__pycache__/args.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/app.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/args.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/.gitkeep\n",
"tcmalloc: large alloc 2970722304 bytes == 0x563f938e0000 @ 0x7f2b932391e7 0x563ebb02c518 0x563ebaff6d17 0x563ebaff8d00 0x563ebaffa1cd 0x563ebb0ecb3d 0x563ebb06e458 0x563ebaf3ad14 0x563ebb0ecc51 0x563ebb11a639 0x563ebb06a017 0x563ebb06902f 0x563ebaffc36c 0x563ebaffc571 0x563ebb06b633 0x563ebb06902f 0x563ebaffc36c 0x563ebaffc571 0x563ebb06b633 0x563ebb06902f 0x563ebaf3ae2b 0x563ebb06b633 0x563ebb06902f 0x563ebaffbaba 0x563ebb06e2c0 0x563ebb06902f 0x563ebaffbaba 0x563ebb06a108 0x563ebb06902f 0x563ebaffc36c 0x563ebaffc571\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/model-best.pt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/train/false-triplets.csv\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/train/result.csv\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/train/score.csv\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/train/triplets.csv\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/val/false-triplets.csv\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/val/result.csv\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/val/score.csv\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/val/triplets.csv\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/data/filter/dev.txt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/data/filter/test.txt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/data/filter/train.txt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/data/post-train/test_pt.txt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/data/post-train/train_pt.txt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/data/unfilter/dev.txt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/data/unfilter/test.txt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/data/unfilter/train.txt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/docs/normalization.md\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/docs/timeline.md\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/main.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/posttraining.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/requirements.txt\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p0-1.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p0-2.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p1-2.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p2-1.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p2-2.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p2-3.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p3-1.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p3-2.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p3-3-test.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p3-3.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p4-1.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p4-2.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/exp/exp-p4-3.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/t5-config-dummy.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/t5-config-embedding-dummy.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/t5-config-eval.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/t5-config-filter-eval.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/t5-config-filter.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/resources/t5-config.yaml\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/scripts/hooks/pre-commit\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/scripts/hooks/setup.sh\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/setup.sh\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/__pycache__/constant.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/__pycache__/evaluation.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/__pycache__/utility.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/constant.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/evaluation.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/frontend/__init__.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/frontend/download.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/frontend/run.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/frontend/utils.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/generator/__init__.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/generator/__pycache__/__init__.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/generator/__pycache__/generation.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/generator/__pycache__/interface.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/generator/generation.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/generator/interface.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/__init__.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/__pycache__/__init__.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/__pycache__/dataset.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/__pycache__/interface.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/__pycache__/loader.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/__pycache__/parser.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/dataset.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/interface.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/loader.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/loader/parser.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/postprocess/__init__.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/postprocess/__pycache__/__init__.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/postprocess/__pycache__/editdistance.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/postprocess/__pycache__/embedding.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/postprocess/__pycache__/interface.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/postprocess/editdistance.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/postprocess/embedding.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/postprocess/interface.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/trainer/__init__.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/trainer/__pycache__/__init__.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/trainer/__pycache__/interface.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/trainer/__pycache__/t5.cpython-37.pyc\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/trainer/interface.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/trainer/t5.py\n",
"Output file downloaded to drive/MyDrive/TA/temp/generative-aste/src/utility.py\n",
"Kernel log downloaded to drive/MyDrive/TA/temp/exp-p4-1.log \n"
]
}
]
},
{
"cell_type": "code",
"source": [
"# TODO: modify the Drive targets\n",
"!cp -r drive/MyDrive/TA/temp/generative-aste/bin/exp-p4-1/ drive/MyDrive/TA/v1/"
],
"metadata": {
"id": "xgmzlRGWcsuh"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!rm -rf drive/MyDrive/TA/temp"
],
"metadata": {
"id": "K4GJ8KZDgJmT"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
""
],
"metadata": {
"id": "78SBuMVinwHr"
},
"execution_count": null,
"outputs": []
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment