Skip to content

Instantly share code, notes, and snippets.

@yf225

yf225/torchtrain_toy_model_compile_log_CUDA_IMA.txt

Created March 27, 2024 22:06
Show Gist options
  • Save yf225/0fc078e54e7419c3bd1278935fa072d5 to your computer and use it in GitHub Desktop.
Save yf225/0fc078e54e7419c3bd1278935fa072d5 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
torchtrain_toy_model_compile_log_CUDA_IMA.txt
This file has been truncated, but you can view the full file.
+ export USE_LIBUV=1
+ USE_LIBUV=1
+ TRAINER_DIR=/home/willfeng/local/torchtrain
+ NGPU=8
+ LOG_RANK=0
+ CONFIG_FILE=./train_configs/toy_model_full_graph_fsdp.toml
+ torchrun --nproc_per_node=8 --rdzv_endpoint=localhost:5972 --local-ranks-filter 0 --role rank --tee 3 train.py --job.config_file ./train_configs/toy_model_full_graph_fsdp.toml
W2024-03-27 14:57:24,673.673000 140480499401728 torch/distributed/run.py:757]
W2024-03-27 14:57:24,673.673000 140480499401728 torch/distributed/run.py:757] *****************************************
W2024-03-27 14:57:24,673.673000 140480499401728 torch/distributed/run.py:757] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
W2024-03-27 14:57:24,673.673000 140480499401728 torch/distributed/run.py:757] *****************************************
[rank0]:2024-03-27 14:57:26,876 - root - INFO - Starting job: LLaMA debug training
[rank0]:2024-03-27 14:57:29,498 - root - WARNING - ENV[TORCH_NCCL_ASYNC_ERROR_HANDLING] = 1 will be overridden to 3 based on job config
[rank0]:2024-03-27 14:57:29,508 - root - INFO - Building 1-D device mesh with ['dp'], [8]
[rank0]:2024-03-27 14:57:29,513 - root - INFO - Building sentencepiece tokenizer locally from ./torchtrain/datasets/tokenizer/tokenizer.model
[rank0]:2024-03-27 14:57:29,520 - root - INFO - SentencePieceTokenizer built: #words 32000, BOS ID 1, EOS ID 2
[rank0]:2024-03-27 14:57:29,520 - root - INFO - Preparing alpaca dataset from HuggingFace
[rank0]:2024-03-27 14:57:31,889 - root - INFO - Building llama toymodel with ModelArgs(dim=256, n_layers=1, n_heads=1, n_kv_heads=None, vocab_size=32000, multiple_of=256, ffn_dim_multiplier=None, norm_eps=1e-05, max_batch_size=32, max_seq_len=32768, depth_init=True)
[rank0]:2024-03-27 14:57:31,900 - root - INFO - Model llama toymodel size: 17,236,736 total parameters
[rank0]:2024-03-27 14:57:31,900 - root - INFO - GPU capacity: NVIDIA H100 (0) with 95.04GiB memory
[rank0]:2024-03-27 14:57:31,906 - root - INFO - Applied FSDP to the model
[rank0]:2024-03-27 14:57:44,590 - root - INFO - Gradient scaling not enabled
[rank0]:2024-03-27 14:57:44,590 - root - INFO - Metrics logging active. Tensorboard logs will be saved at ./outputs/tb/20240327-1457
[rank0]:2024-03-27 14:57:44,591 - root - INFO - Compiling model with torch.compile + full-graph compile FSDP
[rank0]:2024-03-27 14:57:44,600 - root - INFO - Profiling active. Traces will be saved at ./outputs/profiling/traces
[rank0]:2024-03-27 14:57:45,297 - root - INFO - step: 1 loss: 10.7914 memory: 9.33GiB(9.82%) wps: 23,530 mfu: 0.26%
[rank0]:2024-03-27 14:57:45,298 - root - INFO - Synchronizing and adjusting timeout for all ProcessGroups to 0:00:05
[rank0]:[rank0]:I2024-03-27 14:57:45,899.899000 140555295757312 torch/_dynamo/logging.py:55] [0/0] Step 1: torchdynamo start tracing inner /data/users/willfeng/pytorch_yf225/torch/_dynamo/external_utils.py:35
[rank0]:[rank0]:V2024-03-27 14:57:45,900.900000 140555295757312 torch/fx/experimental/symbolic_shapes.py:1980] [0/0] create_env
[rank0]:[rank0]:V2024-03-27 14:57:45,904.904000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COPY_FREE_VARS 1 []
[rank0]:[rank0]:V2024-03-27 14:57:45,904.904000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/_dynamo/external_utils.py:35 in inner (wrap_inline)
[rank0]:[rank0]:V2024-03-27 14:57:45,904.904000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] @functools.wraps(fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,905.905000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,905.905000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/_dynamo/external_utils.py:37 in inner (wrap_inline.inner)
[rank0]:[rank0]:V2024-03-27 14:57:45,905.905000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return fn(*args, **kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,905.905000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PUSH_NULL None []
[rank0]:[rank0]:V2024-03-27 14:57:45,905.905000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF fn [NullVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,905.905000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST args [NullVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,905.905000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_MAP 0 [NullVariable(), LazyVariableTracker(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,905.905000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST kwargs [NullVariable(), LazyVariableTracker(), LazyVariableTracker(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,905.905000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE DICT_MERGE 1 [NullVariable(), LazyVariableTracker(), LazyVariableTracker(), ConstDictVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,906.906000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL_FUNCTION_EX 1 [NullVariable(), LazyVariableTracker(), LazyVariableTracker(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _call_impl from /data/users/willfeng/pytorch_yf225/torch/_dynamo/external_utils.py:37 in inner (wrap_inline.inner)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return fn(*args, **kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _call_impl at 0x484a600, file "/data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py", line 1529>
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1529 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1530 2 LOAD_GLOBAL 0 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_ATTR 1 (_C)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 LOAD_METHOD 2 (_get_tracing_state)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 POP_JUMP_FORWARD_IF_FALSE 7 (to 76)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 62 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 LOAD_ATTR 3 (_slow_forward)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 JUMP_FORWARD 6 (to 88)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 76 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 78 LOAD_ATTR 4 (forward)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 88 STORE_FAST 3 (forward_call)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1533 90 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 92 LOAD_ATTR 5 (_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 102 POP_JUMP_FORWARD_IF_TRUE 57 (to 218)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 104 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 LOAD_ATTR 6 (_backward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 POP_JUMP_FORWARD_IF_TRUE 50 (to 218)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 118 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 LOAD_ATTR 7 (_forward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 130 POP_JUMP_FORWARD_IF_TRUE 43 (to 218)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 132 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 LOAD_ATTR 8 (_forward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 POP_JUMP_FORWARD_IF_TRUE 36 (to 218)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1534 146 LOAD_GLOBAL 18 (_global_backward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1533 158 POP_JUMP_FORWARD_IF_TRUE 29 (to 218)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1534 160 LOAD_GLOBAL 20 (_global_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1533 172 POP_JUMP_FORWARD_IF_TRUE 22 (to 218)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1535 174 LOAD_GLOBAL 22 (_global_forward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1533 186 POP_JUMP_FORWARD_IF_TRUE 15 (to 218)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1535 188 LOAD_GLOBAL 24 (_global_forward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1533 200 POP_JUMP_FORWARD_IF_TRUE 8 (to 218)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1536 202 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 204 LOAD_FAST 3 (forward_call)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 206 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 208 BUILD_MAP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 210 LOAD_FAST 2 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 212 DICT_MERGE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 214 CALL_FUNCTION_EX 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 216 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1538 >> 218 NOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1539 220 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 222 STORE_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1540 224 LOAD_GLOBAL 27 (NULL + set)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 240 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 STORE_FAST 5 (called_always_called_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1542 252 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 254 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 256 STORE_FAST 7 (non_full_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 258 STORE_FAST 6 (full_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1543 260 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 262 STORE_FAST 8 (backward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1544 264 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 266 LOAD_ATTR 6 (_backward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 276 POP_JUMP_FORWARD_IF_TRUE 7 (to 292)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 278 LOAD_GLOBAL 18 (_global_backward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 290 POP_JUMP_FORWARD_IF_FALSE 20 (to 332)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1545 >> 292 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 294 LOAD_METHOD 14 (_get_backward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 316 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 320 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 330 STORE_FAST 8 (backward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1547 >> 332 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 334 LOAD_ATTR 5 (_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 344 POP_JUMP_FORWARD_IF_TRUE 7 (to 360)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 346 LOAD_GLOBAL 20 (_global_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 358 POP_JUMP_FORWARD_IF_FALSE 23 (to 406)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1548 >> 360 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 362 LOAD_METHOD 15 (_get_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 384 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 388 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 398 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 402 STORE_FAST 6 (full_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 404 STORE_FAST 7 (non_full_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1550 >> 406 LOAD_GLOBAL 24 (_global_forward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 418 POP_JUMP_FORWARD_IF_TRUE 7 (to 434)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 420 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 422 LOAD_ATTR 8 (_forward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 432 POP_JUMP_FORWARD_IF_FALSE 189 (to 812)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1551 >> 434 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1552 436 LOAD_GLOBAL 24 (_global_forward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 448 LOAD_METHOD 16 (items)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 470 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 474 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1551 484 LIST_EXTEND 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1553 486 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 488 LOAD_ATTR 8 (_forward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 498 LOAD_METHOD 16 (items)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 520 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 524 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1551 534 LIST_EXTEND 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 536 LIST_TO_TUPLE
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 538 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 540 FOR_ITER 135 (to 812)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 542 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 546 STORE_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 548 STORE_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1555 550 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 552 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 554 LOAD_ATTR 17 (_forward_pre_hooks_with_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 564 CONTAINS_OP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 566 POP_JUMP_FORWARD_IF_FALSE 81 (to 730)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1556 568 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 570 LOAD_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 572 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 574 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 576 LOAD_FAST 2 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 578 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 582 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 592 STORE_FAST 11 (args_kwargs_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1557 594 LOAD_FAST 11 (args_kwargs_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 596 POP_JUMP_FORWARD_IF_NONE 65 (to 728)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1558 598 LOAD_GLOBAL 37 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 610 LOAD_FAST 11 (args_kwargs_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 612 LOAD_GLOBAL 38 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 624 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 628 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 638 POP_JUMP_FORWARD_IF_FALSE 25 (to 690)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 640 LOAD_GLOBAL 41 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 652 LOAD_FAST 11 (args_kwargs_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 654 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 658 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 668 LOAD_CONST 1 (2)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 670 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 676 POP_JUMP_FORWARD_IF_FALSE 6 (to 690)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1559 678 LOAD_FAST 11 (args_kwargs_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 680 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 684 STORE_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 686 STORE_FAST 2 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 688 JUMP_BACKWARD 75 (to 540)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1561 >> 690 LOAD_GLOBAL 43 (NULL + RuntimeError)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1562 702 LOAD_CONST 2 ('forward pre-hook must return None or a tuple of (new_args, new_kwargs), but got ')
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1563 704 LOAD_FAST 11 (args_kwargs_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1562 706 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 708 LOAD_CONST 3 ('.')
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 710 BUILD_STRING 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1561 712 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 716 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 726 RAISE_VARARGS 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1557 >> 728 JUMP_BACKWARD 95 (to 540)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1566 >> 730 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 732 LOAD_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 734 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 736 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 738 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 742 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 752 STORE_FAST 12 (args_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1567 754 LOAD_FAST 12 (args_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 756 POP_JUMP_FORWARD_IF_NONE 26 (to 810)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1568 758 LOAD_GLOBAL 37 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 770 LOAD_FAST 12 (args_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 772 LOAD_GLOBAL 38 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 784 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 788 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 798 POP_JUMP_FORWARD_IF_TRUE 3 (to 806)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1569 800 LOAD_FAST 12 (args_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 802 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 804 STORE_FAST 12 (args_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1570 >> 806 LOAD_FAST 12 (args_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 808 STORE_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 810 JUMP_BACKWARD 136 (to 540)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1572 >> 812 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 814 STORE_FAST 13 (bw_hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1573 816 LOAD_FAST 6 (full_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 818 POP_JUMP_FORWARD_IF_TRUE 2 (to 824)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 820 LOAD_FAST 8 (backward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 822 POP_JUMP_FORWARD_IF_FALSE 43 (to 910)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1574 >> 824 LOAD_GLOBAL 45 (NULL + hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 836 LOAD_ATTR 23 (BackwardHook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 846 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 848 LOAD_FAST 6 (full_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 850 LOAD_FAST 8 (backward_pre_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 852 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 856 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 866 STORE_FAST 13 (bw_hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1575 868 LOAD_FAST 13 (bw_hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 870 LOAD_METHOD 24 (setup_input_hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 892 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 894 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 898 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 908 STORE_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1577 >> 910 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 912 LOAD_FAST 3 (forward_call)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 914 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 916 BUILD_MAP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 918 LOAD_FAST 2 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 920 DICT_MERGE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 922 CALL_FUNCTION_EX 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 924 STORE_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1578 926 LOAD_GLOBAL 22 (_global_forward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 938 POP_JUMP_FORWARD_IF_TRUE 7 (to 954)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 940 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 942 LOAD_ATTR 7 (_forward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 952 POP_JUMP_FORWARD_IF_FALSE 139 (to 1232)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1579 >> 954 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1580 956 LOAD_GLOBAL 22 (_global_forward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 968 LOAD_METHOD 16 (items)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 990 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 994 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1579 1004 LIST_EXTEND 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1581 1006 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1008 LOAD_ATTR 7 (_forward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1018 LOAD_METHOD 16 (items)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1040 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1044 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1579 1054 LIST_EXTEND 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1056 LIST_TO_TUPLE
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1058 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 1060 FOR_ITER 85 (to 1232)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1062 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1066 STORE_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1068 STORE_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1584 1070 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1072 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1074 LOAD_ATTR 25 (_forward_hooks_always_called)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1084 CONTAINS_OP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1086 POP_JUMP_FORWARD_IF_TRUE 9 (to 1106)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1088 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1090 LOAD_GLOBAL 52 (_global_forward_hooks_always_called)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1102 CONTAINS_OP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1104 POP_JUMP_FORWARD_IF_FALSE 21 (to 1148)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1585 >> 1106 LOAD_FAST 5 (called_always_called_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1108 LOAD_METHOD 27 (add)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1130 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1132 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1136 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1146 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1587 >> 1148 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1150 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1152 LOAD_ATTR 28 (_forward_hooks_with_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1162 CONTAINS_OP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1164 POP_JUMP_FORWARD_IF_FALSE 15 (to 1196)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1588 1166 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1168 LOAD_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1170 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1172 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1174 LOAD_FAST 2 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1176 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1178 PRECALL 4
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1182 CALL 4
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1192 STORE_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1194 JUMP_FORWARD 13 (to 1222)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1590 >> 1196 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1198 LOAD_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1200 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1202 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1204 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1206 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1210 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1220 STORE_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1592 >> 1222 LOAD_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1224 POP_JUMP_FORWARD_IF_NONE 2 (to 1230)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1593 1226 LOAD_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1228 STORE_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 1230 JUMP_BACKWARD 86 (to 1060)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1595 >> 1232 LOAD_FAST 13 (bw_hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1234 POP_JUMP_FORWARD_IF_FALSE 90 (to 1416)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1596 1236 LOAD_GLOBAL 37 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1248 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1250 LOAD_GLOBAL 0 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1262 LOAD_ATTR 29 (Tensor)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1272 LOAD_GLOBAL 38 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1284 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1286 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1290 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1300 POP_JUMP_FORWARD_IF_TRUE 36 (to 1374)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1597 1302 LOAD_GLOBAL 61 (NULL + warnings)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1314 LOAD_ATTR 31 (warn)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1324 LOAD_CONST 4 ('For backward hooks to be called, module output should be a Tensor or a tuple of Tensors but received ')
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1599 1326 LOAD_GLOBAL 65 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1338 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1340 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1344 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1597 1354 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1356 BUILD_STRING 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1358 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1362 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1372 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1600 >> 1374 LOAD_FAST 13 (bw_hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1376 LOAD_METHOD 33 (setup_output_hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1398 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1400 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1404 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1414 STORE_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1603 >> 1416 LOAD_FAST 7 (non_full_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1418 POP_JUMP_FORWARD_IF_FALSE 199 (to 1818)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1604 1420 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1422 STORE_FAST 15 (var)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1605 1424 LOAD_GLOBAL 37 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1436 LOAD_FAST 15 (var)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1438 LOAD_GLOBAL 0 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1450 LOAD_ATTR 29 (Tensor)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1460 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1464 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1474 POP_JUMP_FORWARD_IF_TRUE 99 (to 1674)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1606 >> 1476 LOAD_GLOBAL 37 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1488 LOAD_FAST 15 (var)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1490 LOAD_GLOBAL 68 (dict)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1502 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1506 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1516 POP_JUMP_FORWARD_IF_FALSE 44 (to 1606)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1607 1518 LOAD_GLOBAL 71 (NULL + next)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1530 LOAD_CONST 5 (<code object <genexpr> at 0x7fd567f73ee0, file "/data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py", line 1607>)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1532 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1534 LOAD_FAST 15 (var)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1536 LOAD_METHOD 36 (values)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1558 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1562 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1572 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1574 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1578 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1588 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1592 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1602 STORE_FAST 15 (var)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1604 JUMP_FORWARD 8 (to 1622)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1609 >> 1606 LOAD_FAST 15 (var)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1608 LOAD_CONST 6 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1610 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1620 STORE_FAST 15 (var)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1605 >> 1622 LOAD_GLOBAL 37 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1634 LOAD_FAST 15 (var)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1636 LOAD_GLOBAL 0 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1648 LOAD_ATTR 29 (Tensor)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1658 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1662 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1672 POP_JUMP_BACKWARD_IF_FALSE 99 (to 1476)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1610 >> 1674 LOAD_FAST 15 (var)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1676 LOAD_ATTR 37 (grad_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1686 STORE_FAST 16 (grad_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1611 1688 LOAD_FAST 16 (grad_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1690 POP_JUMP_FORWARD_IF_NONE 63 (to 1818)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1612 1692 LOAD_FAST 7 (non_full_backward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1694 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 1696 FOR_ITER 37 (to 1772)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1698 STORE_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1613 1700 LOAD_FAST 16 (grad_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1702 LOAD_METHOD 38 (register_hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1724 LOAD_GLOBAL 79 (NULL + _WrappedHook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1736 LOAD_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1738 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1740 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1744 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1754 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1758 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1768 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1770 JUMP_BACKWARD 38 (to 1696)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1614 >> 1772 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1774 LOAD_METHOD 40 (_maybe_warn_non_full_backward_hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1796 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1798 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1800 LOAD_FAST 16 (grad_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1802 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1806 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1816 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1616 >> 1818 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1820 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 1822 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1618 1824 LOAD_GLOBAL 82 (Exception)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1836 CHECK_EXC_MATCH
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1838 EXTENDED_ARG 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1840 POP_JUMP_FORWARD_IF_FALSE 270 (to 2382)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1842 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1622 1844 LOAD_GLOBAL 22 (_global_forward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1856 LOAD_METHOD 16 (items)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1878 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1882 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1892 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 1894 FOR_ITER 96 (to 2088)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1896 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1900 STORE_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1902 STORE_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1623 1904 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1906 LOAD_GLOBAL 52 (_global_forward_hooks_always_called)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1918 CONTAINS_OP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1920 POP_JUMP_FORWARD_IF_FALSE 82 (to 2086)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1922 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1924 LOAD_FAST 5 (called_always_called_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1926 CONTAINS_OP 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1928 POP_JUMP_FORWARD_IF_FALSE 78 (to 2086)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1624 1930 NOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1625 1932 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1934 LOAD_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1936 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1938 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1940 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1942 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1946 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1956 STORE_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1626 1958 LOAD_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1960 POP_JUMP_FORWARD_IF_NONE 2 (to 1966)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1627 1962 LOAD_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1964 STORE_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 1966 JUMP_BACKWARD 37 (to 1894)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 1968 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1628 1970 LOAD_GLOBAL 82 (Exception)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1982 CHECK_EXC_MATCH
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1984 POP_JUMP_FORWARD_IF_FALSE 46 (to 2078)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1986 STORE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1629 1988 LOAD_GLOBAL 61 (NULL + warnings)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2000 LOAD_ATTR 31 (warn)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2010 LOAD_CONST 7 ('global module forward hook with ``always_call=True`` raised an exception that was silenced as another error was raised in forward: ')
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1630 2012 LOAD_GLOBAL 85 (NULL + str)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2024 LOAD_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2026 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2030 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1629 2040 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2042 BUILD_STRING 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2044 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2048 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2058 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1631 2060 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2062 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2064 STORE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2066 DELETE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2068 JUMP_BACKWARD 88 (to 1894)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2070 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2072 STORE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2074 DELETE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2076 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1628 >> 2078 RERAISE 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2080 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2082 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2084 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2086 JUMP_BACKWARD 97 (to 1894)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1633 >> 2088 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2090 LOAD_ATTR 7 (_forward_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2100 LOAD_METHOD 16 (items)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2122 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2126 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2136 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2138 FOR_ITER 120 (to 2380)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2140 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2144 STORE_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2146 STORE_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1634 2148 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2150 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2152 LOAD_ATTR 25 (_forward_hooks_always_called)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2162 CONTAINS_OP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2164 POP_JUMP_FORWARD_IF_FALSE 106 (to 2378)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2166 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2168 LOAD_FAST 5 (called_always_called_hooks)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2170 CONTAINS_OP 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2172 POP_JUMP_FORWARD_IF_FALSE 102 (to 2378)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1635 2174 NOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1636 2176 LOAD_FAST 9 (hook_id)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2178 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2180 LOAD_ATTR 28 (_forward_hooks_with_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2190 CONTAINS_OP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2192 POP_JUMP_FORWARD_IF_FALSE 15 (to 2224)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1637 2194 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2196 LOAD_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2198 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2200 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2202 LOAD_FAST 2 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2204 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2206 PRECALL 4
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2210 CALL 4
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2220 STORE_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2222 JUMP_FORWARD 13 (to 2250)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1639 >> 2224 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2226 LOAD_FAST 10 (hook)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2228 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2230 LOAD_FAST 1 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2232 LOAD_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2234 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2238 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2248 STORE_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1640 >> 2250 LOAD_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2252 POP_JUMP_FORWARD_IF_NONE 2 (to 2258)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1641 2254 LOAD_FAST 14 (hook_result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2256 STORE_FAST 4 (result)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2258 JUMP_BACKWARD 61 (to 2138)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2260 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1642 2262 LOAD_GLOBAL 82 (Exception)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2274 CHECK_EXC_MATCH
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2276 POP_JUMP_FORWARD_IF_FALSE 46 (to 2370)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2278 STORE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1643 2280 LOAD_GLOBAL 61 (NULL + warnings)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2292 LOAD_ATTR 31 (warn)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2302 LOAD_CONST 8 ('module forward hook with ``always_call=True`` raised an exception that was silenced as another error was raised in forward: ')
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1644 2304 LOAD_GLOBAL 85 (NULL + str)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2316 LOAD_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2318 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2322 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1643 2332 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2334 BUILD_STRING 2
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2336 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2340 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2350 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1645 2352 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2354 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2356 STORE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2358 DELETE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2360 JUMP_BACKWARD 112 (to 2138)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2362 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2364 STORE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2366 DELETE_FAST 17 (e)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2368 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1642 >> 2370 RERAISE 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2372 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2374 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2376 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2378 JUMP_BACKWARD 121 (to 2138)
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1647 >> 2380 RAISE_VARARGS 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1618 >> 2382 RERAISE 0
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 2384 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2386 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2388 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] ExceptionTable:
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 to 1818 -> 1822 [0]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1822 to 1928 -> 2384 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1932 to 1964 -> 1968 [2]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1966 to 1966 -> 2384 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1968 to 1986 -> 2080 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1988 to 2058 -> 2070 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2060 to 2068 -> 2384 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2070 to 2078 -> 2080 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2080 to 2172 -> 2384 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2176 to 2256 -> 2260 [2]
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2258 to 2258 -> 2384 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2260 to 2278 -> 2372 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2280 to 2350 -> 2362 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2352 to 2360 -> 2384 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2362 to 2370 -> 2372 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2372 to 2382 -> 2384 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,908.908000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,911.911000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1529 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,911.911000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _call_impl(self, *args, **kwargs):
[rank0]:[rank0]:V2024-03-27 14:57:45,929.929000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,929.929000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1530 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,929.929000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] forward_call = (self._slow_forward if torch._C._get_tracing_state() else self.forward)
[rank0]:[rank0]:V2024-03-27 14:57:45,929.929000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch []
[rank0]:[rank0]:V2024-03-27 14:57:45,930.930000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _C [PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,931.931000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD _get_tracing_state [PythonModuleVariable(<module 'torch._C' from '/data/users/willfeng/pytorch_yf225/torch/_C.cpython-311-x86_64-linux-gnu.so'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,931.931000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), TorchInGraphFunctionVariable(<built-in method _get_tracing_state of PyCapsule object at 0x7fd590324900>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,931.931000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), TorchInGraphFunctionVariable(<built-in method _get_tracing_state of PyCapsule object at 0x7fd590324900>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,931.931000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 76 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR forward [FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST forward_call [UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1533 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _backward_hooks [FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 218 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _backward_pre_hooks [FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 218 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,932.932000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _forward_hooks [FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 218 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1538 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] try:
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE NOP None []
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1539 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] result = None
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST result [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1540 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] called_always_called_hooks = set()
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL set []
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST called_always_called_hooks [SetVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1542 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] full_backward_hooks, non_full_backward_hooks = [], []
[rank0]:[rank0]:V2024-03-27 14:57:45,933.933000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST non_full_backward_hooks [ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST full_backward_hooks [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1543 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] backward_pre_hooks = []
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST backward_pre_hooks [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1544 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self._backward_pre_hooks or _global_backward_pre_hooks:
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _backward_pre_hooks [FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 292 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _global_backward_pre_hooks []
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 332 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1547 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self._backward_hooks or _global_backward_hooks:
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _backward_hooks [FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 360 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _global_backward_hooks []
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 406 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1550 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _global_forward_pre_hooks or self._forward_pre_hooks:
[rank0]:[rank0]:V2024-03-27 14:57:45,934.934000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _global_forward_pre_hooks []
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 434 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _forward_pre_hooks [FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 820 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1551 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] for hook_id, hook in (
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1552 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] *_global_forward_pre_hooks.items(),
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _global_forward_pre_hooks [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD items [ListVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [ListVariable(), NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [ListVariable(), NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1551 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] for hook_id, hook in (
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LIST_EXTEND 1 [ListVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1553 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,935.935000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] *self._forward_pre_hooks.items(),
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _forward_pre_hooks [ListVariable(), FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD items [ListVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [ListVariable(), NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [ListVariable(), NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1551 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] for hook_id, hook in (
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LIST_EXTEND 1 [ListVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LIST_TO_TUPLE None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 820 [ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(length=1, index=1), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST hook_id [ListIteratorVariable(length=1, index=1), LazyVariableTracker(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST hook [ListIteratorVariable(length=1, index=1), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1555 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if hook_id in self._forward_pre_hooks_with_kwargs:
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST hook_id [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _forward_pre_hooks_with_kwargs [ListIteratorVariable(length=1, index=1), ConstantVariable(), FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CONTAINS_OP 0 [ListIteratorVariable(length=1, index=1), ConstantVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,936.936000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 734 [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1556 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] args_kwargs_result = hook(self, args, kwargs) # type: ignore[misc]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PUSH_NULL None [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST hook [ListIteratorVariable(length=1, index=1), NullVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ListIteratorVariable(length=1, index=1), NullVariable(), FunctoolsPartialVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST args [ListIteratorVariable(length=1, index=1), NullVariable(), FunctoolsPartialVariable(), FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST kwargs [ListIteratorVariable(length=1, index=1), NullVariable(), FunctoolsPartialVariable(), FSDPManagedNNModuleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 3 [ListIteratorVariable(length=1, index=1), NullVariable(), FunctoolsPartialVariable(), FSDPManagedNNModuleVariable(), TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 3 [ListIteratorVariable(length=1, index=1), NullVariable(), FunctoolsPartialVariable(), FSDPManagedNNModuleVariable(), TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _fsdp_state_pre_forward from /data/users/willfeng/pytorch_yf225/torch/nn/modules/module.py:1556 in _call_impl (Module._call_impl) (inline depth: 1)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] args_kwargs_result = hook(self, args, kwargs) # type: ignore[misc]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _fsdp_state_pre_forward at 0x7efc710, file "/data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py", line 40>
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 40 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 45 2 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_ATTR 0 (_training_state)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_GLOBAL 2 (TrainingState)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 LOAD_ATTR 2 (PRE_BACKWARD)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 POP_JUMP_FORWARD_IF_FALSE 4 (to 52)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 44 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 47 >> 52 LOAD_GLOBAL 2 (TrainingState)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 LOAD_ATTR 3 (FORWARD)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 STORE_ATTR 0 (_training_state)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 86 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 LOAD_METHOD 4 (_root_pre_forward)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 LOAD_FAST 1 (module)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 112 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 114 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 130 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 STORE_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 STORE_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 49 138 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 LOAD_ATTR 5 (_mp_policy)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 LOAD_ATTR 6 (cast_forward_inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 POP_JUMP_FORWARD_IF_FALSE 135 (to 432)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 162 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 LOAD_ATTR 5 (_mp_policy)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 174 LOAD_ATTR 7 (param_dtype)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 184 POP_JUMP_FORWARD_IF_FALSE 123 (to 432)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 186 LOAD_GLOBAL 16 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 198 LOAD_ATTR 9 (profiler)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 208 LOAD_METHOD 10 (record_function)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 230 LOAD_CONST 1 ('FSDP::cast_forward_inputs')
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 232 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 246 BEFORE_WITH
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 248 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 51 250 LOAD_GLOBAL 23 (NULL + functools)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 262 LOAD_ATTR 12 (partial)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 272 LOAD_GLOBAL 26 (_cast_fp_tensor)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 284 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 286 LOAD_ATTR 5 (_mp_policy)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 296 LOAD_ATTR 7 (param_dtype)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 51 306 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 310 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 320 STORE_FAST 4 (cast_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 54 322 LOAD_GLOBAL 29 (NULL + tree_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 334 LOAD_FAST 4 (cast_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 336 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 338 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 342 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 352 LOAD_GLOBAL 29 (NULL + tree_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 364 LOAD_FAST 4 (cast_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 366 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 368 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 372 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 382 STORE_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 384 STORE_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 386 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 388 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 390 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 392 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 396 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 406 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 408 JUMP_FORWARD 11 (to 432)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 410 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 412 WITH_EXCEPT_START
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 414 POP_JUMP_FORWARD_IF_TRUE 4 (to 424)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 416 RERAISE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 418 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 420 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 422 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 424 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 426 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 428 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 430 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 55 >> 432 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 434 LOAD_ATTR 15 (_fsdp_param_group)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 444 POP_JUMP_FORWARD_IF_FALSE 31 (to 508)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 446 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 448 LOAD_ATTR 15 (_fsdp_param_group)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 458 LOAD_METHOD 16 (pre_forward)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 480 LOAD_FAST 1 (module)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 482 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 484 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 486 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 490 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 500 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 504 STORE_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 506 STORE_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 57 >> 508 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 510 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 512 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 514 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] ExceptionTable:
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 248 to 384 -> 410 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 410 to 416 -> 418 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 424 to 424 -> 418 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,937.937000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,938.938000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:40 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,938.938000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _fsdp_state_pre_forward(
[rank0]:[rank0]:V2024-03-27 14:57:45,941.941000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,941.941000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:45 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,941.941000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self._training_state == TrainingState.PRE_BACKWARD:
[rank0]:[rank0]:V2024-03-27 14:57:45,941.941000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,941.941000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _training_state [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL TrainingState [EnumVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR PRE_BACKWARD [EnumVariable(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP == [EnumVariable(), EnumVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 52 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:47 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self._training_state = TrainingState.FORWARD
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL TrainingState []
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR FORWARD [UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [EnumVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR _training_state [EnumVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:48 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] args, kwargs = self._root_pre_forward(module, args, kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD _root_pre_forward [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST module [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,942.942000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST args [NullVariable(), UserMethodVariable(), FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST kwargs [NullVariable(), UserMethodVariable(), FSDPManagedNNModuleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 3 [NullVariable(), UserMethodVariable(), FSDPManagedNNModuleVariable(), TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 3 [NullVariable(), UserMethodVariable(), FSDPManagedNNModuleVariable(), TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _root_pre_forward from /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:48 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] args, kwargs = self._root_pre_forward(module, args, kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _root_pre_forward at 0x7efa660, file "/data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py", line 145>
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 145 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 148 2 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_METHOD 0 (_lazy_init)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 40 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 149 42 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 LOAD_ATTR 1 (_state_ctx)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 54 LOAD_ATTR 2 (iter_forward_root)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 POP_JUMP_FORWARD_IF_NONE 4 (to 74)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 66 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 68 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 72 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 151 >> 74 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 78 LOAD_ATTR 1 (_state_ctx)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 STORE_ATTR 2 (iter_forward_root)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 152 98 LOAD_GLOBAL 6 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 LOAD_ATTR 4 (profiler)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 LOAD_METHOD 5 (record_function)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 142 LOAD_CONST 1 ('FSDP::root_pre_forward')
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 148 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 158 BEFORE_WITH
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 153 162 LOAD_GLOBAL 6 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 174 LOAD_ATTR 6 (distributed)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 184 LOAD_ATTR 7 (_functional_collectives)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 194 LOAD_METHOD 8 (is_torchdynamo_compiling)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 216 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 230 POP_JUMP_FORWARD_IF_TRUE 92 (to 416)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 155 232 LOAD_GLOBAL 6 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 244 LOAD_ATTR 9 (cuda)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 254 LOAD_METHOD 10 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 276 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 280 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 290 STORE_FAST 4 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 156 292 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 294 LOAD_ATTR 11 (_comm_ctx)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 304 LOAD_ATTR 12 (all_gather_copy_in_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 314 LOAD_METHOD 13 (wait_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 336 LOAD_FAST 4 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 338 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 342 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 352 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 157 354 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 356 LOAD_ATTR 11 (_comm_ctx)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 366 LOAD_ATTR 14 (all_gather_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 376 LOAD_METHOD 13 (wait_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 398 LOAD_FAST 4 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 400 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 404 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 414 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 158 >> 416 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 418 LOAD_ATTR 15 (_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 428 LOAD_ATTR 16 (type)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 438 LOAD_CONST 2 ('cuda')
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 440 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 446 POP_JUMP_FORWARD_IF_FALSE 97 (to 642)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 159 448 LOAD_GLOBAL 6 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 460 LOAD_ATTR 4 (profiler)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 470 LOAD_METHOD 5 (record_function)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 492 LOAD_CONST 3 ('FSDP::inputs_to_device')
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 494 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 498 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 508 BEFORE_WITH
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 510 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 512 LOAD_GLOBAL 35 (NULL + _to_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 161 524 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 526 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 528 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 530 LOAD_ATTR 15 (_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 540 LOAD_CONST 4 (False)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 542 PRECALL 4
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 546 CALL 4
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 556 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 560 STORE_FAST 5 (args_tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 562 STORE_FAST 6 (kwargs_tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 159 564 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 566 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 568 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 570 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 574 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 584 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 586 JUMP_FORWARD 11 (to 610)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 588 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 590 WITH_EXCEPT_START
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 592 POP_JUMP_FORWARD_IF_TRUE 4 (to 602)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 594 RERAISE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 596 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 598 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 600 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 602 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 604 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 606 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 608 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 163 >> 610 LOAD_FAST 5 (args_tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 612 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 614 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 624 LOAD_FAST 6 (kwargs_tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 626 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 628 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 638 STORE_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 640 STORE_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 152 >> 642 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 644 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 646 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 648 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 652 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 662 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 664 JUMP_FORWARD 11 (to 688)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 666 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 668 WITH_EXCEPT_START
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 670 POP_JUMP_FORWARD_IF_TRUE 4 (to 680)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 672 RERAISE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 674 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 676 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 678 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 680 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 682 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 684 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 686 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 >> 688 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 690 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 692 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 694 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] ExceptionTable:
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 to 508 -> 666 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 510 to 562 -> 588 [2] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 564 to 586 -> 666 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 588 to 594 -> 596 [4] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 596 to 600 -> 666 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 602 to 602 -> 596 [4] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 604 to 640 -> 666 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 666 to 672 -> 674 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 680 to 680 -> 674 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,943.943000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,944.944000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:145 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,944.944000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _root_pre_forward(
[rank0]:[rank0]:V2024-03-27 14:57:45,944.944000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,944.944000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:148 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,944.944000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self._lazy_init()
[rank0]:[rank0]:V2024-03-27 14:57:45,944.944000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,944.944000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD _lazy_init [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,944.944000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,944.944000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _lazy_init from /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:148 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] self._lazy_init()
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~^^
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _lazy_init at 0x7ef0440, file "/data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py", line 166>
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 166 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 173 2 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_ATTR 0 (_is_root)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 POP_JUMP_FORWARD_IF_NONE 2 (to 20)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 174 16 LOAD_CONST 1 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 175 >> 20 LOAD_CONST 2 (True)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 STORE_ATTR 0 (_is_root)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 176 34 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 LOAD_ATTR 1 (_module)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 STORE_FAST 1 (root_module)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 177 48 LOAD_FAST 1 (root_module)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 LOAD_METHOD 2 (named_modules)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 72 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 88 FOR_ITER 91 (to 272)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 90 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 94 STORE_FAST 2 (module_name)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 96 STORE_FAST 3 (module)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 178 98 LOAD_GLOBAL 7 (NULL + _get_module_fsdp_state)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 LOAD_FAST 3 (module)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 112 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 126 COPY 1
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 STORE_FAST 4 (state)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 130 POP_JUMP_FORWARD_IF_NOT_NONE 1 (to 134)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 179 132 JUMP_BACKWARD 23 (to 88)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 180 >> 134 LOAD_FAST 3 (module)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 LOAD_FAST 1 (root_module)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 IS_OP 1
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 POP_JUMP_FORWARD_IF_FALSE 33 (to 208)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 181 142 LOAD_FAST 4 (state)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 LOAD_ATTR 0 (_is_root)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 154 POP_JUMP_FORWARD_IF_NONE 19 (to 194)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 156 LOAD_GLOBAL 9 (NULL + RuntimeError)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 183 168 LOAD_CONST 3 ('FSDP state has already been lazily initialized for ')
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 184 170 LOAD_FAST 2 (module_name)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 183 172 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 174 LOAD_CONST 4 ('\nFSDP requires running forward through the root module first')
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 176 BUILD_STRING 3
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 178 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 192 RAISE_VARARGS 1
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 187 >> 194 LOAD_CONST 5 (False)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 LOAD_FAST 4 (state)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 198 STORE_ATTR 0 (_is_root)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 188 >> 208 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 210 LOAD_ATTR 5 (_state_ctx)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 LOAD_ATTR 6 (all_states)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 230 LOAD_METHOD 7 (append)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 252 LOAD_FAST 4 (state)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 254 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 258 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 268 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 270 JUMP_BACKWARD 92 (to 88)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 189 >> 272 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 274 LOAD_ATTR 8 (_fsdp_param_group)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 284 POP_JUMP_FORWARD_IF_FALSE 19 (to 324)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 286 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 288 LOAD_ATTR 9 (_reshard_after_forward_root)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 298 POP_JUMP_FORWARD_IF_TRUE 12 (to 324)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 193 300 LOAD_CONST 1 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 302 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 304 LOAD_ATTR 8 (_fsdp_param_group)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 314 STORE_ATTR 10 (post_forward_mesh_info)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 194 >> 324 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 326 LOAD_METHOD 11 (_init_fqns)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 348 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 352 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 362 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 195 364 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 366 LOAD_METHOD 12 (_init_shared_state)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 388 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 392 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 402 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 198 404 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 406 LOAD_ATTR 5 (_state_ctx)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 416 LOAD_ATTR 6 (all_states)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 426 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 428 FOR_ITER 34 (to 498)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 430 STORE_FAST 4 (state)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 199 432 LOAD_FAST 4 (state)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 434 LOAD_ATTR 8 (_fsdp_param_group)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 444 POP_JUMP_FORWARD_IF_FALSE 25 (to 496)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 446 LOAD_FAST 4 (state)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 448 LOAD_ATTR 8 (_fsdp_param_group)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 458 LOAD_METHOD 13 (lazy_init)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 480 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 484 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 494 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 496 JUMP_BACKWARD 35 (to 428)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 198 >> 498 LOAD_CONST 1 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 500 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:166 in _lazy_init (FSDPState._lazy_init) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,945.945000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _lazy_init(self) -> None:
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:173 in _lazy_init (FSDPState._lazy_init) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self._is_root is not None:
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _is_root [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE IS_OP 0 [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 24 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:174 in _lazy_init (FSDPState._lazy_init) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return # no-op: already initialized
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _lazy_init at 0x7ef0440, file "/data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py", line 166>
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:149 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self._state_ctx.iter_forward_root is not None:
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,946.946000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _state_ctx [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR iter_forward_root [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE IS_OP 0 [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 78 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:151 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self._state_ctx.iter_forward_root = self
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _state_ctx [UserDefinedObjectVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR iter_forward_root [UserDefinedObjectVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:152 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] with torch.profiler.record_function("FSDP::root_pre_forward"):
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch []
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR profiler [PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD record_function [PythonModuleVariable(<module 'torch.profiler' from '/data/users/willfeng/pytorch_yf225/torch/profiler/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST FSDP::root_pre_forward [NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,947.947000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>), ConstantVariable()]
[rank0]:[rank0]:W2024-03-27 14:57:45,948.948000 140555295757312 torch/_logging/_internal.py:1042] [0/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BEFORE_WITH None [NullContextVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:153 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if not torch.distributed._functional_collectives.is_torchdynamo_compiling():
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch [WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR distributed [WithExitFunctionVariable(), PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _functional_collectives [WithExitFunctionVariable(), PythonModuleVariable(<module 'torch.distributed' from '/data/users/willfeng/pytorch_yf225/torch/distributed/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD is_torchdynamo_compiling [WithExitFunctionVariable(), PythonModuleVariable(<module 'torch.distributed._functional_collectives' from '/data/users/willfeng/pytorch_yf225/torch/distributed/_functional_collectives.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [WithExitFunctionVariable(), NullVariable(), TorchInGraphFunctionVariable(<function is_dynamo_compiling at 0x7fd5268a63e0>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [WithExitFunctionVariable(), NullVariable(), TorchInGraphFunctionVariable(<function is_dynamo_compiling at 0x7fd5268a63e0>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 420 [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:158 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self._device.type == "cuda":
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,948.948000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _device [WithExitFunctionVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR type [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST cuda [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP == [WithExitFunctionVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 646 [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:159 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] with torch.profiler.record_function("FSDP::inputs_to_device"):
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch [WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR profiler [WithExitFunctionVariable(), PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD record_function [WithExitFunctionVariable(), PythonModuleVariable(<module 'torch.profiler' from '/data/users/willfeng/pytorch_yf225/torch/profiler/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST FSDP::inputs_to_device [WithExitFunctionVariable(), NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [WithExitFunctionVariable(), NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [WithExitFunctionVariable(), NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BEFORE_WITH None [WithExitFunctionVariable(), NullContextVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [WithExitFunctionVariable(), WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:160 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] args_tuple, kwargs_tuple = _to_kwargs(
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _to_kwargs [WithExitFunctionVariable(), WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:161 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] args, kwargs, self._device, False
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST args [WithExitFunctionVariable(), WithExitFunctionVariable(), NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST kwargs [WithExitFunctionVariable(), WithExitFunctionVariable(), NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,949.949000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [WithExitFunctionVariable(), WithExitFunctionVariable(), NullVariable(), UserFunctionVariable(), TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _device [WithExitFunctionVariable(), WithExitFunctionVariable(), NullVariable(), UserFunctionVariable(), TupleVariable(), ConstDictVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST False [WithExitFunctionVariable(), WithExitFunctionVariable(), NullVariable(), UserFunctionVariable(), TupleVariable(), ConstDictVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:160 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] args_tuple, kwargs_tuple = _to_kwargs(
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 4 [WithExitFunctionVariable(), WithExitFunctionVariable(), NullVariable(), UserFunctionVariable(), TupleVariable(), ConstDictVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 4 [WithExitFunctionVariable(), WithExitFunctionVariable(), NullVariable(), UserFunctionVariable(), TupleVariable(), ConstDictVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _to_kwargs from /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:160 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] args_tuple, kwargs_tuple = _to_kwargs(
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~^
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] args, kwargs, self._device, False
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ) # same as DDP
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _to_kwargs at 0x4c2ab50, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 243>
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 243 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 251 2 LOAD_FAST 0 (inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 4 POP_JUMP_FORWARD_IF_FALSE 17 (to 40)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 LOAD_GLOBAL 1 (NULL + _recursive_to)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 0 (inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_FAST 2 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 LOAD_FAST 3 (use_side_stream_for_tensor_copies)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 JUMP_FORWARD 1 (to 42)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 252 >> 40 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 249 >> 42 STORE_FAST 4 (moved_inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 256 44 LOAD_FAST 1 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 255 46 POP_JUMP_FORWARD_IF_FALSE 17 (to 82)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 LOAD_GLOBAL 1 (NULL + _recursive_to)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 LOAD_FAST 1 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 62 LOAD_FAST 2 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 LOAD_FAST 3 (use_side_stream_for_tensor_copies)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 66 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 80 JUMP_FORWARD 1 (to 84)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 257 >> 82 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 254 >> 84 STORE_FAST 5 (moved_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 259 86 LOAD_GLOBAL 3 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 98 LOAD_FAST 4 (moved_inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 100 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 104 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 114 LOAD_GLOBAL 3 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 126 LOAD_FAST 5 (moved_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 132 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 142 COMPARE_OP 0 (<)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 148 POP_JUMP_FORWARD_IF_FALSE 74 (to 298)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 260 150 LOAD_FAST 4 (moved_inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 152 LOAD_METHOD 2 (extend)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 174 LOAD_CONST 1 (<code object <listcomp> at 0x7fd567f24d30, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 260>)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 176 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 178 LOAD_GLOBAL 7 (NULL + range)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 190 LOAD_GLOBAL 3 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 202 LOAD_FAST 5 (moved_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 204 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 208 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 LOAD_GLOBAL 3 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 230 LOAD_FAST 0 (inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 232 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 246 BINARY_OP 10 (-)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 254 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 264 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 266 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 270 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 280 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 284 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 294 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 296 JUMP_FORWARD 105 (to 508)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 261 >> 298 LOAD_GLOBAL 3 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 310 LOAD_FAST 5 (moved_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 312 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 316 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 326 LOAD_GLOBAL 3 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 338 LOAD_FAST 4 (moved_inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 340 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 344 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 354 COMPARE_OP 0 (<)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 360 POP_JUMP_FORWARD_IF_FALSE 73 (to 508)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 262 362 LOAD_FAST 5 (moved_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 364 LOAD_METHOD 2 (extend)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 386 LOAD_CONST 2 (<code object <listcomp> at 0x7fd52a1917c0, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 262>)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 388 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 390 LOAD_GLOBAL 7 (NULL + range)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 402 LOAD_GLOBAL 3 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 414 LOAD_FAST 4 (moved_inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 416 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 420 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 430 LOAD_GLOBAL 3 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 442 LOAD_FAST 5 (moved_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 444 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 448 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 458 BINARY_OP 10 (-)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 462 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 466 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 476 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 478 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 482 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 492 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 496 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 506 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 263 >> 508 LOAD_GLOBAL 9 (NULL + tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 520 LOAD_FAST 4 (moved_inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 522 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 526 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 536 LOAD_GLOBAL 9 (NULL + tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 548 LOAD_FAST 5 (moved_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 550 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 554 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 564 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 566 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,950.950000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,951.951000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:243 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,951.951000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _to_kwargs(
[rank0]:[rank0]:V2024-03-27 14:57:45,954.954000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,954.954000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:251 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,954.954000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if inputs
[rank0]:[rank0]:V2024-03-27 14:57:45,954.954000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST inputs []
[rank0]:[rank0]:V2024-03-27 14:57:45,954.954000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:250 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,954.954000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] _recursive_to(inputs, target_device, use_side_stream_for_tensor_copies)
[rank0]:[rank0]:V2024-03-27 14:57:45,954.954000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 40 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,955.955000 140555295757312 torch/_dynamo/output_graph.py:1943] [0/0] create_graph_input L_args_0_ L['args'][0]
[rank0]:[rank0]:V2024-03-27 14:57:45,955.955000 140555295757312 torch/_dynamo/variables/builder.py:1890] [0/0] wrap_to_fake L['args'][0] (8, 2048) StatefulSymbolicContext(dynamic_sizes=[<DimDynamic.STATIC: 2>, <DimDynamic.STATIC: 2>], constraint_sizes=[None, None], view_base_context=None, tensor_source=GetItemSource(base=LocalSource(local_name='args', cell_or_freevar=False), index=0, index_is_slice=False), shape_env_to_source_to_symbol_cache={}) <class 'torch.Tensor'>
[rank0]:[rank0]:V2024-03-27 14:57:45,956.956000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _recursive_to []
[rank0]:[rank0]:V2024-03-27 14:57:45,956.956000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST inputs [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,956.956000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST target_device [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,956.956000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST use_side_stream_for_tensor_copies [NullVariable(), UserFunctionVariable(), TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,956.956000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 3 [NullVariable(), UserFunctionVariable(), TupleVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,956.956000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 3 [NullVariable(), UserFunctionVariable(), TupleVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _recursive_to from /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:250 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] _recursive_to(inputs, target_device, use_side_stream_for_tensor_copies)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _recursive_to at 0x7fd528028730, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 89>
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 MAKE_CELL 1 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 MAKE_CELL 2 (use_side_stream_for_tensor_copies)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 MAKE_CELL 4 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 89 6 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 92 8 LOAD_CLOSURE 1 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 LOAD_CLOSURE 4 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_CLOSURE 2 (use_side_stream_for_tensor_copies)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 BUILD_TUPLE 3
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_CONST 1 (<code object to_map at 0x4c29920, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 92>)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 STORE_DEREF 4 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 133 22 NOP
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 24 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 LOAD_DEREF 4 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 LOAD_FAST 0 (inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 STORE_FAST 3 (res)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 46 LOAD_CONST 2 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 STORE_DEREF 4 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 JUMP_FORWARD 7 (to 66)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 52 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 54 LOAD_CONST 2 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 STORE_DEREF 4 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 RERAISE 0
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 60 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 62 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 137 >> 66 LOAD_FAST 3 (res)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 68 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] ExceptionTable:
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 to 44 -> 52 [0]
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 to 58 -> 60 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL target_device []
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL use_side_stream_for_tensor_copies []
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL to_map []
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:89 in _recursive_to (_recursive_to) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _recursive_to(inputs, target_device, use_side_stream_for_tensor_copies):
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:92 in _recursive_to (_recursive_to.to_map) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def to_map(obj):
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CLOSURE target_device []
[rank0]:[rank0]:V2024-03-27 14:57:45,957.957000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CLOSURE to_map [NewCellVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CLOSURE use_side_stream_for_tensor_copies [NewCellVariable(), NewCellVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 3 [NewCellVariable(), NewCellVariable(), NewCellVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object to_map at 0x4c29920, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 92> [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 8 [TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_DEREF to_map [NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:133 in _recursive_to (_recursive_to) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] try:
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE NOP None []
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:134 in _recursive_to (_recursive_to) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] res = to_map(inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PUSH_NULL None []
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF to_map [NullVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST inputs [NullVariable(), NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), NestedUserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,958.958000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), NestedUserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,959.959000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call to_map from /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:134 in _recursive_to (_recursive_to) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,959.959000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] res = to_map(inputs)
[rank0]:[rank0]:V2024-03-27 14:57:45,959.959000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object to_map at 0x4c29920, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 92>
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 COPY_FREE_VARS 3
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 MAKE_CELL 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 92 4 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 93 6 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_GLOBAL 2 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 LOAD_ATTR 2 (Tensor)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 LOAD_GLOBAL 6 (PackedSequence)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 54 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 EXTENDED_ARG 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 72 POP_JUMP_FORWARD_IF_FALSE 407 (to 888)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 94 74 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 LOAD_GLOBAL 6 (PackedSequence)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 100 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 104 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 114 POP_JUMP_FORWARD_IF_FALSE 12 (to 140)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 118 LOAD_ATTR 4 (data)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 LOAD_ATTR 5 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 JUMP_FORWARD 6 (to 152)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 140 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 142 LOAD_ATTR 5 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 152 STORE_FAST 1 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 95 154 LOAD_FAST 1 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 156 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 158 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 POP_JUMP_FORWARD_IF_FALSE 3 (to 172)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 96 166 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 170 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 97 >> 172 LOAD_DEREF 8 (use_side_stream_for_tensor_copies)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 174 POP_JUMP_FORWARD_IF_TRUE 22 (to 220)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 98 176 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 178 LOAD_METHOD 6 (to)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 202 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 206 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 216 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 101 >> 220 LOAD_GLOBAL 15 (NULL + getattr)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 232 LOAD_GLOBAL 2 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 244 LOAD_FAST 1 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 246 LOAD_ATTR 8 (type)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 256 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 258 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 262 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 272 STORE_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 102 274 LOAD_FAST 1 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 276 LOAD_ATTR 8 (type)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 286 LOAD_CONST 1 ('cpu')
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 288 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 294 POP_JUMP_FORWARD_IF_TRUE 2 (to 300)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 296 LOAD_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 298 POP_JUMP_FORWARD_IF_NOT_NONE 22 (to 344)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 103 >> 300 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 302 LOAD_METHOD 6 (to)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 324 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 326 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 330 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 340 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 342 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 >> 344 LOAD_GLOBAL 19 (NULL + _get_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 356 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 358 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 362 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 372 STORE_FAST 3 (stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 107 374 LOAD_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 376 LOAD_METHOD 10 (stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 398 LOAD_FAST 3 (stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 400 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 404 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 414 BEFORE_WITH
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 416 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 108 418 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 420 LOAD_METHOD 6 (to)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 442 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 444 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 448 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 458 STORE_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 107 460 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 462 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 464 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 466 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 470 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 480 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 482 JUMP_FORWARD 11 (to 506)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 484 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 486 WITH_EXCEPT_START
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 488 POP_JUMP_FORWARD_IF_TRUE 4 (to 498)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 490 RERAISE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 492 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 494 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 496 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 498 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 500 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 502 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 504 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 >> 506 LOAD_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 508 LOAD_METHOD 5 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 530 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 532 LOAD_ATTR 11 (index)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 542 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 546 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 556 BEFORE_WITH
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 558 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 111 560 LOAD_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 562 LOAD_METHOD 12 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 584 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 588 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 598 STORE_FAST 5 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 113 600 LOAD_FAST 5 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 602 LOAD_METHOD 13 (wait_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 624 LOAD_FAST 3 (stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 626 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 630 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 640 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 642 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 654 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 656 LOAD_GLOBAL 6 (PackedSequence)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 668 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 672 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 682 POP_JUMP_FORWARD_IF_FALSE 27 (to 738)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 117 684 LOAD_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 686 LOAD_ATTR 4 (data)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 696 LOAD_METHOD 14 (record_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 718 LOAD_FAST 5 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 720 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 724 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 734 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 736 JUMP_FORWARD 49 (to 836)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 119 >> 738 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 750 LOAD_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 752 LOAD_GLOBAL 2 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 764 LOAD_ATTR 2 (Tensor)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 774 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 778 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 788 POP_JUMP_FORWARD_IF_TRUE 2 (to 794)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 790 LOAD_ASSERTION_ERROR
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 RAISE_VARARGS 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 >> 794 LOAD_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 796 LOAD_METHOD 14 (record_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 818 LOAD_FAST 5 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 820 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 824 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 834 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 >> 836 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 838 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 840 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 842 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 846 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 856 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 858 JUMP_FORWARD 11 (to 882)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 860 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 862 WITH_EXCEPT_START
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 864 POP_JUMP_FORWARD_IF_TRUE 4 (to 874)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 866 RERAISE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 868 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 870 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 872 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 874 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 876 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 878 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 880 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 121 >> 882 LOAD_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 884 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 886 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 122 >> 888 LOAD_GLOBAL 31 (NULL + _is_namedtuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 900 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 902 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 906 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 916 POP_JUMP_FORWARD_IF_FALSE 35 (to 988)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 123 918 LOAD_CLOSURE 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 920 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 922 LOAD_CONST 2 (<code object <listcomp> at 0x7fd5281fef10, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 123>)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 924 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 926 LOAD_GLOBAL 33 (NULL + zip)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 938 LOAD_GLOBAL 35 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 950 LOAD_DEREF 7 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 952 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 954 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 958 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 968 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 970 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 972 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 976 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 986 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 124 >> 988 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1000 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1002 LOAD_GLOBAL 36 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1014 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1018 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1028 POP_JUMP_FORWARD_IF_FALSE 55 (to 1140)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1030 LOAD_GLOBAL 39 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1042 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1044 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1048 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1058 LOAD_CONST 3 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1060 COMPARE_OP 4 (>)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1066 POP_JUMP_FORWARD_IF_FALSE 36 (to 1140)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 125 1068 LOAD_GLOBAL 41 (NULL + list)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1080 LOAD_GLOBAL 33 (NULL + zip)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1092 LOAD_GLOBAL 35 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1104 LOAD_DEREF 7 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1106 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1108 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1112 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1122 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1124 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1128 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1138 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 126 >> 1140 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1152 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1154 LOAD_GLOBAL 40 (list)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1166 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1170 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1180 POP_JUMP_FORWARD_IF_FALSE 52 (to 1286)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1182 LOAD_GLOBAL 39 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1194 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1196 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1200 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1210 LOAD_CONST 3 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1212 COMPARE_OP 4 (>)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1218 POP_JUMP_FORWARD_IF_FALSE 33 (to 1286)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 127 1220 LOAD_CONST 4 (<code object <listcomp> at 0x7fd5281fea60, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 127>)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1222 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1224 LOAD_GLOBAL 33 (NULL + zip)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1236 LOAD_GLOBAL 35 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1248 LOAD_DEREF 7 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1250 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1252 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1256 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1266 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1268 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1270 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1274 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1284 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 >> 1286 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1298 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1300 LOAD_GLOBAL 42 (dict)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1312 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1316 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1326 POP_JUMP_FORWARD_IF_FALSE 72 (to 1472)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1328 LOAD_GLOBAL 39 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1340 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1342 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1346 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1356 LOAD_CONST 3 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1358 COMPARE_OP 4 (>)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1364 POP_JUMP_FORWARD_IF_FALSE 53 (to 1472)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 129 1366 LOAD_CLOSURE 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1368 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1370 LOAD_CONST 5 (<code object <listcomp> at 0x7fd528028230, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 129>)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1372 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1374 LOAD_GLOBAL 33 (NULL + zip)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1386 LOAD_GLOBAL 35 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1398 LOAD_DEREF 7 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1400 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1402 LOAD_METHOD 22 (items)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1424 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1428 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1438 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1442 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1452 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1454 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1456 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1460 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1470 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 130 >> 1472 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1474 BUILD_LIST 1
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1476 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] ExceptionTable:
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 416 to 458 -> 484 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 484 to 490 -> 492 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 498 to 498 -> 492 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 558 to 834 -> 860 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 860 to 866 -> 868 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 874 to 874 -> 868 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,960.960000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COPY_FREE_VARS 3 []
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL obj []
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:92 in to_map (_recursive_to.to_map) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def to_map(obj):
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:93 in to_map (_recursive_to.to_map) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if isinstance(obj, (torch.Tensor, PackedSequence)):
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL isinstance []
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF obj [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR Tensor [NullVariable(), BuiltinVariable(), TupleVariable(), PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,961.961000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL PackedSequence [NullVariable(), BuiltinVariable(), TupleVariable(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [NullVariable(), BuiltinVariable(), TupleVariable(), UserDefinedClassVariable(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 890 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:122 in to_map (_recursive_to.to_map) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _is_namedtuple(obj):
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _is_namedtuple []
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF obj [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _is_namedtuple from /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:122 in to_map (_recursive_to.to_map) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if _is_namedtuple(obj):
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _is_namedtuple at 0x7fd5282076d0, file "/data/users/willfeng/pytorch_yf225/torch/nn/parallel/scatter_gather.py", line 13>
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 13 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 2 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_GLOBAL 2 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 JUMP_IF_FALSE_OR_POP 31 (to 106)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 LOAD_GLOBAL 5 (NULL + hasattr)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 LOAD_FAST 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_CONST 1 ('_asdict')
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 JUMP_IF_FALSE_OR_POP 15 (to 106)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 LOAD_GLOBAL 5 (NULL + hasattr)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 LOAD_FAST 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 90 LOAD_CONST 2 ('_fields')
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 92 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 96 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 15 >> 106 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,962.962000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,963.963000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/parallel/scatter_gather.py:13 in _is_namedtuple (_is_namedtuple) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,963.963000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _is_namedtuple(obj: Any) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:45,964.964000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,964.964000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/parallel/scatter_gather.py:16 in _is_namedtuple (_is_namedtuple) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,964.964000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] isinstance(obj, tuple) and hasattr(obj, "_asdict") and hasattr(obj, "_fields")
[rank0]:[rank0]:V2024-03-27 14:57:45,964.964000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL isinstance []
[rank0]:[rank0]:V2024-03-27 14:57:45,964.964000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST obj [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,964.964000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,964.964000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,964.964000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,964.964000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_IF_FALSE_OR_POP 106 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL hasattr []
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST obj [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST _asdict [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_IF_FALSE_OR_POP 106 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/nn/parallel/scatter_gather.py:15 in _is_namedtuple (_is_namedtuple) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _is_namedtuple at 0x7fd5282076d0, file "/data/users/willfeng/pytorch_yf225/torch/nn/parallel/scatter_gather.py", line 13>
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 990 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:124 in to_map (_recursive_to.to_map) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if isinstance(obj, tuple) and len(obj) > 0:
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL isinstance []
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF obj [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 1142 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF obj [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,965.965000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP > [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 1142 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:125 in to_map (_recursive_to.to_map) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return list(zip(*map(to_map, obj)))
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL list []
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL zip [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL map [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF to_map [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF obj [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,966.966000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call to_map from /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:125 in to_map (_recursive_to.to_map) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return list(zip(*map(to_map, obj)))
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object to_map at 0x4c29920, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 92>
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 COPY_FREE_VARS 3
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 MAKE_CELL 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 92 4 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 93 6 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_GLOBAL 2 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 LOAD_ATTR 2 (Tensor)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 LOAD_GLOBAL 6 (PackedSequence)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 54 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 EXTENDED_ARG 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 72 POP_JUMP_FORWARD_IF_FALSE 407 (to 888)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 94 74 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 LOAD_GLOBAL 6 (PackedSequence)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 100 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 104 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 114 POP_JUMP_FORWARD_IF_FALSE 12 (to 140)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 118 LOAD_ATTR 4 (data)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 LOAD_ATTR 5 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 JUMP_FORWARD 6 (to 152)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 140 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 142 LOAD_ATTR 5 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 152 STORE_FAST 1 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 95 154 LOAD_FAST 1 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 156 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 158 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 POP_JUMP_FORWARD_IF_FALSE 3 (to 172)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 96 166 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 170 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 97 >> 172 LOAD_DEREF 8 (use_side_stream_for_tensor_copies)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 174 POP_JUMP_FORWARD_IF_TRUE 22 (to 220)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 98 176 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 178 LOAD_METHOD 6 (to)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 202 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 206 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 216 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 101 >> 220 LOAD_GLOBAL 15 (NULL + getattr)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 232 LOAD_GLOBAL 2 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 244 LOAD_FAST 1 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 246 LOAD_ATTR 8 (type)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 256 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 258 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 262 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 272 STORE_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 102 274 LOAD_FAST 1 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 276 LOAD_ATTR 8 (type)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 286 LOAD_CONST 1 ('cpu')
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 288 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 294 POP_JUMP_FORWARD_IF_TRUE 2 (to 300)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 296 LOAD_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 298 POP_JUMP_FORWARD_IF_NOT_NONE 22 (to 344)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 103 >> 300 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 302 LOAD_METHOD 6 (to)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 324 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 326 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 330 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 340 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 342 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 >> 344 LOAD_GLOBAL 19 (NULL + _get_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 356 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 358 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 362 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 372 STORE_FAST 3 (stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 107 374 LOAD_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 376 LOAD_METHOD 10 (stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 398 LOAD_FAST 3 (stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 400 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 404 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 414 BEFORE_WITH
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 416 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 108 418 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 420 LOAD_METHOD 6 (to)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 442 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 444 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 448 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 458 STORE_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 107 460 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 462 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 464 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 466 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 470 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 480 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 482 JUMP_FORWARD 11 (to 506)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 484 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 486 WITH_EXCEPT_START
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 488 POP_JUMP_FORWARD_IF_TRUE 4 (to 498)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 490 RERAISE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 492 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 494 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 496 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 498 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 500 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 502 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 504 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 >> 506 LOAD_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 508 LOAD_METHOD 5 (device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 530 LOAD_DEREF 6 (target_device)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 532 LOAD_ATTR 11 (index)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 542 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 546 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 556 BEFORE_WITH
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 558 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 111 560 LOAD_FAST 2 (device_mod)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 562 LOAD_METHOD 12 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 584 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 588 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 598 STORE_FAST 5 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 113 600 LOAD_FAST 5 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 602 LOAD_METHOD 13 (wait_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 624 LOAD_FAST 3 (stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 626 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 630 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 640 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 642 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 654 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 656 LOAD_GLOBAL 6 (PackedSequence)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 668 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 672 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 682 POP_JUMP_FORWARD_IF_FALSE 27 (to 738)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 117 684 LOAD_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 686 LOAD_ATTR 4 (data)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 696 LOAD_METHOD 14 (record_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 718 LOAD_FAST 5 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 720 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 724 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 734 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 736 JUMP_FORWARD 49 (to 836)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 119 >> 738 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 750 LOAD_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 752 LOAD_GLOBAL 2 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 764 LOAD_ATTR 2 (Tensor)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 774 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 778 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 788 POP_JUMP_FORWARD_IF_TRUE 2 (to 794)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 790 LOAD_ASSERTION_ERROR
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 RAISE_VARARGS 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 >> 794 LOAD_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 796 LOAD_METHOD 14 (record_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 818 LOAD_FAST 5 (current_stream)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 820 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 824 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 834 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 >> 836 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 838 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 840 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 842 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 846 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 856 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 858 JUMP_FORWARD 11 (to 882)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 860 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 862 WITH_EXCEPT_START
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 864 POP_JUMP_FORWARD_IF_TRUE 4 (to 874)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 866 RERAISE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 868 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 870 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 872 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 874 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 876 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 878 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 880 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 121 >> 882 LOAD_FAST 4 (output)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 884 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 886 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 122 >> 888 LOAD_GLOBAL 31 (NULL + _is_namedtuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 900 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 902 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 906 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 916 POP_JUMP_FORWARD_IF_FALSE 35 (to 988)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 123 918 LOAD_CLOSURE 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 920 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 922 LOAD_CONST 2 (<code object <listcomp> at 0x7fd5281fef10, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 123>)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 924 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 926 LOAD_GLOBAL 33 (NULL + zip)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 938 LOAD_GLOBAL 35 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 950 LOAD_DEREF 7 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 952 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 954 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 958 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 968 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 970 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 972 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 976 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 986 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 124 >> 988 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1000 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1002 LOAD_GLOBAL 36 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1014 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1018 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1028 POP_JUMP_FORWARD_IF_FALSE 55 (to 1140)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1030 LOAD_GLOBAL 39 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1042 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1044 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1048 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1058 LOAD_CONST 3 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1060 COMPARE_OP 4 (>)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1066 POP_JUMP_FORWARD_IF_FALSE 36 (to 1140)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 125 1068 LOAD_GLOBAL 41 (NULL + list)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1080 LOAD_GLOBAL 33 (NULL + zip)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1092 LOAD_GLOBAL 35 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1104 LOAD_DEREF 7 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1106 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1108 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1112 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1122 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1124 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1128 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1138 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 126 >> 1140 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1152 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1154 LOAD_GLOBAL 40 (list)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1166 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1170 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1180 POP_JUMP_FORWARD_IF_FALSE 52 (to 1286)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1182 LOAD_GLOBAL 39 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1194 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1196 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1200 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1210 LOAD_CONST 3 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1212 COMPARE_OP 4 (>)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1218 POP_JUMP_FORWARD_IF_FALSE 33 (to 1286)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 127 1220 LOAD_CONST 4 (<code object <listcomp> at 0x7fd5281fea60, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 127>)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1222 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1224 LOAD_GLOBAL 33 (NULL + zip)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1236 LOAD_GLOBAL 35 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1248 LOAD_DEREF 7 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1250 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1252 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1256 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1266 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1268 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1270 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1274 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1284 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 >> 1286 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1298 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1300 LOAD_GLOBAL 42 (dict)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1312 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1316 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1326 POP_JUMP_FORWARD_IF_FALSE 72 (to 1472)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1328 LOAD_GLOBAL 39 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1340 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1342 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1346 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1356 LOAD_CONST 3 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1358 COMPARE_OP 4 (>)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1364 POP_JUMP_FORWARD_IF_FALSE 53 (to 1472)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 129 1366 LOAD_CLOSURE 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1368 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1370 LOAD_CONST 5 (<code object <listcomp> at 0x7fd528028230, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 129>)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1372 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1374 LOAD_GLOBAL 33 (NULL + zip)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1386 LOAD_GLOBAL 35 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1398 LOAD_DEREF 7 (to_map)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1400 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1402 LOAD_METHOD 22 (items)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1424 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1428 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1438 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1442 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1452 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1454 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1456 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1460 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1470 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 130 >> 1472 LOAD_DEREF 0 (obj)
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1474 BUILD_LIST 1
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 1476 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] ExceptionTable:
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 416 to 458 -> 484 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 484 to 490 -> 492 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 498 to 498 -> 492 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 558 to 834 -> 860 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 860 to 866 -> 868 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 874 to 874 -> 868 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:45,967.967000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,968.968000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COPY_FREE_VARS 3 []
[rank0]:[rank0]:V2024-03-27 14:57:45,968.968000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL obj []
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:92 in to_map (_recursive_to.to_map) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def to_map(obj):
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:93 in to_map (_recursive_to.to_map) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if isinstance(obj, (torch.Tensor, PackedSequence)):
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL isinstance []
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF obj [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch [NullVariable(), BuiltinVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR Tensor [NullVariable(), BuiltinVariable(), LazyVariableTracker(), PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL PackedSequence [NullVariable(), BuiltinVariable(), LazyVariableTracker(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [NullVariable(), BuiltinVariable(), LazyVariableTracker(), UserDefinedClassVariable(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), LazyVariableTracker(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), LazyVariableTracker(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 890 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:94 in to_map (_recursive_to.to_map) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] device = obj.data.device if isinstance(obj, PackedSequence) else obj.device
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL isinstance []
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF obj [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL PackedSequence [NullVariable(), BuiltinVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), LazyVariableTracker(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), LazyVariableTracker(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 138 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF obj []
[rank0]:[rank0]:V2024-03-27 14:57:45,969.969000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR device [LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST device [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:95 in to_map (_recursive_to.to_map) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if device == target_device:
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST device []
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF target_device [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP == [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 170 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:96 in to_map (_recursive_to.to_map) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (obj,)
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF obj []
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 1 [LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object to_map at 0x4c29920, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 92>
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL_FUNCTION_EX 0 [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object to_map at 0x4c29920, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 92>
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST res [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:136 in _recursive_to (_recursive_to) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,970.970000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] to_map = None # type: ignore[assignment]
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_DEREF to_map [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_FORWARD 66 []
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:137 in _recursive_to (_recursive_to) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return res
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST res []
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _recursive_to at 0x7fd528028730, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 89>
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_FORWARD 42 [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:249 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] moved_inputs = (
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST moved_inputs [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:256 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if kwargs
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST kwargs []
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:255 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] _recursive_to(kwargs, target_device, use_side_stream_for_tensor_copies)
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 82 [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:257 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] else []
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:254 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] moved_kwargs = (
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST moved_kwargs [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:259 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if len(moved_inputs) < len(moved_kwargs):
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST moved_inputs [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,971.971000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST moved_kwargs [ConstantVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [ConstantVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [ConstantVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP < [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 298 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:261 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] elif len(moved_kwargs) < len(moved_inputs):
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST moved_kwargs [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST moved_inputs [ConstantVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [ConstantVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [ConstantVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP < [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 508 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:262 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] moved_kwargs.extend([{} for _ in range(len(moved_inputs) - len(moved_kwargs))])
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST moved_kwargs []
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD extend [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object <listcomp> at 0x7fd52a1917c0, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 262> [NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 0 [NullVariable(), GetAttrVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL range [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,972.972000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST moved_inputs [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST moved_kwargs [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), ConstantVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), ConstantVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), ConstantVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_OP 10 [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), NullVariable(), BuiltinVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), RangeVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), GetAttrVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call <listcomp> from /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:262 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] moved_kwargs.extend([{} for _ in range(len(moved_inputs) - len(moved_kwargs))])
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object <listcomp> at 0x7fd52a1917c0, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 262>
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 262 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_FAST 0 (.0)
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 6 FOR_ITER 4 (to 16)
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 8 STORE_FAST 1 (_)
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 BUILD_MAP 0
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LIST_APPEND 2
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 JUMP_BACKWARD 5 (to 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 16 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,973.973000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:262 in <listcomp> (_to_kwargs) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] moved_kwargs.extend([{} for _ in range(len(moved_inputs) - len(moved_kwargs))])
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST .0 [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 16 [ListVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST _ [ListVariable(), ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_MAP 0 [ListVariable(), ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LIST_APPEND 2 [ListVariable(), ListIteratorVariable(length=1, index=1), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_BACKWARD 6 [ListVariable(), ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 16 [ListVariable(), ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object <listcomp> at 0x7fd52a1917c0, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 262>
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), GetAttrVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), GetAttrVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/utils.py:263 in _to_kwargs (_to_kwargs) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return tuple(moved_inputs), tuple(moved_kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple []
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST moved_inputs [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,974.974000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST moved_kwargs [TupleVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [TupleVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [TupleVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _to_kwargs at 0x4c2ab50, file "/data/users/willfeng/pytorch_yf225/torch/distributed/utils.py", line 243>
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE UNPACK_SEQUENCE 2 [WithExitFunctionVariable(), WithExitFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST args_tuple [WithExitFunctionVariable(), WithExitFunctionVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST kwargs_tuple [WithExitFunctionVariable(), WithExitFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:159 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] with torch.profiler.record_function("FSDP::inputs_to_device"):
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [WithExitFunctionVariable(), WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [WithExitFunctionVariable(), WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [WithExitFunctionVariable(), WithExitFunctionVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [WithExitFunctionVariable(), WithExitFunctionVariable(), ConstantVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [WithExitFunctionVariable(), WithExitFunctionVariable(), ConstantVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_FORWARD 614 [WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:163 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] args, kwargs = args_tuple[0], kwargs_tuple[0]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST args_tuple [WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 [WithExitFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [WithExitFunctionVariable(), TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST kwargs_tuple [WithExitFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 [WithExitFunctionVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [WithExitFunctionVariable(), TupleVariable(), TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST kwargs [WithExitFunctionVariable(), TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,975.975000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST args [WithExitFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:152 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] with torch.profiler.record_function("FSDP::root_pre_forward"):
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [WithExitFunctionVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [WithExitFunctionVariable(), ConstantVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [WithExitFunctionVariable(), ConstantVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_FORWARD 692 []
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:164 in _root_pre_forward (FSDPState._root_pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return args, kwargs
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST args []
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST kwargs [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _root_pre_forward at 0x7efa660, file "/data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py", line 145>
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE UNPACK_SEQUENCE 2 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST args [ConstDictVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST kwargs [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:49 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self._mp_policy.cast_forward_inputs and self._mp_policy.param_dtype:
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _mp_policy [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,976.976000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR cast_forward_inputs [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 432 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _mp_policy [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR param_dtype [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 432 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:50 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] with torch.profiler.record_function("FSDP::cast_forward_inputs"):
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch []
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR profiler [PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD record_function [PythonModuleVariable(<module 'torch.profiler' from '/data/users/willfeng/pytorch_yf225/torch/profiler/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST FSDP::cast_forward_inputs [NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BEFORE_WITH None [NullContextVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:51 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] cast_fn = functools.partial(
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL functools [WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR partial [WithExitFunctionVariable(), NullVariable(), PythonModuleVariable(<module 'functools' from '/data/users/willfeng/miniconda3/lib/python3.11/functools.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:52 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,977.977000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] _cast_fp_tensor, self._mp_policy.param_dtype
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _cast_fp_tensor [WithExitFunctionVariable(), NullVariable(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [WithExitFunctionVariable(), NullVariable(), UserDefinedClassVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _mp_policy [WithExitFunctionVariable(), NullVariable(), UserDefinedClassVariable(), UserFunctionVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR param_dtype [WithExitFunctionVariable(), NullVariable(), UserDefinedClassVariable(), UserFunctionVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:51 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] cast_fn = functools.partial(
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [WithExitFunctionVariable(), NullVariable(), UserDefinedClassVariable(), UserFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [WithExitFunctionVariable(), NullVariable(), UserDefinedClassVariable(), UserFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST cast_fn [WithExitFunctionVariable(), FunctoolsPartialVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:54 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] args, kwargs = tree_map(cast_fn, args), tree_map(cast_fn, kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tree_map [WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST cast_fn [WithExitFunctionVariable(), NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST args [WithExitFunctionVariable(), NullVariable(), UserFunctionVariable(), FunctoolsPartialVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [WithExitFunctionVariable(), NullVariable(), UserFunctionVariable(), FunctoolsPartialVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,978.978000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [WithExitFunctionVariable(), NullVariable(), UserFunctionVariable(), FunctoolsPartialVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call tree_map from /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:54 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] args, kwargs = tree_map(cast_fn, args), tree_map(cast_fn, kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object tree_map at 0x7fd577fe5920, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 859>
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 MAKE_CELL 6 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 859 2 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 898 4 LOAD_GLOBAL 1 (NULL + tree_flatten)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_FAST 1 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 KW_NAMES 1
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 40 STORE_FAST 4 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 STORE_DEREF 6 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 899 44 LOAD_FAST 4 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 BUILD_LIST 1
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 LOAD_CLOSURE 6 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 LOAD_CONST 2 (<code object <listcomp> at 0x7fd567f03c30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 899>)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 54 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 LOAD_FAST 3 (rests)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 BINARY_OP 0 (+)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 78 STORE_FAST 5 (flat_args)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 900 80 LOAD_DEREF 6 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 LOAD_METHOD 1 (unflatten)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 104 LOAD_GLOBAL 5 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 LOAD_FAST 0 (func)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 118 BUILD_LIST 1
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 LOAD_FAST 5 (flat_args)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 122 LIST_EXTEND 1
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 124 LIST_TO_TUPLE
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 126 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 132 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 142 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL treespec []
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:859 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,979.979000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def tree_map(
[rank0]:[rank0]:V2024-03-27 14:57:45,991.991000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,991.991000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:898 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,991.991000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] leaves, treespec = tree_flatten(tree, is_leaf=is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,991.991000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tree_flatten []
[rank0]:[rank0]:V2024-03-27 14:57:45,991.991000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,991.991000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST is_leaf [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,991.991000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE KW_NAMES ('is_leaf',) [NullVariable(), UserFunctionVariable(), TupleVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,991.991000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), UserFunctionVariable(), TupleVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,991.991000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), UserFunctionVariable(), TupleVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call tree_flatten from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:898 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] leaves, treespec = tree_flatten(tree, is_leaf=is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object tree_flatten at 0x7fd567f553e0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 799>
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 799 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 806 2 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 STORE_FAST 2 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 807 6 LOAD_GLOBAL 1 (NULL + _tree_flatten_helper)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_FAST 2 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 LOAD_FAST 1 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 KW_NAMES 1
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 40 STORE_FAST 3 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 808 42 LOAD_FAST 2 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 LOAD_FAST 3 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:799 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def tree_flatten(
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:806 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] leaves: List[Any] = []
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST leaves [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:807 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] spec = _tree_flatten_helper(tree, leaves, is_leaf=is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _tree_flatten_helper []
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST is_leaf [NullVariable(), UserFunctionVariable(), TupleVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,992.992000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE KW_NAMES ('is_leaf',) [NullVariable(), UserFunctionVariable(), TupleVariable(), ListVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 3 [NullVariable(), UserFunctionVariable(), TupleVariable(), ListVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 3 [NullVariable(), UserFunctionVariable(), TupleVariable(), ListVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _tree_flatten_helper from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:807 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] spec = _tree_flatten_helper(tree, leaves, is_leaf=is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _tree_flatten_helper at 0x7fd59043adb0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 778>
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 MAKE_CELL 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 MAKE_CELL 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 778 4 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 783 6 LOAD_GLOBAL 1 (NULL + _is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_DEREF 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 KW_NAMES 1
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 POP_JUMP_FORWARD_IF_FALSE 28 (to 96)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 784 40 LOAD_DEREF 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 LOAD_METHOD 1 (append)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 66 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 80 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 785 82 LOAD_GLOBAL 4 (_LEAF_SPEC)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 94 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 787 >> 96 LOAD_GLOBAL 7 (NULL + _get_node_type)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 108 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 114 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 124 STORE_FAST 3 (node_type)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 788 126 LOAD_GLOBAL 8 (SUPPORTED_NODES)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 LOAD_FAST 3 (node_type)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 LOAD_ATTR 5 (flatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 STORE_FAST 4 (flatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 789 162 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 LOAD_FAST 4 (flatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 166 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 172 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 STORE_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 188 STORE_FAST 6 (context)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 190 LOAD_CLOSURE 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 192 LOAD_CLOSURE 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 194 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 LOAD_CONST 2 (<code object <listcomp> at 0x7fd567f552f0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 792>)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 198 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 793 200 LOAD_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 202 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 204 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 208 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 STORE_FAST 7 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 796 220 LOAD_GLOBAL 13 (NULL + TreeSpec)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 232 LOAD_FAST 3 (node_type)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 234 LOAD_FAST 6 (context)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 LOAD_FAST 7 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 238 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 242 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 252 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL leaves []
[rank0]:[rank0]:V2024-03-27 14:57:45,993.993000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:778 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _tree_flatten_helper(
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:783 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _is_leaf(tree, is_leaf=is_leaf):
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF is_leaf [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE KW_NAMES ('is_leaf',) [NullVariable(), UserFunctionVariable(), TupleVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), UserFunctionVariable(), TupleVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), UserFunctionVariable(), TupleVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _is_leaf from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:783 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if _is_leaf(tree, is_leaf=is_leaf):
[rank0]:[rank0]:V2024-03-27 14:57:45,994.994000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _is_leaf at 0x7fd577fdf880, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 608>
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 608 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 609 2 LOAD_FAST 1 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 IS_OP 1
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 8 POP_JUMP_FORWARD_IF_FALSE 11 (to 32)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_FAST 1 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 JUMP_IF_TRUE_OR_POP 21 (to 74)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 32 LOAD_GLOBAL 1 (NULL + _get_node_type)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 610 44 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 609 46 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 611 60 LOAD_GLOBAL 2 (SUPPORTED_NODES)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 609 72 CONTAINS_OP 1
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 74 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:608 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _is_leaf(tree: PyTree, is_leaf: Optional[Callable[[PyTree], bool]] = None) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE IS_OP 1 [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 32 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _get_node_type []
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:610 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] tree
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,995.995000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _get_node_type from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~^
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] tree
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ) not in SUPPORTED_NODES
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 601 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 602 2 LOAD_GLOBAL 1 (NULL + _is_namedtuple_instance)
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 POP_JUMP_FORWARD_IF_FALSE 7 (to 46)
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 603 32 LOAD_GLOBAL 2 (namedtuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 604 >> 46 LOAD_GLOBAL 5 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:601 in _get_node_type (_get_node_type) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _get_node_type(tree: Any) -> Any:
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _is_namedtuple_instance []
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,996.996000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _is_namedtuple_instance from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 590 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 591 2 LOAD_GLOBAL 1 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 STORE_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 592 32 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 LOAD_ATTR 1 (__bases__)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 STORE_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 593 46 LOAD_GLOBAL 5 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 POP_JUMP_FORWARD_IF_TRUE 17 (to 118)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 84 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 LOAD_CONST 2 (0)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 98 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 POP_JUMP_FORWARD_IF_FALSE 2 (to 122)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 594 >> 118 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 595 >> 122 LOAD_GLOBAL 9 (NULL + getattr)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 LOAD_CONST 4 ('_fields')
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 154 STORE_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 596 156 LOAD_GLOBAL 11 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 170 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 POP_JUMP_FORWARD_IF_TRUE 2 (to 202)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 597 198 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 598 >> 202 LOAD_GLOBAL 13 (NULL + all)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 214 LOAD_CONST 5 (<code object <genexpr> at 0x7fd567f03a30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 598>)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 216 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 222 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 226 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 240 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:590 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _is_namedtuple_instance(tree: Any) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:591 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] typ = type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST typ [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:592 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] bases = typ.__bases__
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST typ []
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR __bases__ [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST bases [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:593 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:45,997.997000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if len(bases) != 1 or bases[0] != tuple:
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 118 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases []
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [BuiltinVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 122 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:594 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return False
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST False []
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 46 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:604 in _get_node_type (_get_node_type) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,998.998000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:45,999.999000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:611 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,999.999000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] ) not in SUPPORTED_NODES
[rank0]:[rank0]:V2024-03-27 14:57:45,999.999000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL SUPPORTED_NODES [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:45,999.999000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:45,999.999000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:45,999.999000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CONTAINS_OP 1 [BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _is_leaf at 0x7fd577fdf880, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 608>
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 96 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:787 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] node_type = _get_node_type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _get_node_type []
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _get_node_type from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:787 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] node_type = _get_node_type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 601 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 602 2 LOAD_GLOBAL 1 (NULL + _is_namedtuple_instance)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 POP_JUMP_FORWARD_IF_FALSE 7 (to 46)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 603 32 LOAD_GLOBAL 2 (namedtuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 604 >> 46 LOAD_GLOBAL 5 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:601 in _get_node_type (_get_node_type) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,000.000000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _get_node_type(tree: Any) -> Any:
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _is_namedtuple_instance []
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _is_namedtuple_instance from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 590 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 591 2 LOAD_GLOBAL 1 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 STORE_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 592 32 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 LOAD_ATTR 1 (__bases__)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 STORE_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 593 46 LOAD_GLOBAL 5 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 POP_JUMP_FORWARD_IF_TRUE 17 (to 118)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 84 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 LOAD_CONST 2 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 98 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 POP_JUMP_FORWARD_IF_FALSE 2 (to 122)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 594 >> 118 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 595 >> 122 LOAD_GLOBAL 9 (NULL + getattr)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 LOAD_CONST 4 ('_fields')
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 154 STORE_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 596 156 LOAD_GLOBAL 11 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 170 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 POP_JUMP_FORWARD_IF_TRUE 2 (to 202)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 597 198 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 598 >> 202 LOAD_GLOBAL 13 (NULL + all)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 214 LOAD_CONST 5 (<code object <genexpr> at 0x7fd567f03a30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 598>)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 216 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 222 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 226 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 240 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:590 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,001.001000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _is_namedtuple_instance(tree: Any) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:591 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] typ = type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST typ [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:592 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] bases = typ.__bases__
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST typ []
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR __bases__ [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST bases [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:593 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if len(bases) != 1 or bases[0] != tuple:
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 118 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases []
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [BuiltinVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,002.002000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 122 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:594 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return False
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST False []
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 46 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:604 in _get_node_type (_get_node_type) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST node_type [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:788 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] flatten_fn = SUPPORTED_NODES[node_type].flatten_fn
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL SUPPORTED_NODES []
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST node_type [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [ConstDictVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,003.003000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR flatten_fn [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST flatten_fn [UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:789 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] child_pytrees, context = flatten_fn(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PUSH_NULL None []
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST flatten_fn [NullVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _tuple_flatten from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:789 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] child_pytrees, context = flatten_fn(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _tuple_flatten at 0x7fd567f5c3b0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 363>
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 363 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 364 2 LOAD_GLOBAL 1 (NULL + list)
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (d)
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:363 in _tuple_flatten (_tuple_flatten) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _tuple_flatten(d: Tuple[Any, ...]) -> Tuple[List[Any], Context]:
[rank0]:[rank0]:V2024-03-27 14:57:46,004.004000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:364 in _tuple_flatten (_tuple_flatten) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return list(d), None
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL list []
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST d [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [ListVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _tuple_flatten at 0x7fd567f5c3b0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 363>
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE UNPACK_SEQUENCE 2 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST child_pytrees [ConstantVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST context [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:792 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] children_specs = [
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CLOSURE is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CLOSURE leaves [NewCellVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [NewCellVariable(), NewCellVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object <listcomp> at 0x7fd567f552f0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 792> [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 8 [TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:793 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,005.005000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] _tree_flatten_helper(child, leaves, is_leaf=is_leaf) for child in child_pytrees
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST child_pytrees [NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:792 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] children_specs = [
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [NestedUserFunctionVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NestedUserFunctionVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NestedUserFunctionVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call <listcomp> from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:792 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] children_specs = [
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] _tree_flatten_helper(child, leaves, is_leaf=is_leaf) for child in child_pytrees
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ]
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object <listcomp> at 0x7fd567f552f0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 792>
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 COPY_FREE_VARS 2
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 2 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 LOAD_FAST 0 (.0)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 8 FOR_ITER 20 (to 50)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 793 10 STORE_FAST 1 (child)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_GLOBAL 1 (NULL + _tree_flatten_helper)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 LOAD_FAST 1 (child)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 LOAD_DEREF 3 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 LOAD_DEREF 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 KW_NAMES 0
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 46 LIST_APPEND 2
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 JUMP_BACKWARD 21 (to 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 50 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COPY_FREE_VARS 2 []
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:792 in <listcomp> (_tree_flatten_helper) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,006.006000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] children_specs = [
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST .0 [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 50 [ListVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:793 in <listcomp> (_tree_flatten_helper) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] _tree_flatten_helper(child, leaves, is_leaf=is_leaf) for child in child_pytrees
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST child [ListVariable(), ListIteratorVariable(length=1, index=1), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _tree_flatten_helper [ListVariable(), ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST child [ListVariable(), ListIteratorVariable(length=1, index=1), NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF leaves [ListVariable(), ListIteratorVariable(length=1, index=1), NullVariable(), UserFunctionVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF is_leaf [ListVariable(), ListIteratorVariable(length=1, index=1), NullVariable(), UserFunctionVariable(), TensorVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE KW_NAMES ('is_leaf',) [ListVariable(), ListIteratorVariable(length=1, index=1), NullVariable(), UserFunctionVariable(), TensorVariable(), ListVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 3 [ListVariable(), ListIteratorVariable(length=1, index=1), NullVariable(), UserFunctionVariable(), TensorVariable(), ListVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,007.007000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 3 [ListVariable(), ListIteratorVariable(length=1, index=1), NullVariable(), UserFunctionVariable(), TensorVariable(), ListVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _tree_flatten_helper from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:793 in <listcomp> (_tree_flatten_helper) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] _tree_flatten_helper(child, leaves, is_leaf=is_leaf) for child in child_pytrees
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _tree_flatten_helper at 0x7fd59043adb0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 778>
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 MAKE_CELL 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 MAKE_CELL 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 778 4 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 783 6 LOAD_GLOBAL 1 (NULL + _is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_DEREF 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 KW_NAMES 1
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 POP_JUMP_FORWARD_IF_FALSE 28 (to 96)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 784 40 LOAD_DEREF 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 LOAD_METHOD 1 (append)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 66 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 80 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 785 82 LOAD_GLOBAL 4 (_LEAF_SPEC)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 94 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 787 >> 96 LOAD_GLOBAL 7 (NULL + _get_node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 108 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 114 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 124 STORE_FAST 3 (node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 788 126 LOAD_GLOBAL 8 (SUPPORTED_NODES)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 LOAD_FAST 3 (node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 LOAD_ATTR 5 (flatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 STORE_FAST 4 (flatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 789 162 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 LOAD_FAST 4 (flatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 166 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 172 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 STORE_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 188 STORE_FAST 6 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 190 LOAD_CLOSURE 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 192 LOAD_CLOSURE 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 194 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 LOAD_CONST 2 (<code object <listcomp> at 0x7fd567f552f0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 792>)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 198 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 793 200 LOAD_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 202 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 204 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 208 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 STORE_FAST 7 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 796 220 LOAD_GLOBAL 13 (NULL + TreeSpec)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 232 LOAD_FAST 3 (node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 234 LOAD_FAST 6 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 LOAD_FAST 7 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 238 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 242 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 252 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL leaves []
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:778 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _tree_flatten_helper(
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:783 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _is_leaf(tree, is_leaf=is_leaf):
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF is_leaf [NullVariable(), UserFunctionVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE KW_NAMES ('is_leaf',) [NullVariable(), UserFunctionVariable(), TensorVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), UserFunctionVariable(), TensorVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,008.008000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), UserFunctionVariable(), TensorVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _is_leaf from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:783 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if _is_leaf(tree, is_leaf=is_leaf):
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _is_leaf at 0x7fd577fdf880, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 608>
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 608 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 609 2 LOAD_FAST 1 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 IS_OP 1
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 8 POP_JUMP_FORWARD_IF_FALSE 11 (to 32)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_FAST 1 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 JUMP_IF_TRUE_OR_POP 21 (to 74)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 32 LOAD_GLOBAL 1 (NULL + _get_node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 610 44 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 609 46 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 611 60 LOAD_GLOBAL 2 (SUPPORTED_NODES)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 609 72 CONTAINS_OP 1
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 74 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:608 in _is_leaf (_is_leaf) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _is_leaf(tree: PyTree, is_leaf: Optional[Callable[[PyTree], bool]] = None) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE IS_OP 1 [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 32 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _get_node_type []
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:610 in _is_leaf (_is_leaf) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] tree
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _get_node_type from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~^
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] tree
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ) not in SUPPORTED_NODES
[rank0]:[rank0]:V2024-03-27 14:57:46,009.009000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 601 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 602 2 LOAD_GLOBAL 1 (NULL + _is_namedtuple_instance)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 POP_JUMP_FORWARD_IF_FALSE 7 (to 46)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 603 32 LOAD_GLOBAL 2 (namedtuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 604 >> 46 LOAD_GLOBAL 5 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:601 in _get_node_type (_get_node_type) (inline depth: 9)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _get_node_type(tree: Any) -> Any:
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 9)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _is_namedtuple_instance []
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _is_namedtuple_instance from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 9)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 590 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 591 2 LOAD_GLOBAL 1 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 STORE_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 592 32 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 LOAD_ATTR 1 (__bases__)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 STORE_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 593 46 LOAD_GLOBAL 5 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 POP_JUMP_FORWARD_IF_TRUE 17 (to 118)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 84 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 LOAD_CONST 2 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 98 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 POP_JUMP_FORWARD_IF_FALSE 2 (to 122)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 594 >> 118 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 595 >> 122 LOAD_GLOBAL 9 (NULL + getattr)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 LOAD_CONST 4 ('_fields')
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 154 STORE_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 596 156 LOAD_GLOBAL 11 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 170 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 POP_JUMP_FORWARD_IF_TRUE 2 (to 202)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 597 198 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 598 >> 202 LOAD_GLOBAL 13 (NULL + all)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 214 LOAD_CONST 5 (<code object <genexpr> at 0x7fd567f03a30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 598>)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 216 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 222 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 226 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 240 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,010.010000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:590 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 10)
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _is_namedtuple_instance(tree: Any) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:591 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 10)
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] typ = type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST typ [UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:592 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 10)
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] bases = typ.__bases__
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST typ []
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR __bases__ [UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST bases [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:593 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 10)
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if len(bases) != 1 or bases[0] != tuple:
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,011.011000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 118 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases []
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [UserDefinedClassVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 122 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:594 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 10)
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return False
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST False []
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 46 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:604 in _get_node_type (_get_node_type) (inline depth: 9)
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:611 in _is_leaf (_is_leaf) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] ) not in SUPPORTED_NODES
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL SUPPORTED_NODES [UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,012.012000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CONTAINS_OP 1 [UserDefinedClassVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _is_leaf at 0x7fd577fdf880, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 608>
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 96 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:784 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] leaves.append(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF leaves []
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD append [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), GetAttrVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), GetAttrVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:785 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return _LEAF_SPEC
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _LEAF_SPEC []
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _tree_flatten_helper at 0x7fd59043adb0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 778>
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:792 in <listcomp> (_tree_flatten_helper) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] children_specs = [
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LIST_APPEND 2 [ListVariable(), ListIteratorVariable(length=1, index=1), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_BACKWARD 8 [ListVariable(), ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 50 [ListVariable(), ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,013.013000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object <listcomp> at 0x7fd567f552f0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 792>
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST children_specs [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:796 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return TreeSpec(node_type, context, children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL TreeSpec []
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST node_type [NullVariable(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST context [NullVariable(), UserDefinedClassVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST children_specs [NullVariable(), UserDefinedClassVariable(), BuiltinVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 3 [NullVariable(), UserDefinedClassVariable(), BuiltinVariable(), ConstantVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 3 [NullVariable(), UserDefinedClassVariable(), BuiltinVariable(), ConstantVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call __init__ from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:796 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return TreeSpec(node_type, context, children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,014.014000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object __init__ at 0x7fd567f71020, file "<string>", line 2>
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 3 2 LOAD_FAST 1 (type)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 STORE_ATTR 0 (type)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 16 LOAD_FAST 2 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 STORE_ATTR 1 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 5 30 LOAD_FAST 3 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 STORE_ATTR 2 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 44 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 LOAD_METHOD 3 (__post_init__)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 68 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 72 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 84 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:2 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:3 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST type []
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR type [BuiltinVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:4 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST context []
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR context [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:5 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST children_specs []
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR children_specs [ListVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:6 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD __post_init__ [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,015.015000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call __post_init__ from <string>:6 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call]
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object __post_init__ at 0x7fd567f0dc50, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 629>
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 629 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 630 2 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_GLOBAL 1 (NULL + sum)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_CONST 2 (<code object <genexpr> at 0x7fd567f5c9d0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 630>)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 LOAD_ATTR 1 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 62 BINARY_OP 0 (+)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 66 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 68 STORE_ATTR 2 (num_nodes)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 631 78 LOAD_GLOBAL 1 (NULL + sum)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 90 LOAD_CONST 3 (<code object <genexpr> at 0x7fd567f5cab0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 631>)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 92 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 94 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 96 LOAD_ATTR 1 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 108 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 112 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 122 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 126 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 STORE_ATTR 3 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 632 148 LOAD_GLOBAL 9 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 162 LOAD_ATTR 1 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 172 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 176 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 188 STORE_ATTR 5 (num_children)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 198 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:629 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def __post_init__(self) -> None:
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:630 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_nodes = 1 + sum(spec.num_nodes for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 []
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL sum [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object <genexpr> at 0x7fd567f5c9d0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 630> [ConstantVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 0 [ConstantVariable(), NullVariable(), BuiltinVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,016.016000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR children_specs [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call <genexpr> from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:630 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] self.num_nodes = 1 + sum(spec.num_nodes for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object <genexpr> at 0x7fd567f5c9d0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 630>
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 630 0 RETURN_GENERATOR
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 LOAD_FAST 0 (.0)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 8 FOR_ITER 11 (to 32)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 STORE_FAST 1 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_FAST 1 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_ATTR 0 (num_nodes)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 YIELD_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 RESUME 1
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 JUMP_BACKWARD 12 (to 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 32 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:630 in <genexpr> (TreeSpec.__post_init__) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_nodes = 1 + sum(spec.num_nodes for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_GENERATOR None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST .0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 32 [ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST spec [ListIteratorVariable(length=1, index=1), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST spec [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,017.017000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_nodes [ListIteratorVariable(length=1, index=1), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE YIELD_VALUE None [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 1 [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_BACKWARD 8 [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 32 [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object <genexpr> at 0x7fd567f5c9d0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 630>
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [ConstantVariable(), NullVariable(), BuiltinVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [ConstantVariable(), NullVariable(), BuiltinVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_OP 0 [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR num_nodes [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:631 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_leaves = sum(spec.num_leaves for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL sum []
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object <genexpr> at 0x7fd567f5cab0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 631> [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 0 [NullVariable(), BuiltinVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR children_specs [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,018.018000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call <genexpr> from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:631 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] self.num_leaves = sum(spec.num_leaves for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object <genexpr> at 0x7fd567f5cab0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 631>
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 631 0 RETURN_GENERATOR
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 LOAD_FAST 0 (.0)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 8 FOR_ITER 11 (to 32)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 STORE_FAST 1 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_FAST 1 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_ATTR 0 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 YIELD_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 RESUME 1
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 JUMP_BACKWARD 12 (to 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 32 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:631 in <genexpr> (TreeSpec.__post_init__) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_leaves = sum(spec.num_leaves for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_GENERATOR None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST .0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 32 [ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST spec [ListIteratorVariable(length=1, index=1), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST spec [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_leaves [ListIteratorVariable(length=1, index=1), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE YIELD_VALUE None [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 1 [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_BACKWARD 8 [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 32 [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:46,019.019000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object <genexpr> at 0x7fd567f5cab0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 631>
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR num_leaves [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:632 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_children = len(self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR children_specs [NullVariable(), BuiltinVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR num_children [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object __post_init__ at 0x7fd567f0dc50, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 629>
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object __init__ at 0x7fd567f71020, file "<string>", line 2>
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _tree_flatten_helper at 0x7fd59043adb0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 778>
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST spec [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:808 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,020.020000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return leaves, spec
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves []
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST spec [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [ListVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object tree_flatten at 0x7fd567f553e0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 799>
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE UNPACK_SEQUENCE 2 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST leaves [UserDefinedObjectVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_DEREF treespec [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:899 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] flat_args = [leaves] + [treespec.flatten_up_to(r) for r in rests]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves []
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 1 [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CLOSURE treespec [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 1 [ListVariable(), NewCellVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object <listcomp> at 0x7fd567f03c30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 899> [ListVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 8 [ListVariable(), TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST rests [ListVariable(), NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [ListVariable(), NestedUserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [ListVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [ListVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call <listcomp> from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:899 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] flat_args = [leaves] + [treespec.flatten_up_to(r) for r in rests]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object <listcomp> at 0x7fd567f03c30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 899>
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 COPY_FREE_VARS 1
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 899 2 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 LOAD_FAST 0 (.0)
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 8 FOR_ITER 23 (to 56)
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 STORE_FAST 1 (r)
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_DEREF 2 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_METHOD 0 (flatten_up_to)
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 LOAD_FAST 1 (r)
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 LIST_APPEND 2
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 54 JUMP_BACKWARD 24 (to 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 56 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,021.021000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COPY_FREE_VARS 1 []
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:899 in <listcomp> (tree_map) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] flat_args = [leaves] + [treespec.flatten_up_to(r) for r in rests]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST .0 [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 56 [ListVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object <listcomp> at 0x7fd567f03c30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 899>
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_OP 0 [ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST flat_args [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:900 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return treespec.unflatten(map(func, *flat_args))
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF treespec []
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD unflatten [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL map [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST func [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 1 [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), FunctoolsPartialVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST flat_args [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LIST_EXTEND 1 [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LIST_TO_TUPLE None [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,022.022000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL_FUNCTION_EX 0 [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _cast_fp_tensor from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:900 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return treespec.unflatten(map(func, *flat_args))
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _cast_fp_tensor at 0x7fd50f4d3cb0, file "/data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_common.py", line 144>
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 146 2 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 1 (x)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_GLOBAL 2 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 LOAD_ATTR 2 (Tensor)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 145 52 POP_JUMP_FORWARD_IF_FALSE 31 (to 116)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 147 54 LOAD_GLOBAL 3 (NULL + torch)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 66 LOAD_ATTR 3 (is_floating_point)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 LOAD_FAST 1 (x)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 78 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 145 92 POP_JUMP_FORWARD_IF_FALSE 11 (to 116)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 148 94 LOAD_FAST 1 (x)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 96 LOAD_ATTR 4 (dtype)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 LOAD_FAST 0 (dtype)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 108 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 114 POP_JUMP_FORWARD_IF_FALSE 2 (to 120)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 >> 116 LOAD_FAST 1 (x)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 118 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 151 >> 120 LOAD_FAST 1 (x)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 122 LOAD_METHOD 5 (to)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 LOAD_FAST 0 (dtype)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 146 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_common.py:144 in _cast_fp_tensor (_cast_fp_tensor) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,023.023000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _cast_fp_tensor(dtype: torch.dtype, x: torch.Tensor) -> torch.Tensor:
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_common.py:146 in _cast_fp_tensor (_cast_fp_tensor) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] not isinstance(x, torch.Tensor)
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL isinstance []
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST x [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch [NullVariable(), BuiltinVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR Tensor [NullVariable(), BuiltinVariable(), TensorVariable(), PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), TensorVariable(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), TensorVariable(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_common.py:145 in _cast_fp_tensor (_cast_fp_tensor) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if (
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 116 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_common.py:147 in _cast_fp_tensor (_cast_fp_tensor) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] or not torch.is_floating_point(x)
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch []
[rank0]:[rank0]:V2024-03-27 14:57:46,025.025000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR is_floating_point [NullVariable(), PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST x [NullVariable(), TorchInGraphFunctionVariable(<built-in method is_floating_point of type object at 0x7fd58f929d80>)]
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), TorchInGraphFunctionVariable(<built-in method is_floating_point of type object at 0x7fd58f929d80>), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), TorchInGraphFunctionVariable(<built-in method is_floating_point of type object at 0x7fd58f929d80>), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_common.py:145 in _cast_fp_tensor (_cast_fp_tensor) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if (
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 116 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_common.py:150 in _cast_fp_tensor (_cast_fp_tensor) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return x
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST x []
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _cast_fp_tensor at 0x7fd50f4d3cb0, file "/data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_common.py", line 144>
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserMethodVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserMethodVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call unflatten from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:900 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return treespec.unflatten(map(func, *flat_args))
[rank0]:[rank0]:V2024-03-27 14:57:46,026.026000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object unflatten at 0x464f840, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 734>
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 734 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 735 2 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_GLOBAL 2 (list)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 LOAD_GLOBAL 4 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 40 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 POP_JUMP_FORWARD_IF_TRUE 15 (to 88)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 736 58 LOAD_GLOBAL 3 (NULL + list)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 72 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 STORE_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 737 >> 88 LOAD_GLOBAL 7 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 100 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 102 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 118 LOAD_ATTR 4 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 POP_JUMP_FORWARD_IF_FALSE 43 (to 222)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 738 136 LOAD_GLOBAL 11 (NULL + ValueError)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 739 148 LOAD_CONST 1 ('treespec.unflatten(leaves): `leaves` has length ')
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 LOAD_GLOBAL 7 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 162 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 178 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 180 LOAD_CONST 2 (' but the spec refers to a pytree that holds ')
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 740 182 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 184 LOAD_ATTR 4 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 739 194 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 LOAD_CONST 3 (' items (')
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 741 198 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 739 200 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 202 LOAD_CONST 4 (').')
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 204 BUILD_STRING 7
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 738 206 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 210 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 RAISE_VARARGS 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 743 >> 222 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 224 LOAD_METHOD 6 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 246 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 260 POP_JUMP_FORWARD_IF_FALSE 8 (to 278)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 744 262 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 264 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 266 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 276 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 746 >> 278 LOAD_GLOBAL 14 (SUPPORTED_NODES)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 290 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 292 LOAD_ATTR 8 (type)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 302 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 312 LOAD_ATTR 9 (unflatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 322 STORE_FAST 2 (unflatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 749 324 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 326 STORE_FAST 3 (start)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 750 328 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 330 STORE_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 751 332 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 334 STORE_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 752 336 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 338 LOAD_ATTR 10 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 348 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 350 FOR_ITER 62 (to 476)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 352 STORE_FAST 6 (child_spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 753 354 LOAD_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 356 LOAD_FAST 6 (child_spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 358 LOAD_ATTR 4 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 368 BINARY_OP 13 (+=)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 372 STORE_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 754 374 LOAD_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 376 LOAD_METHOD 11 (append)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 398 LOAD_FAST 6 (child_spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 400 LOAD_METHOD 12 (unflatten)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 422 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 424 LOAD_FAST 3 (start)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 426 LOAD_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 428 BUILD_SLICE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 430 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 440 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 444 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 454 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 458 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 468 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 755 470 LOAD_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 472 STORE_FAST 3 (start)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 474 JUMP_BACKWARD 63 (to 350)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 757 >> 476 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 478 LOAD_FAST 2 (unflatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 480 LOAD_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 482 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 484 LOAD_ATTR 13 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 494 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 498 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 508 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:734 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def unflatten(self, leaves: Iterable[Any]) -> PyTree:
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:735 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if not isinstance(leaves, (list, tuple)):
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL isinstance []
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL list [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 88 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:737 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,027.027000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if len(leaves) != self.num_leaves:
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_leaves [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 222 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:743 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self.is_leaf():
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD is_leaf [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call is_leaf from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:743 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if self.is_leaf():
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~^^
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object is_leaf at 0x7fd567f55020, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 650>
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 650 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 651 2 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_ATTR 0 (num_nodes)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 JUMP_IF_FALSE_OR_POP 10 (to 44)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 LOAD_ATTR 1 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 44 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,028.028000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:650 in is_leaf (TreeSpec.is_leaf) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def is_leaf(self) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:651 in is_leaf (TreeSpec.is_leaf) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return self.num_nodes == 1 and self.num_leaves == 1
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_nodes [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP == [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_IF_FALSE_OR_POP 44 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object is_leaf at 0x7fd567f55020, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 650>
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 278 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:746 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] unflatten_fn = SUPPORTED_NODES[self.type].unflatten_fn
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL SUPPORTED_NODES []
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR type [ConstDictVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,029.029000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [ConstDictVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR unflatten_fn [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST unflatten_fn [UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:749 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] start = 0
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST start [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:750 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] end = 0
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST end [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:751 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] child_pytrees = []
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST child_pytrees [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:752 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] for child_spec in self.children_specs:
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR children_specs [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 476 [ListIteratorVariable(length=1, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST child_spec [ListIteratorVariable(length=1, index=1), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:753 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] end += child_spec.num_leaves
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST end [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST child_spec [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_leaves [ListIteratorVariable(length=1, index=1), ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,030.030000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_OP 13 [ListIteratorVariable(length=1, index=1), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST end [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:754 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] child_pytrees.append(child_spec.unflatten(leaves[start:end]))
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST child_pytrees [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD append [ListIteratorVariable(length=1, index=1), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST child_spec [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD unflatten [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST start [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), NullVariable(), UserMethodVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST end [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), NullVariable(), UserMethodVariable(), TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_SLICE 2 [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), NullVariable(), UserMethodVariable(), TupleVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), NullVariable(), UserMethodVariable(), TupleVariable(), SliceVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), NullVariable(), UserMethodVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,031.031000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), NullVariable(), UserMethodVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call unflatten from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:754 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] child_pytrees.append(child_spec.unflatten(leaves[start:end]))
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object unflatten at 0x464f840, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 734>
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 734 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 735 2 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_GLOBAL 2 (list)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 LOAD_GLOBAL 4 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 40 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 POP_JUMP_FORWARD_IF_TRUE 15 (to 88)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 736 58 LOAD_GLOBAL 3 (NULL + list)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 72 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 STORE_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 737 >> 88 LOAD_GLOBAL 7 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 100 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 102 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 118 LOAD_ATTR 4 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 POP_JUMP_FORWARD_IF_FALSE 43 (to 222)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 738 136 LOAD_GLOBAL 11 (NULL + ValueError)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 739 148 LOAD_CONST 1 ('treespec.unflatten(leaves): `leaves` has length ')
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 LOAD_GLOBAL 7 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 162 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 178 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 180 LOAD_CONST 2 (' but the spec refers to a pytree that holds ')
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 740 182 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 184 LOAD_ATTR 4 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 739 194 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 LOAD_CONST 3 (' items (')
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 741 198 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 739 200 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 202 LOAD_CONST 4 (').')
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 204 BUILD_STRING 7
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 738 206 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 210 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 RAISE_VARARGS 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 743 >> 222 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 224 LOAD_METHOD 6 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 246 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 260 POP_JUMP_FORWARD_IF_FALSE 8 (to 278)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 744 262 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 264 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 266 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 276 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 746 >> 278 LOAD_GLOBAL 14 (SUPPORTED_NODES)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 290 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 292 LOAD_ATTR 8 (type)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 302 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 312 LOAD_ATTR 9 (unflatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 322 STORE_FAST 2 (unflatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 749 324 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 326 STORE_FAST 3 (start)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 750 328 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 330 STORE_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 751 332 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 334 STORE_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 752 336 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 338 LOAD_ATTR 10 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 348 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 350 FOR_ITER 62 (to 476)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 352 STORE_FAST 6 (child_spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 753 354 LOAD_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 356 LOAD_FAST 6 (child_spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 358 LOAD_ATTR 4 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 368 BINARY_OP 13 (+=)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 372 STORE_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 754 374 LOAD_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 376 LOAD_METHOD 11 (append)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 398 LOAD_FAST 6 (child_spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 400 LOAD_METHOD 12 (unflatten)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 422 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 424 LOAD_FAST 3 (start)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 426 LOAD_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 428 BUILD_SLICE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 430 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 440 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 444 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 454 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 458 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 468 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 755 470 LOAD_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 472 STORE_FAST 3 (start)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 474 JUMP_BACKWARD 63 (to 350)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 757 >> 476 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 478 LOAD_FAST 2 (unflatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 480 LOAD_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 482 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 484 LOAD_ATTR 13 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 494 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 498 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 508 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:734 in unflatten (TreeSpec.unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def unflatten(self, leaves: Iterable[Any]) -> PyTree:
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:735 in unflatten (TreeSpec.unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if not isinstance(leaves, (list, tuple)):
[rank0]:[rank0]:V2024-03-27 14:57:46,032.032000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL isinstance []
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL list [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 88 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:737 in unflatten (TreeSpec.unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if len(leaves) != self.num_leaves:
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,033.033000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_leaves [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 222 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:743 in unflatten (TreeSpec.unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self.is_leaf():
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD is_leaf [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call is_leaf from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:743 in unflatten (TreeSpec.unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if self.is_leaf():
[rank0]:[rank0]:V2024-03-27 14:57:46,034.034000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~^^
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object is_leaf at 0x7fd567f55020, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 650>
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 650 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 651 2 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_ATTR 0 (num_nodes)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 JUMP_IF_FALSE_OR_POP 10 (to 44)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 LOAD_ATTR 1 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 44 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:650 in is_leaf (TreeSpec.is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def is_leaf(self) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:651 in is_leaf (TreeSpec.is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return self.num_nodes == 1 and self.num_leaves == 1
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_nodes [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP == [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_IF_FALSE_OR_POP 44 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_leaves [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP == [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object is_leaf at 0x7fd567f55020, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 650>
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 278 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:744 in unflatten (TreeSpec.unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,035.035000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return leaves[0]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves []
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object unflatten at 0x464f840, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 734>
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [ListIteratorVariable(length=1, index=1), NullVariable(), GetAttrVariable(), TensorVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:755 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] start = end
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST end [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST start [ListIteratorVariable(length=1, index=1), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_BACKWARD 350 [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:752 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] for child_spec in self.children_specs:
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 476 [ListIteratorVariable(length=1, index=1)]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:757 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return unflatten_fn(child_pytrees, self.context)
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PUSH_NULL None []
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST unflatten_fn [NullVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST child_pytrees [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [NullVariable(), UserFunctionVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR context [NullVariable(), UserFunctionVariable(), ListVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), UserFunctionVariable(), ListVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,036.036000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), UserFunctionVariable(), ListVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _tuple_unflatten from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:757 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return unflatten_fn(child_pytrees, self.context)
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _tuple_unflatten at 0x7fd567f5c490, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 374>
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 374 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 375 2 LOAD_GLOBAL 1 (NULL + tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (values)
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:374 in _tuple_unflatten (_tuple_unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _tuple_unflatten(values: Iterable[Any], context: Context) -> Tuple[Any, ...]:
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:375 in _tuple_unflatten (_tuple_unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return tuple(values)
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple []
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST values [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,037.037000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _tuple_unflatten at 0x7fd567f5c490, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 374>
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object unflatten at 0x464f840, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 734>
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object tree_map at 0x7fd577fe5920, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 859>
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tree_map [WithExitFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST cast_fn [WithExitFunctionVariable(), TupleVariable(), NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST kwargs [WithExitFunctionVariable(), TupleVariable(), NullVariable(), UserFunctionVariable(), FunctoolsPartialVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [WithExitFunctionVariable(), TupleVariable(), NullVariable(), UserFunctionVariable(), FunctoolsPartialVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [WithExitFunctionVariable(), TupleVariable(), NullVariable(), UserFunctionVariable(), FunctoolsPartialVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call tree_map from /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:54 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] args, kwargs = tree_map(cast_fn, args), tree_map(cast_fn, kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object tree_map at 0x7fd577fe5920, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 859>
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 MAKE_CELL 6 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 859 2 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 898 4 LOAD_GLOBAL 1 (NULL + tree_flatten)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_FAST 1 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 KW_NAMES 1
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 40 STORE_FAST 4 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 STORE_DEREF 6 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 899 44 LOAD_FAST 4 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 BUILD_LIST 1
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 LOAD_CLOSURE 6 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 BUILD_TUPLE 1
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 LOAD_CONST 2 (<code object <listcomp> at 0x7fd567f03c30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 899>)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 54 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 LOAD_FAST 3 (rests)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 BINARY_OP 0 (+)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 78 STORE_FAST 5 (flat_args)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 900 80 LOAD_DEREF 6 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 LOAD_METHOD 1 (unflatten)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 104 LOAD_GLOBAL 5 (NULL + map)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 LOAD_FAST 0 (func)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 118 BUILD_LIST 1
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 LOAD_FAST 5 (flat_args)
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 122 LIST_EXTEND 1
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 124 LIST_TO_TUPLE
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 126 CALL_FUNCTION_EX 0
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 132 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 142 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,038.038000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL treespec []
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:859 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def tree_map(
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:898 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] leaves, treespec = tree_flatten(tree, is_leaf=is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tree_flatten []
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST is_leaf [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE KW_NAMES ('is_leaf',) [NullVariable(), UserFunctionVariable(), ConstDictVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), UserFunctionVariable(), ConstDictVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), UserFunctionVariable(), ConstDictVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call tree_flatten from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:898 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] leaves, treespec = tree_flatten(tree, is_leaf=is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object tree_flatten at 0x7fd567f553e0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 799>
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 799 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 806 2 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 STORE_FAST 2 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 807 6 LOAD_GLOBAL 1 (NULL + _tree_flatten_helper)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_FAST 2 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 LOAD_FAST 1 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 KW_NAMES 1
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 40 STORE_FAST 3 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 808 42 LOAD_FAST 2 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 LOAD_FAST 3 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:799 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def tree_flatten(
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:806 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] leaves: List[Any] = []
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST leaves [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:807 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,039.039000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] spec = _tree_flatten_helper(tree, leaves, is_leaf=is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _tree_flatten_helper []
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST is_leaf [NullVariable(), UserFunctionVariable(), ConstDictVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE KW_NAMES ('is_leaf',) [NullVariable(), UserFunctionVariable(), ConstDictVariable(), ListVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 3 [NullVariable(), UserFunctionVariable(), ConstDictVariable(), ListVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 3 [NullVariable(), UserFunctionVariable(), ConstDictVariable(), ListVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _tree_flatten_helper from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:807 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] spec = _tree_flatten_helper(tree, leaves, is_leaf=is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _tree_flatten_helper at 0x7fd59043adb0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 778>
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 MAKE_CELL 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 MAKE_CELL 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 778 4 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 783 6 LOAD_GLOBAL 1 (NULL + _is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_DEREF 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 KW_NAMES 1
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 POP_JUMP_FORWARD_IF_FALSE 28 (to 96)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 784 40 LOAD_DEREF 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 LOAD_METHOD 1 (append)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 66 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 80 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 785 82 LOAD_GLOBAL 4 (_LEAF_SPEC)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 94 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 787 >> 96 LOAD_GLOBAL 7 (NULL + _get_node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 108 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 114 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 124 STORE_FAST 3 (node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 788 126 LOAD_GLOBAL 8 (SUPPORTED_NODES)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 LOAD_FAST 3 (node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 LOAD_ATTR 5 (flatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 STORE_FAST 4 (flatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 789 162 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 LOAD_FAST 4 (flatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 166 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 172 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 STORE_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 188 STORE_FAST 6 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 190 LOAD_CLOSURE 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 192 LOAD_CLOSURE 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 194 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 LOAD_CONST 2 (<code object <listcomp> at 0x7fd567f552f0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 792>)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 198 MAKE_FUNCTION 8 (closure)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 793 200 LOAD_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 202 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 204 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 208 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 STORE_FAST 7 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 796 220 LOAD_GLOBAL 13 (NULL + TreeSpec)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 232 LOAD_FAST 3 (node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 234 LOAD_FAST 6 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 LOAD_FAST 7 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 238 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 242 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 252 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,040.040000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL leaves []
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_CELL is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:778 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _tree_flatten_helper(
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:783 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _is_leaf(tree, is_leaf=is_leaf):
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF is_leaf [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE KW_NAMES ('is_leaf',) [NullVariable(), UserFunctionVariable(), ConstDictVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), UserFunctionVariable(), ConstDictVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), UserFunctionVariable(), ConstDictVariable(), LazyVariableTracker()]
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _is_leaf from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:783 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if _is_leaf(tree, is_leaf=is_leaf):
[rank0]:[rank0]:V2024-03-27 14:57:46,041.041000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _is_leaf at 0x7fd577fdf880, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 608>
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 608 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 609 2 LOAD_FAST 1 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 IS_OP 1
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 8 POP_JUMP_FORWARD_IF_FALSE 11 (to 32)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_FAST 1 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 JUMP_IF_TRUE_OR_POP 21 (to 74)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 32 LOAD_GLOBAL 1 (NULL + _get_node_type)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 610 44 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 609 46 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 50 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 611 60 LOAD_GLOBAL 2 (SUPPORTED_NODES)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 609 72 CONTAINS_OP 1
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 74 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:608 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _is_leaf(tree: PyTree, is_leaf: Optional[Callable[[PyTree], bool]] = None) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE IS_OP 1 [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 32 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _get_node_type []
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:610 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] tree
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _get_node_type from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~^
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] tree
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ) not in SUPPORTED_NODES
[rank0]:[rank0]:V2024-03-27 14:57:46,042.042000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 601 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 602 2 LOAD_GLOBAL 1 (NULL + _is_namedtuple_instance)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 POP_JUMP_FORWARD_IF_FALSE 7 (to 46)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 603 32 LOAD_GLOBAL 2 (namedtuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 604 >> 46 LOAD_GLOBAL 5 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:601 in _get_node_type (_get_node_type) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _get_node_type(tree: Any) -> Any:
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _is_namedtuple_instance []
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _is_namedtuple_instance from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 590 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 591 2 LOAD_GLOBAL 1 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 STORE_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 592 32 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 LOAD_ATTR 1 (__bases__)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 STORE_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 593 46 LOAD_GLOBAL 5 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 POP_JUMP_FORWARD_IF_TRUE 17 (to 118)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 84 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 LOAD_CONST 2 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 98 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 POP_JUMP_FORWARD_IF_FALSE 2 (to 122)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 594 >> 118 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 595 >> 122 LOAD_GLOBAL 9 (NULL + getattr)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 LOAD_CONST 4 ('_fields')
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 154 STORE_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 596 156 LOAD_GLOBAL 11 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 170 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 POP_JUMP_FORWARD_IF_TRUE 2 (to 202)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 597 198 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 598 >> 202 LOAD_GLOBAL 13 (NULL + all)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 214 LOAD_CONST 5 (<code object <genexpr> at 0x7fd567f03a30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 598>)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 216 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 222 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 226 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 240 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,043.043000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:590 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _is_namedtuple_instance(tree: Any) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:591 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] typ = type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST typ [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:592 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] bases = typ.__bases__
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST typ []
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR __bases__ [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST bases [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:593 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if len(bases) != 1 or bases[0] != tuple:
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 118 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases []
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,044.044000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [BuiltinVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 122 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:594 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return False
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST False []
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 46 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:604 in _get_node_type (_get_node_type) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:611 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] ) not in SUPPORTED_NODES
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL SUPPORTED_NODES [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:609 in _is_leaf (_is_leaf) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return (is_leaf is not None and is_leaf(tree)) or _get_node_type(
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CONTAINS_OP 1 [BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _is_leaf at 0x7fd577fdf880, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 608>
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 96 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:787 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,045.045000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] node_type = _get_node_type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _get_node_type []
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _get_node_type from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:787 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] node_type = _get_node_type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 601 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 602 2 LOAD_GLOBAL 1 (NULL + _is_namedtuple_instance)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 POP_JUMP_FORWARD_IF_FALSE 7 (to 46)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 603 32 LOAD_GLOBAL 2 (namedtuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 604 >> 46 LOAD_GLOBAL 5 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:601 in _get_node_type (_get_node_type) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _get_node_type(tree: Any) -> Any:
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL _is_namedtuple_instance []
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,046.046000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _is_namedtuple_instance from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:602 in _get_node_type (_get_node_type) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if _is_namedtuple_instance(tree):
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 590 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 591 2 LOAD_GLOBAL 1 (NULL + type)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 STORE_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 592 32 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 LOAD_ATTR 1 (__bases__)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 STORE_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 593 46 LOAD_GLOBAL 5 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 60 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 74 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 POP_JUMP_FORWARD_IF_TRUE 17 (to 118)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 84 LOAD_FAST 2 (bases)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 LOAD_CONST 2 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 98 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 110 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 POP_JUMP_FORWARD_IF_FALSE 2 (to 122)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 594 >> 118 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 595 >> 122 LOAD_GLOBAL 9 (NULL + getattr)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 LOAD_FAST 1 (typ)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 LOAD_CONST 4 ('_fields')
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 140 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 144 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 154 STORE_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 596 156 LOAD_GLOBAL 11 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 170 LOAD_GLOBAL 6 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 POP_JUMP_FORWARD_IF_TRUE 2 (to 202)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 597 198 LOAD_CONST 3 (False)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 598 >> 202 LOAD_GLOBAL 13 (NULL + all)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 214 LOAD_CONST 5 (<code object <genexpr> at 0x7fd567f03a30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 598>)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 216 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 218 LOAD_FAST 3 (fields)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 222 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 226 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 236 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 240 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:590 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _is_namedtuple_instance(tree: Any) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:591 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] typ = type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST typ [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:592 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,047.047000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] bases = typ.__bases__
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST typ []
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR __bases__ [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST bases [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:593 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if len(bases) != 1 or bases[0] != tuple:
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 118 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST bases []
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [BuiltinVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 122 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:594 in _is_namedtuple_instance (_is_namedtuple_instance) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return False
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST False []
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _is_namedtuple_instance at 0x7fd59043abf0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 590>
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 46 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:604 in _get_node_type (_get_node_type) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return type(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL type []
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,048.048000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _get_node_type at 0x7fd577fdf770, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 601>
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST node_type [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:788 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] flatten_fn = SUPPORTED_NODES[node_type].flatten_fn
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL SUPPORTED_NODES []
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST node_type [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [ConstDictVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR flatten_fn [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST flatten_fn [UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:789 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] child_pytrees, context = flatten_fn(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PUSH_NULL None []
[rank0]:[rank0]:V2024-03-27 14:57:46,049.049000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST flatten_fn [NullVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST tree [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserFunctionVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _dict_flatten from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:789 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] child_pytrees, context = flatten_fn(tree)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _dict_flatten at 0x7fd577fa6e70, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 391>
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 391 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 392 2 LOAD_GLOBAL 1 (NULL + list)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 0 (d)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_METHOD 1 (values)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 66 LOAD_GLOBAL 1 (NULL + list)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 78 LOAD_FAST 0 (d)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 80 LOAD_METHOD 2 (keys)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 102 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 120 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 130 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 132 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:391 in _dict_flatten (_dict_flatten) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _dict_flatten(d: Dict[Any, Any]) -> Tuple[List[Any], Context]:
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:392 in _dict_flatten (_dict_flatten) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return list(d.values()), list(d.keys())
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL list []
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST d [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD values [NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), BuiltinVariable(), NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,050.050000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), BuiltinVariable(), NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), DictValues()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), DictValues()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL list [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST d [ListVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD keys [ListVariable(), NullVariable(), BuiltinVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [ListVariable(), NullVariable(), BuiltinVariable(), NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [ListVariable(), NullVariable(), BuiltinVariable(), NullVariable(), GetAttrVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [ListVariable(), NullVariable(), BuiltinVariable(), DictKeys()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [ListVariable(), NullVariable(), BuiltinVariable(), DictKeys()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _dict_flatten at 0x7fd577fa6e70, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 391>
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE UNPACK_SEQUENCE 2 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST child_pytrees [ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST context [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:792 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] children_specs = [
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CLOSURE is_leaf []
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CLOSURE leaves [NewCellVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,051.051000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [NewCellVariable(), NewCellVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object <listcomp> at 0x7fd567f552f0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 792> [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 8 [TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:793 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] _tree_flatten_helper(child, leaves, is_leaf=is_leaf) for child in child_pytrees
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST child_pytrees [NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:792 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] children_specs = [
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [NestedUserFunctionVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call <listcomp> from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:792 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] children_specs = [
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] _tree_flatten_helper(child, leaves, is_leaf=is_leaf) for child in child_pytrees
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object <listcomp> at 0x7fd567f552f0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 792>
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 COPY_FREE_VARS 2
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 2 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 LOAD_FAST 0 (.0)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 8 FOR_ITER 20 (to 50)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 793 10 STORE_FAST 1 (child)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_GLOBAL 1 (NULL + _tree_flatten_helper)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 LOAD_FAST 1 (child)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 LOAD_DEREF 3 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 LOAD_DEREF 2 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 KW_NAMES 0
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 PRECALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 CALL 3
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 792 46 LIST_APPEND 2
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 JUMP_BACKWARD 21 (to 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 50 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,052.052000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COPY_FREE_VARS 2 []
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:792 in <listcomp> (_tree_flatten_helper) (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] children_specs = [
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST .0 [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 50 [ListVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object <listcomp> at 0x7fd567f552f0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 792>
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST children_specs [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:796 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return TreeSpec(node_type, context, children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL TreeSpec []
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST node_type [NullVariable(), UserDefinedClassVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST context [NullVariable(), UserDefinedClassVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST children_specs [NullVariable(), UserDefinedClassVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 3 [NullVariable(), UserDefinedClassVariable(), BuiltinVariable(), ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,053.053000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 3 [NullVariable(), UserDefinedClassVariable(), BuiltinVariable(), ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call __init__ from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:796 in _tree_flatten_helper (_tree_flatten_helper) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return TreeSpec(node_type, context, children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object __init__ at 0x7fd567f71020, file "<string>", line 2>
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 3 2 LOAD_FAST 1 (type)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 STORE_ATTR 0 (type)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 16 LOAD_FAST 2 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 STORE_ATTR 1 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 5 30 LOAD_FAST 3 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 STORE_ATTR 2 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 44 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 LOAD_METHOD 3 (__post_init__)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 68 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 72 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 82 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 84 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:2 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:3 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST type []
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR type [BuiltinVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:4 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST context []
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR context [ListVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:5 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST children_specs []
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,054.054000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR children_specs [ListVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line <string>:6 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD __post_init__ [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call __post_init__ from <string>:6 in __init__ (inline depth: 6)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object __post_init__ at 0x7fd567f0dc50, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 629>
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 629 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 630 2 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_GLOBAL 1 (NULL + sum)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_CONST 2 (<code object <genexpr> at 0x7fd567f5c9d0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 630>)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 18 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 20 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 LOAD_ATTR 1 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 32 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 62 BINARY_OP 0 (+)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 66 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 68 STORE_ATTR 2 (num_nodes)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 631 78 LOAD_GLOBAL 1 (NULL + sum)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 90 LOAD_CONST 3 (<code object <genexpr> at 0x7fd567f5cab0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 631>)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 92 MAKE_FUNCTION 0
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 94 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 96 LOAD_ATTR 1 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 108 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 112 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 122 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 126 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 136 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 STORE_ATTR 3 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 632 148 LOAD_GLOBAL 9 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 160 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 162 LOAD_ATTR 1 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 172 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 176 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 186 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 188 STORE_ATTR 5 (num_children)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 198 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 200 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:629 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def __post_init__(self) -> None:
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:630 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_nodes = 1 + sum(spec.num_nodes for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 []
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL sum [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,055.055000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object <genexpr> at 0x7fd567f5c9d0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 630> [ConstantVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 0 [ConstantVariable(), NullVariable(), BuiltinVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR children_specs [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [ConstantVariable(), NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call <genexpr> from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:630 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] self.num_nodes = 1 + sum(spec.num_nodes for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object <genexpr> at 0x7fd567f5c9d0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 630>
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 630 0 RETURN_GENERATOR
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 LOAD_FAST 0 (.0)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 8 FOR_ITER 11 (to 32)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 STORE_FAST 1 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_FAST 1 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_ATTR 0 (num_nodes)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 YIELD_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 RESUME 1
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 JUMP_BACKWARD 12 (to 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 32 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:630 in <genexpr> (TreeSpec.__post_init__) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_nodes = 1 + sum(spec.num_nodes for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_GENERATOR None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST .0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 32 [ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,056.056000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object <genexpr> at 0x7fd567f5c9d0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 630>
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [ConstantVariable(), NullVariable(), BuiltinVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [ConstantVariable(), NullVariable(), BuiltinVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_OP 0 [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR num_nodes [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:631 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_leaves = sum(spec.num_leaves for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL sum []
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object <genexpr> at 0x7fd567f5cab0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 631> [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 0 [NullVariable(), BuiltinVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR children_specs [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), BuiltinVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call <genexpr> from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:631 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] self.num_leaves = sum(spec.num_leaves for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object <genexpr> at 0x7fd567f5cab0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 631>
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 631 0 RETURN_GENERATOR
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 2 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 LOAD_FAST 0 (.0)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 8 FOR_ITER 11 (to 32)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 STORE_FAST 1 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_FAST 1 (spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_ATTR 0 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 YIELD_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 RESUME 1
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 JUMP_BACKWARD 12 (to 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 32 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:631 in <genexpr> (TreeSpec.__post_init__) (inline depth: 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,057.057000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_leaves = sum(spec.num_leaves for spec in self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_GENERATOR None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST .0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 32 [ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object <genexpr> at 0x7fd567f5cab0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 631>
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR num_leaves [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:632 in __post_init__ (TreeSpec.__post_init__) (inline depth: 7)
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self.num_children = len(self.children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR children_specs [NullVariable(), BuiltinVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_ATTR num_children [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object __post_init__ at 0x7fd567f0dc50, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 629>
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None []
[rank0]:[rank0]:V2024-03-27 14:57:46,058.058000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object __init__ at 0x7fd567f71020, file "<string>", line 2>
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _tree_flatten_helper at 0x7fd59043adb0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 778>
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST spec [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:808 in tree_flatten (tree_flatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return leaves, spec
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves []
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST spec [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [ListVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object tree_flatten at 0x7fd567f553e0, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 799>
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE UNPACK_SEQUENCE 2 [TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST leaves [UserDefinedObjectVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_DEREF treespec [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:899 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] flat_args = [leaves] + [treespec.flatten_up_to(r) for r in rests]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves []
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 1 [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CLOSURE treespec [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 1 [ListVariable(), NewCellVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST <code object <listcomp> at 0x7fd567f03c30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 899> [ListVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE MAKE_FUNCTION 8 [ListVariable(), TupleVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST rests [ListVariable(), NestedUserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [ListVariable(), NestedUserFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [ListVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,059.059000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [ListVariable(), NestedUserFunctionVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call <listcomp> from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:899 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] flat_args = [leaves] + [treespec.flatten_up_to(r) for r in rests]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object <listcomp> at 0x7fd567f03c30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 899>
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 0 COPY_FREE_VARS 1
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 899 2 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 6 LOAD_FAST 0 (.0)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 8 FOR_ITER 23 (to 56)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 10 STORE_FAST 1 (r)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 12 LOAD_DEREF 2 (treespec)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_METHOD 0 (flatten_up_to)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 LOAD_FAST 1 (r)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 LIST_APPEND 2
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 54 JUMP_BACKWARD 24 (to 8)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 56 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COPY_FREE_VARS 1 []
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:899 in <listcomp> (tree_map) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] flat_args = [leaves] + [treespec.flatten_up_to(r) for r in rests]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST .0 [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 56 [ListVariable(), ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object <listcomp> at 0x7fd567f03c30, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 899>
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_OP 0 [ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST flat_args [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:900 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return treespec.unflatten(map(func, *flat_args))
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_DEREF treespec []
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD unflatten [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL map [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST func [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,060.060000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 1 [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), FunctoolsPartialVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST flat_args [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LIST_EXTEND 1 [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LIST_TO_TUPLE None [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL_FUNCTION_EX 0 [NullVariable(), UserMethodVariable(), NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), UserMethodVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), UserMethodVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call unflatten from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:900 in tree_map (tree_map) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return treespec.unflatten(map(func, *flat_args))
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object unflatten at 0x464f840, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 734>
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 734 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 735 2 LOAD_GLOBAL 1 (NULL + isinstance)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 LOAD_GLOBAL 2 (list)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 LOAD_GLOBAL 4 (tuple)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 40 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 42 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 56 POP_JUMP_FORWARD_IF_TRUE 15 (to 88)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 736 58 LOAD_GLOBAL 3 (NULL + list)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 70 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 72 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 76 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 86 STORE_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 737 >> 88 LOAD_GLOBAL 7 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 100 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 102 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 106 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 116 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 118 LOAD_ATTR 4 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 COMPARE_OP 3 (!=)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 134 POP_JUMP_FORWARD_IF_FALSE 43 (to 222)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 738 136 LOAD_GLOBAL 11 (NULL + ValueError)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 739 148 LOAD_CONST 1 ('treespec.unflatten(leaves): `leaves` has length ')
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 150 LOAD_GLOBAL 7 (NULL + len)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 162 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 178 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 180 LOAD_CONST 2 (' but the spec refers to a pytree that holds ')
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 740 182 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 184 LOAD_ATTR 4 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 739 194 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 196 LOAD_CONST 3 (' items (')
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 741 198 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 739 200 FORMAT_VALUE 0
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 202 LOAD_CONST 4 (').')
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 204 BUILD_STRING 7
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 738 206 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 210 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 220 RAISE_VARARGS 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 743 >> 222 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 224 LOAD_METHOD 6 (is_leaf)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 246 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 250 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 260 POP_JUMP_FORWARD_IF_FALSE 8 (to 278)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 744 262 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 264 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 266 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 276 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 746 >> 278 LOAD_GLOBAL 14 (SUPPORTED_NODES)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 290 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 292 LOAD_ATTR 8 (type)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 302 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 312 LOAD_ATTR 9 (unflatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 322 STORE_FAST 2 (unflatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 749 324 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 326 STORE_FAST 3 (start)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 750 328 LOAD_CONST 5 (0)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 330 STORE_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 751 332 BUILD_LIST 0
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 334 STORE_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 752 336 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 338 LOAD_ATTR 10 (children_specs)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 348 GET_ITER
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 350 FOR_ITER 62 (to 476)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 352 STORE_FAST 6 (child_spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 753 354 LOAD_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 356 LOAD_FAST 6 (child_spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 358 LOAD_ATTR 4 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 368 BINARY_OP 13 (+=)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 372 STORE_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 754 374 LOAD_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 376 LOAD_METHOD 11 (append)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 398 LOAD_FAST 6 (child_spec)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 400 LOAD_METHOD 12 (unflatten)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 422 LOAD_FAST 1 (leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 424 LOAD_FAST 3 (start)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 426 LOAD_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 428 BUILD_SLICE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 430 BINARY_SUBSCR
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 440 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 444 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 454 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 458 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 468 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 755 470 LOAD_FAST 4 (end)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 472 STORE_FAST 3 (start)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 474 JUMP_BACKWARD 63 (to 350)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 757 >> 476 PUSH_NULL
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 478 LOAD_FAST 2 (unflatten_fn)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 480 LOAD_FAST 5 (child_pytrees)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 482 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 484 LOAD_ATTR 13 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 494 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 498 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 508 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,061.061000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:734 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def unflatten(self, leaves: Iterable[Any]) -> PyTree:
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:735 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if not isinstance(leaves, (list, tuple)):
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL isinstance []
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL list [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL tuple [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_TUPLE 2 [NullVariable(), BuiltinVariable(), TupleVariable(), BuiltinVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), TupleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_TRUE 88 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:737 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if len(leaves) != self.num_leaves:
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL len []
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST leaves [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_leaves [ConstantVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,062.062000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP != [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 222 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:743 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self.is_leaf():
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD is_leaf [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 0 [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call is_leaf from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:743 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] if self.is_leaf():
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~^^
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object is_leaf at 0x7fd567f55020, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 650>
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 650 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 651 2 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 4 LOAD_ATTR 0 (num_nodes)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 16 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 22 JUMP_IF_FALSE_OR_POP 10 (to 44)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 LOAD_ATTR 1 (num_leaves)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 36 LOAD_CONST 1 (1)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 38 COMPARE_OP 2 (==)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 44 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:650 in is_leaf (TreeSpec.is_leaf) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def is_leaf(self) -> bool:
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:651 in is_leaf (TreeSpec.is_leaf) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return self.num_nodes == 1 and self.num_leaves == 1
[rank0]:[rank0]:V2024-03-27 14:57:46,063.063000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_nodes [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP == [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_IF_FALSE_OR_POP 44 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR num_leaves [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 1 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE COMPARE_OP == [ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object is_leaf at 0x7fd567f55020, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 650>
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 278 [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:746 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] unflatten_fn = SUPPORTED_NODES[self.type].unflatten_fn
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL SUPPORTED_NODES []
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,064.064000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR type [ConstDictVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BINARY_SUBSCR None [ConstDictVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR unflatten_fn [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST unflatten_fn [UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:749 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] start = 0
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST start [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:750 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] end = 0
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST end [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:751 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] child_pytrees = []
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BUILD_LIST 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST child_pytrees [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:752 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] for child_spec in self.children_specs:
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR children_specs [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE GET_ITER None [ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE FOR_ITER 476 [ListIteratorVariable(length=0, index=0)]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:757 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return unflatten_fn(child_pytrees, self.context)
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PUSH_NULL None []
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST unflatten_fn [NullVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST child_pytrees [NullVariable(), UserFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self [NullVariable(), UserFunctionVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR context [NullVariable(), UserFunctionVariable(), ListVariable(), UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), UserFunctionVariable(), ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,065.065000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), UserFunctionVariable(), ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call _dict_unflatten from /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:757 in unflatten (TreeSpec.unflatten) (inline depth: 4)
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] return unflatten_fn(child_pytrees, self.context)
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object _dict_unflatten at 0x7fd567f03630, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 402>
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 402 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 403 2 LOAD_GLOBAL 1 (NULL + dict)
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_GLOBAL 3 (NULL + zip)
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 26 LOAD_FAST 1 (context)
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 28 LOAD_FAST 0 (values)
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 30 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 34 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 44 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 58 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:402 in _dict_unflatten (_dict_unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def _dict_unflatten(values: Iterable[Any], context: Context) -> Dict[Any, Any]:
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py:403 in _dict_unflatten (_dict_unflatten) (inline depth: 5)
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] return dict(zip(context, values))
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL dict []
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL zip [NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,066.066000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST context [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST values [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [NullVariable(), BuiltinVariable(), NullVariable(), BuiltinVariable(), ListVariable(), ListVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), BuiltinVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object _dict_unflatten at 0x7fd567f03630, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 402>
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object unflatten at 0x464f840, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 734>
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RETURN_VALUE None [ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:2402] [0/0] DONE INLINING <code object tree_map at 0x7fd577fe5920, file "/data/users/willfeng/pytorch_yf225/torch/utils/_pytree.py", line 859>
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST kwargs [WithExitFunctionVariable(), TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE STORE_FAST args [WithExitFunctionVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:50 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] with torch.profiler.record_function("FSDP::cast_forward_inputs"):
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [WithExitFunctionVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,067.067000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST None [WithExitFunctionVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 2 [WithExitFunctionVariable(), ConstantVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 2 [WithExitFunctionVariable(), ConstantVariable(), ConstantVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE JUMP_FORWARD 432 []
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:55 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] if self._fsdp_param_group:
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _fsdp_param_group [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_JUMP_FORWARD_IF_FALSE 508 [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:56 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] args, kwargs = self._fsdp_param_group.pre_forward(module, args, kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST self []
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR _fsdp_param_group [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD pre_forward [UserDefinedObjectVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST module [NullVariable(), UserMethodVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST args [NullVariable(), UserMethodVariable(), FSDPManagedNNModuleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_FAST kwargs [NullVariable(), UserMethodVariable(), FSDPManagedNNModuleVariable(), TupleVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 3 [NullVariable(), UserMethodVariable(), FSDPManagedNNModuleVariable(), TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,068.068000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 3 [NullVariable(), UserMethodVariable(), FSDPManagedNNModuleVariable(), TupleVariable(), ConstDictVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] TRACE inlined call pre_forward from /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_state.py:56 in _fsdp_state_pre_forward (_fsdp_state_pre_forward) (inline depth: 2)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] args, kwargs = self._fsdp_param_group.pre_forward(module, args, kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2356] [0/0] [__trace_call] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] INLINING <code object pre_forward at 0x7fd50f473870, file "/data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_param_group.py", line 282>
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 282 0 RESUME 0
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 285 2 LOAD_GLOBAL 0 (torch)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 14 LOAD_ATTR 1 (profiler)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 24 LOAD_METHOD 2 (record_function)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 46 LOAD_CONST 1 ('FSDP::pre_forward')
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 48 PRECALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 52 CALL 1
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 62 BEFORE_WITH
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 286 66 LOAD_GLOBAL 6 (TrainingState)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 78 LOAD_ATTR 4 (FORWARD)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 88 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 90 STORE_ATTR 5 (_training_state)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 287 100 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 102 LOAD_METHOD 6 (unshard)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 124 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 128 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 138 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 288 140 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 142 LOAD_METHOD 7 (wait_for_unshard)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 164 PRECALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 168 CALL 0
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 178 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 289 180 LOAD_FAST 0 (self)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 182 LOAD_METHOD 8 (_register_post_backward_hook)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 204 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 206 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 208 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 212 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 222 UNPACK_SEQUENCE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 226 STORE_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 228 STORE_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 290 230 LOAD_FAST 2 (args)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 232 LOAD_FAST 3 (kwargs)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 234 BUILD_TUPLE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0]
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 285 236 SWAP 2
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 238 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 240 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 242 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 244 PRECALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 248 CALL 2
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 258 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 260 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 262 PUSH_EXC_INFO
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 264 WITH_EXCEPT_START
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 266 POP_JUMP_FORWARD_IF_TRUE 4 (to 276)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 268 RERAISE 2
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 270 COPY 3
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 272 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 274 RERAISE 1
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] >> 276 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 278 POP_EXCEPT
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 280 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 282 POP_TOP
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 284 LOAD_CONST 0 (None)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 286 RETURN_VALUE
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] ExceptionTable:
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 64 to 234 -> 262 [1] lasti
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 262 to 268 -> 270 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] 276 to 276 -> 270 [3] lasti
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:2357] [0/0] , inlined according trace_rules.lookup
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_param_group.py:282 in pre_forward (FSDPParamGroup.pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,069.069000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] def pre_forward(
[rank0]:[rank0]:V2024-03-27 14:57:46,074.074000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE RESUME 0 []
[rank0]:[rank0]:V2024-03-27 14:57:46,074.074000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_param_group.py:285 in pre_forward (FSDPParamGroup.pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,074.074000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] with torch.profiler.record_function("FSDP::pre_forward"):
[rank0]:[rank0]:V2024-03-27 14:57:46,074.074000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_GLOBAL torch []
[rank0]:[rank0]:V2024-03-27 14:57:46,075.075000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_ATTR profiler [PythonModuleVariable(<module 'torch' from '/data/users/willfeng/pytorch_yf225/torch/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:46,075.075000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_METHOD record_function [PythonModuleVariable(<module 'torch.profiler' from '/data/users/willfeng/pytorch_yf225/torch/profiler/__init__.py'>)]
[rank0]:[rank0]:V2024-03-27 14:57:46,075.075000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE LOAD_CONST FSDP::pre_forward [NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>)]
[rank0]:[rank0]:V2024-03-27 14:57:46,075.075000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE PRECALL 1 [NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,075.075000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE CALL 1 [NullVariable(), TorchCtxManagerClassVariable(<class 'torch.autograd.profiler.record_function'>), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,075.075000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE BEFORE_WITH None [NullContextVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,075.075000 140555295757312 torch/_dynamo/symbolic_convert.py:763] [0/0] TRACE POP_TOP None [WithExitFunctionVariable(), ConstantVariable()]
[rank0]:[rank0]:V2024-03-27 14:57:46,075.075000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] TRACE starts_line /data/users/willfeng/pytorch_yf225/torch/distributed/_composable/fsdp/_fsdp_param_group.py:286 in pre_forward (FSDPParamGroup.pre_forward) (inline depth: 3)
[rank0]:[rank0]:V2024-03-27 14:57:46,075.075000 140555295757312 torch/_dynamo/symbolic_convert.py:740] [0/0] [__trace_source] self._training_state = Train
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment