In order to connect to the remote machine without the need to provide *.pem key each time, you can add these lines to the ~/.ssh/config:
Host 1.2.3.4
User root
IdentityFile /path/to/my.pem
Also, if you're using midnight commander, you can now freely connect to the remote machine as well by just specifying user and host:
[email protected]