Once you have started the SSH agent with:

eval $(ssh-agent)

you have to add your private key to it:

ssh-add

This will ask you your passphrase just once, and then you should be allowed to push, provided that you uploaded the public key to the git server.

github - Git keeps asking me for my ssh key passphrase - Stack Overflow