Tramp: Waiting for prompts from remote shell

I have an issue with tramp against a specific host when I use ssh-keys for authentication. It works fine when I don't use ssh-key authentication, but I would like to prevent entering the password all the time.

Don't know if it's related, but I use zsh for shell.

I have followed both the tips of setting tramp-chunksize to 500 and adding the following code to .zshrc:

if [[ "$TERM" == "dumb" ]]then  unsetopt zle  unsetopt prompt_cr  unsetopt prompt_subst  unfunction precmd  unfunction preexec  PS1='$ 'fi

But it's not working and it's still hanging with the message Tramp: Waiting for prompts from remote shell

Any ideas on how to prevent this annoying issue? Thanks.