Full credit goes to Cabel Sasser on this one for sharing the original tip.
In iTerm type:
$ sudo vim /etc/pam.d/sudo
Enter your credentials
Add the following line after the first line:
auth sufficient pam_tid.so
Your file should now look something like this:
# sudo: auth account password session auth sufficient pam_tid.so auth sufficient pam_smartcard.so auth required pam_opendirectory.so account required pam_permit.so password required pam_deny.so session required pam_permit.so
The sudo file is readonly. To save and exit type:
:wq!
(Optional since we're focusing on iTerm) To Reload your shell:
$ $SHELL
Go to Prefs -> Advanced -> Allow sessions to survive logging out and back in and set value to No.
Restart iTerm
Type any command with sudo to make sure it works:
$ sudo echo