1. Use any PLIST editor.

2. Open Skype preference file (~/Library/Preferences/com.skype.skype.plist)

3. Change property DialpadOpen from YES to NO

4. Save the file.

This annoying dial pad is now always hidden and you can open it when needed.