Whenever you delete mail, it lands up in the Deleted Items folders. This necessitates your having to empty this folder every once in a while. However you can if you wish, make your Windows Live Mail empty the Deleted Items folder on Exit.
To do so, click on the Windows Live Mail button on the extreme top left and click on Options > Mail.
Click on Advanced tab and then click on maintenance.
Here check Empty messages from the ‘deleted Items’ folder on Exit.
Click Close > Apply > OK.
Now everytime you exit Windows Live Mail, it will empty the contents of the Deleted Items folder.
Do note should you ever want to recover a deleted mail item from this folder, you would not be able to do so now.