I found a lot of temporary files in \User\AppData\Local\Temp. Normally this is only an issue if you're short on space, but I found even temporary downloaded Internet files there (I was suspecting they would be saved somewhere). This is an issue if you're downloading private files (e.g. bank statements) that you don't want to keep.
Supposedly you can delete everything in Temp\ and it won't affect properly-written programs. I tried out the program CCleaner which is supposed to do that and more. It works, but it deletes recent files from Office (ok, fine), even those you've pinned to the menu!