Why do I have white on my hand after I rub my siding?

This is oxidation and it is usually chalky. This can be cleaned off but it takes a lot of time and labor to remove this from your siding there is no easy way to get it off.