If the floor is hard (tile, wood, etc) I will generally wear shoes unless I see whoever I am with take theirs off. If the floor is carpet, I will usually either ask whether I should take them off or just take them off.
After reading through your comments, I forget how nice where I live is. Since I live in San Diego, my shoes have never been muddy in their lifetimes. Once, my old pair of shoes got muddy, but I was in Miami. The bottom of my shoes are the color of the sole, but a little darker, and I've never washed them. Anyway, as a result nobody around here cares if you walk on their hard floors with shoes on, because there's virtually nothing to be tracked inside. If your shoes are dirty, then that is an uncommon ailment and most people would know that and take them off because they know they're dirty. However, it's pretty common for people to not want shoes on their carpet. It's also fairly (but less so, I'd say) common for people to not care because, again, the average pair of shoes here is not that dirty.