having a good pair of slim or skinny jeans helps a lot when throwing something like a roundhouse kick-- you don't have as big of a chance to get friction burns & your leg hair doesn't rip off. same concept applies to something like a leather jacket: there's a larger chance to avoid friction burns from taking hits, or dealing hits with ur elbows
shoes of course provide soooo much more benefit in a fight. having your feet stepped on barefoot is one of my least favorite feelings, so with shoes that's no longer a thing to worry about, and the sole really helps for both protection and your offensive ability with hook/heel kicks particularly
so basically in hand-to-hand combat casual clothes will just lessen the chance/severity of skin tearing and burning due to friction, but they provide negligable protection from direct impact so i wouldn't necessarily count it as armor