JPEG is for when you want to share something online and don't want to eat up peoples bandwidth.
PNG is for editing, giving out when a user wants a higher-quality version, or for certain types of pictures.
When I say "certain type," I mean things that tend to have a lot of regions that are the exact same colour, like text on a constant background, pixel art, etc.
Then there are more exotic types, like WebP, gif (For static images), apng, etc. Let's not get into those...