This will sound ridiculous, but Darling is wearing green Margaret is wearing green under her coat, Olivia wears green, maybe Olivia was born after Darling and Margaret got together
Now humor me for a second here, perhaps Gray discovered Olivia at some point and bought, or traded for her with Darling, or somehow stole Olivia when she was just a newborn, which considering the fact that Gray Mann was stolen (by a bird) as a newborn, this may not seem far off from something that he'd do
I love speculation
And i also love how Spy just has an array of miniscule objects on his person