Well, it's not letting me comment, but YouTube hates me sometimes.
You should have given him as much junk as you could like I did, I filled his inventory up so I'd be as prepared as possible. In fact, after creating a distraction, I accidentally discovered I could retrieve my items through the wall just beyond the door he opened. That's how good I was.
Also, you should have had Lydia wait. I read there's a problem with followers where they'd keep trying to get in but can't, so they get messed up and lost sometimes.
Anyone see problems with what I typed? I wish it'd tell you what you're doing wrong, but then starfishs would change one letter to bypass it, so I understand that.