Hi Mel,
One has to ask: what were you doing with the door panels off? First would be to recover any screws or other fasteners etc that might have fallen in. If a door glass feels loose, possibly its not being 'pinched' adequately by the two bolts, brackets and rubber bits that hold it to the guide mechanism.
If you've had the panels off once, it'll be that much easier to do it a second time, right?

Maybe open the driver's side to start and 1) find any missing fasteners, and 2) make sure all of the electrical connectors were adequately locked in place. [Removing the speaker is wonderful as that's a massive access point in to the door]
If I'm not mistaken, I believe the driver's door module is the master to the pax door module as the slave. While it may be working to operate the slave, I guess it's possible it may not be able to make its own glass operate properly.
All said, we have to recall:
9 out of 10 electrical problems are a bad connection so it's really critical (-especially- with this car) that every pin of every connector be at least visually inspected carefully. I have a few magnifying lenses and good flashlights to help with that task and they're indeed all very helpful.
HTH!
Gord