Well all of the important parts of modern physics are contained in:
Weinberg: Quantum Theory of Fields (Three Volumes).
Shankar: Principles of Quantum Mechanics, 2nd edition.
Wald: General Relativity.
Goldstein: Classical Mechanics, 3rd Edition.
Those four books would be enough to reconstruct everything we know about the physical world. However since we are just choosing books for their intellectual worth as creative works, I'd go with Weinberg, Volume 1. It would allow people to recover the art of thinking like a physicist, which I think is closer to what Straggler was aiming for.
I wouldn't include Newton's Principia, as I don't know what it would demonstrate to them. Newton didn't use Calculus throughout the whole Principia since there was cultural pressure at the time against its foundational ideas. For that reason the whole thing is written using ugly geometric arguments that take pages to prove what Newton himself could prove in four lines using calculus. I would rather include Euclid since it shows, once again, the art of how to be a mathematician, the way a mathematician thinks. It would be between Euclid and a book called "Munkres: Topology", I don't know which I'd choose.
That ends the part where I somewhat know what I'm talking about.
One thing I'm not sure of is what language some of these books should be in. I read Euclid in Thomas Heath's translation, the best English translation in my opinion, especially for the footnotes. I've also read it in Gaelic, in a translation of about the same age as Heath. It's funny how the different translations brought different aspects across. After talking to somebody who actually knew Greek, I think it was because sometimes English brought the Greek across better, sometimes Gaelic did. However they were obviously shadows of the "real thing" in some way. For that reason I'd include Euclid in the original Greek perhaps.
For Eastern literature I'd have:
Murasaki Shikibu: The tale of Genji
Cao Xueqin: The story of the stone/Dream of the Red Chamber (It's hard to drop the other Chinese classics, but I think this is the best of them)
Mencius, this is basically a book about the Chinese philosopher Mencius. I honestly think he develops Confucian philosophy better than Confucius, who was sometimes a bit vague.
I think it's important to include Chinese philosophy since it has a different focus to Western Philosophy, it's typically more focused on the ethics of everyday actions, which contrasts with the general abstraction of Western Philosophy. (I'm painting both with broad brush stokes here).
Anyway, enough rambling!