I guess the only assumption I feel I'm making is that, regardless of how fast technology develops it develops instantaneously over geologic time.
Up to a point, but I don't think we can auomatically assume that there won't be a brick wall we can't overcome between current levels of technology and that required to engineer on a galactic scale. Simple tool use, after all, seems to have been around for many millions of years in all sorts of species, but it took the arisal of our species to turn this into the complex technology we have today. There could be an upper limit to our acheivements.
Of course, it's also possible that galatic scale engineering just isn't acheivable full stop, due to basic physicl constraints on how much energy can be produced or how fast it's possible to travel.
On the more general question of whether intelligent life would arise everywhere at about the same time, this seems very unlikely to me. Looking at the history of life on our planet, there seem to be certain jumps in its development. It's possible that, for most of the life's prescence here, there were no eukaryotic cells. Their creation could well be a bizarre and unlikely fluke, without which the biosphere would still just consist of varieties of bacteria.
Given the vastness of the universe, the same fluke (or similar flukes allowing multicellular, inteligent life to eventually appear) could happen plenty of times, but happening at the same time seems unlikely even on geological scales. It might take 4 billion years on planet A, 1 billion years on planet B, 8.5 billion yearson planet C etc. etc.
As for the Fermi paradox, the universe is huge! Why would we expect to notice evidence of civillisations unimaginably distant from us. We've only recently been able to (indirectly) observe the prescence of planets in other solar systems.