That's very strange. It really shouldn't take that long and the server is not struggling as far as I can see. All I can assume is that either something has broken with your connection to the server or it is your connection. Not sure but I can guess that during patching if you experience packet loss or they arrive in the wrong order then the client has to download them again.

There is probably a 90% chance that the Devs can't do anything about this. It would be nice if the most update GBF file could be hosted somewhere for those that seem to be able to play fine but can't patch (Maybe the patching method needs a look over to make sure no bugs are creeping in)