I would understand if that was happening all the time then yes I would take it to my ISP but since it loads normally in the mornings or just after midnight (UK Time) then I have no idea what else can I do or what can I tell to my ISP without sounding like and idiot ;o

(Me)- Hi, my game loads slowly in the evenings, please sort it out
(Virgin media technical support)- Please click Start (left hand corner) then please click on Internet Explorer
- I'm sorry but I don't use IE due to security concerns
- What browser do you use then?
- I'm using Chrome or Firefox
- Please start Chrome
- Ok but how will that help me?
- Oh it's just something we always want to check
- Which is?
- Please go on speedtest website
- Dude I just told you that my game is loading slowly, what the @*$! do you want me to do speed test for?...
Clear the mess on your end and I'll do the same on my end hoping it will do the trick.

So that's what I can expect from my ISP and I had nearly the same conversation with them regarding latency issues which they could only resolve by telling me to change settings in router and they cleaned on their end as well.
When I had fibre optic broadband without phone line it was running perfect but since I moved to different city I'm on broadband through phone line and it's bad imo. slow, with issues and costly.

I have sent TICKET TO SUPPORT OVER 2 WEEKS AGO but I'm yet to receive any response. I guess it's just lucky me with crap broadband and my ticket getting lost in the post.
If I can't rely on support then who can I rely on? in game chat or perhaps forum (which seems to be a ghost town) ?

If anyone knows any magic trick to resolve my issue I would be glad.

Thanks.