Topic: WHAT IS CAUSING THE DISCONNECTS?

I've had almost no problems before today (1 or 2 in the last week).  Now I'm getting chain disconnects on both accounts and at the moment it's so bad I can't even get back in the game at all even with just 1 running.

There are no other problems with either my machine or connection, I've been on corp ts the entire time.

Edit: Please do not use profanity in the forums. - DEV Calvin

Re: WHAT IS CAUSING THE DISCONNECTS?

same here disconnects really bad tonight. Just a net problem, I am sure...

John 3:16 - Timothy 2:23

Re: WHAT IS CAUSING THE DISCONNECTS?

Noticed that. Its been growing worse. After the third discon I said to hell with it. Its not worth losing my gear over. If this keeps up, I can see lots of people just leaving.

If you can't kill it, don't make it mad.

Re: WHAT IS CAUSING THE DISCONNECTS?

I think I like the game, overall. However, this issue will kill the market for the casual US players if they persist at this frequency.

Re: WHAT IS CAUSING THE DISCONNECTS?

Would be nice to get a Dev reply on what is causing the disconnects if their is ANYTHING they can do server side to help. I've played free to play MMO's with worse servers and not got DC'ed half as much as I do with Perpetuum.

Re: WHAT IS CAUSING THE DISCONNECTS?

Here is what we plan to do about this issue:

- we will introduce a monitoring system that allows us to collect disconnect data, hopefully without user interaction
- we will establish which hosting service is the best for us in Hungary (there are only two major ones, so this shouldn't be too hard)
- we will get in contact with the companies where our packets get lost and try to solve the routing issues
- we will assess the costs involved in moving the server to a better location

In the meantime the more detailed data we have from you, the better. We need to find a pattern, but it is hard because the majority of players - even from the US - do not experience this.

Re: WHAT IS CAUSING THE DISCONNECTS?

DEV Calvin wrote:

Here is what we plan to do about this issue:

- we will introduce a monitoring system that allows us to collect disconnect data, hopefully without user interaction
- we will establish which hosting service is the best for us in Hungary (there are only two major ones, so this shouldn't be too hard)
- we will get in contact with the companies where our packets get lost and try to solve the routing issues
- we will assess the costs involved in moving the server to a better location

In the meantime the more detailed data we have from you, the better. We need to find a pattern, but it is hard because the majority of players - even from the US - do not experience this.

Thanks for the reply. smile
It would be nice if the client (After DC'ing) would have the server/client runa  build in traceroute an report the data back to the server for later analysis. This would be useful when ping start to spike. If it count be backgrounded (And disabled for PVPers who don't want any more network traffic than we already have to use) then people who play on alpha or in safe areas and lag a lot could let the client gather the data by itself without any interaction. It would also make the data easier to sort and see patterns because you wouldn't have to deal with different formatting issues.

Re: WHAT IS CAUSING THE DISCONNECTS?

Looking into hosting is OK, but they all use the same NAP, since there's only one out of Hungary, and if your going to move going across the road or across the continent is probably the same level of logistics; you just can't fluff the server's pillow and kiss it good night anymore smile

Monitoring is great, I think you'll find that the issue is more widespread than what is being reported here on the forums.

Be sure to do a TCO on a new hosting site, there's more than just bw and rack space to look at, I'd rather you stay where you are then over-extend and have to shutdown sad

Re: WHAT IS CAUSING THE DISCONNECTS?

Arga wrote:

Looking into hosting is OK, but they all use the same NAP, since there's only one out of Hungary, and if your going to move going across the road or across the continent is probably the same level of logistics; you just can't fluff the server's pillow and kiss it good night anymore smile

Monitoring is great, I think you'll find that the issue is more widespread than what is being reported here on the forums.

Be sure to do a TCO on a new hosting site, there's more than just bw and rack space to look at, I'd rather you stay where you are then over-extend and have to shutdown sad


Well... If they stay where they are, and matters get worse, I suspect they will lose enough players to make the game even more unprofitable than it is now.

Lets face it, this isn't almost seven years ago, when CCP started EVE. There are any number of games that players can spend their time and money on these days.  I like Perpetuum(or I wouldn't still be here). But as it stands its becoming unplayable.

I may let that go on for a few months(just to build up EP), but if it continues beyond that, I'll have to cut my losses and regretfully move on. I suspect many others have a similar perspective.  Hosting a game like this in Hungry, would be much like CCP attempting to host EVE out of Iceland.

The telcom infrastructure in eastern Europe just isn't ready for prime time.  Its improved a bit over the years, but its no where near what it needs to be for this type of large scale, precise application.

If you can't kill it, don't make it mad.

Re: WHAT IS CAUSING THE DISCONNECTS?

Crepitus wrote:

I've had almost no problems before today (1 or 2 in the last week).  Now I'm getting chain disconnects on both accounts and at the moment it's so bad I can't even get back in the game at all even with just 1 running.

There are no other problems with either my machine or connection, I've been on corp ts the entire time.

Edit: Please do not use profanity in the forums. - DEV Calvin

2 1/2 month old necrothread is still relevant.  The win saddens me ;/

Re: WHAT IS CAUSING THE DISCONNECTS?

from my analysis, it really doesn't look like a 'eastern' EU infrastructure issue, it looks like the problem is in Germany; but there could be some government packet inspection (firewalling) and not a hardware issue for all in/out to eastern EU if fraNAP is government ownd *tinfoil hat in place*

Re: WHAT IS CAUSING THE DISCONNECTS?

Arga wrote:

from my analysis, it really doesn't look like a 'eastern' EU infrastructure issue, it looks like the problem is in Germany; but there could be some government packet inspection (firewalling) and not a hardware issue for all in/out to eastern EU if fraNAP is government ownd *tinfoil hat in place*


You would think that if it's the only link out as you said before (I have no reason to doubt you I just have looked myself) that there would be enough people *** that they would do something about it.  I don't just mean for Perpetuum but for all the businesses and whoever else using that link.

Re: WHAT IS CAUSING THE DISCONNECTS?

CrepitusGoldenGoose wrote:
Arga wrote:

from my analysis, it really doesn't look like a 'eastern' EU infrastructure issue, it looks like the problem is in Germany; but there could be some government packet inspection (firewalling) and not a hardware issue for all in/out to eastern EU if fraNAP is government ownd *tinfoil hat in place*


You would think that if it's the only link out as you said before (I have no reason to doubt you I just have looked myself) that there would be enough people *** that they would do something about it.  I don't just mean for Perpetuum but for all the businesses and whoever else using that link.

What makes you think they haven't? Keep in mind that much of that is still State controlled, and as such is largely unaccountable to the users. It also tends to attract the least skilled personnel(other wise they'd be making more in the private sector...) ^^ That's a NASTY combination that one finds in many of the People's Utopias over there. Its slowly changing, but they are having to fight generations worth of damage done by governments of the past.

If you look at one of the recent maps of the connections, there are several single(or at most dual) connection bottlenecks.

Look at Hungry on a map, and then examine the countries that border it on all sides.  Thats exactly what I was speaking of.

If you can't kill it, don't make it mad.

Re: WHAT IS CAUSING THE DISCONNECTS?

No, it's not just 1 'link', its a single Network Access Point (NAP). US has moved away with public NAP's in favor of private peering between the large ISP's (the public naps still provide access for smaller ISP's), so in that respect I agree with Wriath's assesment of the EU.

Hungary isn't the USA. By having a single NAP, they can control and monitor all traffic in/out of the country. But again, the single NAP is in Hungary, while the traffic issue is in Germany. So changing hosts in budapest isn't going to change how traffic is routed, because it all goes out the same NAP, to fraNAP, because that's the lowest hop count to the USA. Players in the EU will have a different experience, because they don't need to hop on the AC-1 fiber link that land in Germany.

Re: WHAT IS CAUSING THE DISCONNECTS?

Wraithbane wrote:

What makes you think they haven't? Keep in mind that much of that is still State controlled, and as such is largely unaccountable to the users. It also tends to attract the least skilled personnel(other wise they'd be making more in the private sector...) ^^ That's a NASTY combination that one finds in many of the People's Utopias over there. Its slowly changing, but they are having to fight generations worth of damage done by governments of the past.

If you look at one of the recent maps of the connections, there are several single(or at most dual) connection bottlenecks.

Look at Hungry on a map, and then examine the countries that border it on all sides.  Thats exactly what I was speaking of.

While it is this quite amusing, there is not truth to this. The biggest ISPs in Hungary are the likes of T-Com, GTS Group or UPC. Obvioulsly all are huge multinational companies. There is also no difference in technology and infrastructure. We do not have a state owned telecom company.

Re: WHAT IS CAUSING THE DISCONNECTS?

My ping times to perpetuum-online.com are 179ms from work.
Ping times to fra03.cogent, which is the landing in germany, are 161ms.

So from Germany to the server is only 18ms, which means the network between Germany and Budapest isn't where the latency problem lies.

From me to the last US router, ping times are 63ms, which means its 116ms of delay from the shores of the US until the packet is next routed in FRA03.

Here is the Cogent Looking Glass Trace from FRA to Perp

TRACE from Frankfurt to perpetuum-online.com
 
--------------------------------------------------------------------------------

1 vl6.mpd01.fra03.atlas.cogentco.com (130.117.254.33) 143.094 ms 0.913 ms
 2 te0-1-0-7.ccr21.fra03.atlas.cogentco.com (130.117.1.54) 0.513 ms te0-0-0-5.mpd22.fra03.atlas.cogentco.com (130.117.3.198) 0.635 ms
 3 te0-1-0-4.ccr22.muc01.atlas.cogentco.com (130.117.50.246) 6.130 ms te0-0-0-0.ccr21.muc01.atlas.cogentco.com (130.117.0.166) 20.389 ms
 4 te0-3-0-5.ccr21.bts01.atlas.cogentco.com (154.54.38.250) 20.394 ms te0-3-0-5.ccr22.bts01.atlas.cogentco.com (154.54.39.10) 20.464 ms
 5 te7-7.ccr01.bud01.atlas.cogentco.com (154.54.38.189) 24.588 ms te1-1.ccr01.bud01.atlas.cogentco.com (130.117.3.54) 217.075 ms
 6 te1-1.ccr01.bud03.atlas.cogentco.com (130.117.3.74) 146.664 ms 204.043 ms
 7 149.6.182.42 (149.6.182.42) 20.469 ms 20.478 ms
 8 * *
 9 * *
 10 81.183.2.225 (81.183.2.225) 20.890 ms 21.582 ms
 11 84.2.225.113 (84.2.225.113) 20.765 ms 20.724 ms
 12 * *
 13 * *
 DONE

Here is from London to Me

PING from London to 38.20.57.57
 
--------------------------------------------------------------------------------

PING 38.20.57.57: 64 data bytes
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=0. time=145. ms
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=1. time=145. ms
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=2. time=145. ms
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=3. time=146. ms
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=4. time=145. ms
 
----38.20.57.57 PING Statistics----
 5 packets transmitted, 5 packets received, 0% packet loss
 round-trip (ms) min/avg/max/stddev = 145./145./146./0.063
PING from Bucharest to 38.20.57.57
 
--------------------------------------------------------------------------------

PING 38.20.57.57: 64 data bytes
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=0. time=189. ms
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=1. time=189. ms
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=2. time=189. ms
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=3. time=189. ms
 72 bytes from gi1-37.3528.mpd03.lax01.atlas.cogentco.com (38.20.57.57): icmp_seq=4. time=189. ms
 
----38.20.57.57 PING Statistics----
 5 packets transmitted, 5 packets received, 0% packet loss
 round-trip (ms) min/avg/max/stddev = 189./189./189./0.083

17 (edited by Arga 2011-05-17 19:10:49)

Re: WHAT IS CAUSING THE DISCONNECTS?

True Calvin, but the single peering point for the telecoms 'may' be owned by the state, but it really isn't the issue for latency anyway, as the traces show, packet delay though that NAP is negligable.

Edit: But I agree about infrastructure, Cogent owns/leases fiber in EU and even from as far east as Bucharest, Romania, the signal is good and strong.

Re: WHAT IS CAUSING THE DISCONNECTS?

Another thought... if you set up a mirrored 'relay' in london, that could allow the US to be routed through something other than FRA (which my gut tells me is the issue) but keep the core server in Hungary.

Conceptually, the london server would just be a front end with a VPN back to Hungary where it access the middleware (actual perp server). We would actually get a little more latency, but it may eliminate the micro-disconnects.

Re: WHAT IS CAUSING THE DISCONNECTS?

After a little digging, your current provider is owned (or directly connected to) Deutsche Telekom AG, which helps explains why they peer through Bratislava to munich to frankurt to use the AC1 fiber, and why the ISP said they couldn't reroute.

Changing hosting/ISP's in hungary may acutally make a difference, just verify that the colocation ISP's peers with US telecoms with a EU prescene as well as EU telecoms,  or fraNET has hosting in Frankfurt, which would be less travel than to london.

http://www.net-lab.net/e-trolley/page_1188/index.html

Or Munich too http://www.inxs.de/carriers.shtml

Re: WHAT IS CAUSING THE DISCONNECTS?

DEV Calvin wrote:
Wraithbane wrote:

What makes you think they haven't? Keep in mind that much of that is still State controlled, and as such is largely unaccountable to the users. It also tends to attract the least skilled personnel(other wise they'd be making more in the private sector...) ^^ That's a NASTY combination that one finds in many of the People's Utopias over there. Its slowly changing, but they are having to fight generations worth of damage done by governments of the past.

If you look at one of the recent maps of the connections, there are several single(or at most dual) connection bottlenecks.

Look at Hungry on a map, and then examine the countries that border it on all sides.  Thats exactly what I was speaking of.

While it is this quite amusing, there is not truth to this. The biggest ISPs in Hungary are the likes of T-Com, GTS Group or UPC. Obvioulsly all are huge multinational companies. There is also no difference in technology and infrastructure. We do not have a state owned telecom company.

Ok... But can that also be said about the countries that surround Hungry?... I clearly stated that it is slowly changing. I'm making do with what information is available in english and on the internet. Given the regions past history, what else am I to do?  Cal, this was in no way directed at you or the other Dev's. You have to deal with the situation as it exists. My concern is with the technical issues involved in attempting to host a game of this type in Hungry.

If you can't kill it, don't make it mad.

Re: WHAT IS CAUSING THE DISCONNECTS?

Trace from home

  7    15 ms    18 ms    18 ms  ae-1-0.pr0.lax00.tbone.rr.com [66.109.6.129]
  8    17 ms    18 ms    18 ms  66.109.9.206
  9   194 ms   193 ms   197 ms  204.245.36.142
10   193 ms   193 ms   191 ms  81.183.0.78
11   192 ms   194 ms   195 ms  81.183.0.37
12   194 ms   194 ms   194 ms  81.183.2.225
13   194 ms   194 ms   190 ms  84.2.225.113
14   195 ms   194 ms   195 ms  perpetuum-online.com [195.228.152.150]

Trace complete.

Re: WHAT IS CAUSING THE DISCONNECTS?

Arga wrote:

Trace from home

  7    15 ms    18 ms    18 ms  ae-1-0.pr0.lax00.tbone.rr.com [66.109.6.129]
  8    17 ms    18 ms    18 ms  66.109.9.206
  9   194 ms   193 ms   197 ms  204.245.36.142
10   193 ms   193 ms   191 ms  81.183.0.78
11   192 ms   194 ms   195 ms  81.183.0.37
12   194 ms   194 ms   194 ms  81.183.2.225
13   194 ms   194 ms   190 ms  84.2.225.113
14   195 ms   194 ms   195 ms  perpetuum-online.com [195.228.152.150]

Trace complete.

Arga, have you tried something like this? It might be more useful than a typical troute.

http://www.dslreports.com/smokeping

If you can't kill it, don't make it mad.

Re: WHAT IS CAUSING THE DISCONNECTS?

Wraithbane wrote:

Ok... But can that also be said about the countries that surround Hungry?... I clearly stated that it is slowly changing. I'm making do with what information is available in english and on the internet. Given the regions past history, what else am I to do?  Cal, this was in no way directed at you or the other Dev's. You have to deal with the situation as it exists. My concern is with the technical issues involved in attempting to host a game of this type in Hungry.

Don't worry, I - or we - are not the least bit offended, otherwise we would find ourselves in the wrong business. smile

As for the surrounding countries, the same thing can be said about all of them. Capitalist pigdog corporations have installed the same capitalist pigdog hardware and connected it to their capitalist pigdog networks using state of the art information technology skills clearly stolen from our dear comrades.

24 (edited by Alexander 2011-05-18 12:31:36)

Re: WHAT IS CAUSING THE DISCONNECTS?

Not a huge issue but given that most jumps for me increase my latency by around 5ms per jump I get a spike always from:

27ms/avg  te0-3-0-1.mpd21.fra03.atlas.cogentco.com [130.117.2.42]
51ms/avg  te0-0-0-0.ccr21.muc01.atlas.cogentco.com [130.117.0.166]

Re: WHAT IS CAUSING THE DISCONNECTS?

Thanks Wraith, that's a nice tool.

I posted my trace here so that I would have it at work, since I have RoadRunner service it takes a completely different path than the Cogent connection at work, so I'm going to track those IP's down and see if they go through the fraNAP or not.