So there is a new tab in the ingame profiler (open console and enter "openprofiler").
This shows the number and size of all the packets your client receives over time.
command=packet name
count=number of packets received so far
total=summed size of the received packets in bytes
avg=average size of the packets in bytes
When you experience lag, please check this and look for any unusually large numbers, and note the name of that packet. Sorting based on count or total is recommended.
What counts as unusual? Here are my statistics when I was shooting some NPCs on gamma (without noticing any lag): http://pbrd.co/1GvKoIL
Two things to note:
ZCMD 4 is robot movement (player or NPC), these are very small packets, but you get a lot of them.
ZCMD 46 is layer update (terrain, plants, blocks, etc.), over time this generates the largest traffic in size.
This is only testing a theory of players possibly getting hammered by unusual server packets, causing lag. If it's not the cause then we're still clueless, or it's not something with our servers.