BuyVM Las Vegas Network Outage - Router Replacement

As you may have noticed, there was another outage on our Las Vegas router early this morning lasting approximately two hours. While unfortunate and inexcusable, this has allowed us to pinpoint the exact cause of the recent networking problems with our Vegas deployment.

The prior filtering provider, CNServers, completely blocked all UDP on our filtered IPs. When we switched Vegas over to Staminus for filtering (who mitigates and cleans UDP, but doesn't block it), the sudden influx of excess UDP is causing severe CPU spikes and seizing up the router. When we first started seeing these spikes, we initially attributed it to 'growing pains' from the recent increase in orders and new services - it wasn't until we replaced Vyatta that we were able to see the actual cause.

At this time, we've had Staminus block all UDP in our Vegas deployment (this only affects Filtered IPs). This will keep the network stable until we are able to get new equipment in place that will be able to sufficiently 'tank' the extra workloads.

We apologize for all of the turbulence in the network lately - as soon as we have a replacement for the router picked out and a time/date set for the swap, we will send another notification out. Thank you for your patience while we work to get these issues resolved for good.

---
Aldryic C'boäs,
Frantech Staff