Firebot Outage Report: 7/14 – 7/16/2009
The last few nights, between 1am and 3am 4am, firebot has been offline, because of an ongoing connectivity issue. At the current time, I have no eta for when the issues will be resolved. The situation is pretty straightforward though, the outages are due to construction along the highway where I live and firebot is hosted, its being expanded to multiple lanes resulting in the need for my ISP, TimeWarner cable / RoadRunner, to move their fiber optic lines to make way for the new road. Though this has been ongoing it would seem they’ve reached the point where the overnight customer-facing work is being done, as each night, like clockwork, the cable has been going out completely.
Other recent events affecting firebot’s stability and connectivity over the past few weeks which are not external that are:
- Migrating to a new server. After 6 years of service, the machine he previously ran on was replaced as another more powerful machine became available. Firebot now runs on an AMD Athlon XP 2400+ with 2gb of RAM, 320gb disk running Windows Server 2003 R2 Enterprise Edition.. (Which, while definitely not current, is more than adequate.) Previously it was an Intel Celeron 1.7ghz with 1.2gb of ram, and 120gb disk, which was not performing well anymore. This is also the reason why at times there were two instances of the bot running.
- New network hardware, the network has been upgraded to gigabit (and Wireless-N), requiring the replacement of both the switch his machine was on, as well as the router. This caused a nightly (twice daily actually) connection reset, due to a bug in the firmware of the router and how it handles static dhcp leases (it for some reason, issues a dhcpnack before reissuing the same address to the machine, resulting in a brief loss of connectivity.) I’ve worked around it for now. (For those curious, the router is a Netgear WNR3500.)
