Buckaroo rebooting

UH-Simon · 4th January 2008, 09:16 AM

We need to reboot the Buckaroo server for a filesystem check. Server will be offline for a few minutes while this completes. Thank you for yor patience.

UH-Simon · 4th January 2008, 11:12 AM

The filesystem check is repeatedly failing. We're checking the hardware over now.

UH-Simon · 4th January 2008, 12:08 PM

Unfortunately there has been a serious drive/file issue on the server. We are now checking the drives over and will have to use our server backups to restore the server. We will update this thread once this has been completed. We cannot give an ETA for this process as it is dependent on the amount of data involved.

UH-Simon · 4th January 2008, 01:45 PM

The restore of the server is still running. We apologise for any inconvenience this outage may be causing for customers on Buckaroo but please rest assured that this has our full an undivided attention and we hope to resolve this issue asap.

Thank you.

UH-Simon · 4th January 2008, 04:56 PM

Apologies for the lack of updates, we're still waiting on the restore to complete (it's proving to be very slow going) and will then fully check the box over to work out what has occurred. We are also speaking to our backup software partner to try and work out why this particular restore is taking so long to complete.

Thank you for your continued patience.

UH-Simon · 4th January 2008, 10:40 PM

Restore has just passed 90%. Once the restore has been completed and all sites confirmed back online we will be starting a detailed post-mortem on the server to find out exactly what occurred and why CDP took so long to restore. We will update you all as soon as we have a clear picture of what occurred.

UH-Matt · 5th January 2008, 01:09 AM

The server and service are now back online. We cannot apologise enough for the inconvenience this extreme downtime must have caused for those resellers on our buckaroo server. Once we have analysed all the data we will be sending you information on the issue/cause/resolution and also a credit to your account towards the unacceptable downtime.

Thank you,

UH-Simon · 12th January 2008, 10:11 PM

As part of the cleanup from last weekends outage the server is running a manual filesystem check to get any other bad data back in to line. We will update you once this is complete. Thanks.

UH-Simon · 12th January 2008, 10:40 PM

The filesystem completed but there are a couple of issues that our engineers wil need to discuss. We will keep you all update over the coming days with any further maintenance we may need to plan.

UH-Simon · 29th January 2008, 04:58 PM

Just an update. Buckaroo has had another small outage. The services will all be back online shortly and our discussions as to how tackle the remaining issues are ongoing.