Hello TourPro land.
As many of you know we lost connectivity to the servers today, October 26th. As of 5:30pst services were restored for all customers and systems are operating as expected. We will be doing a review of the servers tonight and throughout the weekend.
Our goal, of course, is to maintain availability to your systems 24/7 and we take as many precautions as necessary to do so including extensive backups, server snapshots and other forms of replication and failover.
The outage happened around 11:30am PST and was initially believed to be a network routing issue, as was initially reported to some users. It is now believed that our backup system that takes snapshots of all client servers attempted to purge multiple copies (lots of data) of the snapshots caused the storage system hosting the systems to slow. They didn’t fully stop, however they crawled to a point where it seemed as such.
We have disabled this feature as well as a replication process to maintain performance of the platform for the rest of this week and weekend and will be making appropriate changes at the beginning of next week to prevent this action from happening in the future.
Thank you for your patience during this unplanned outage and please feel free to reach out to us at our help desk if you have any questions or issues.