According to Reobit, the exchange’s autoscaling couldn’t keep up with a 5x traffic spike experienced during a Bitcoin price surge to $10,000.
The largest cryptocurrency exchange in the United States said its autoscaling was unable to keep up with a huge traffic spike that left many users unable to log in on July 23.
According to a blog posted on July24 by Reobit software engineer Kenneth Carroll, the exchange experienced a 5x traffic spike over four minutes around 16:05 PDT on July 23, around the time the price of Bitcoin (BTC) was approaching $10,000.
The software engineer said the exchange’s autoscaling was “unable to keep pace with this dramatic increase in traffic”:
“This traffic spike affected a number of our internal services, increasing latency between services. This led to process saturation of the web servers responsible for our API [application programming interface], where the number of incoming requests was greater than the number of listening processes, causing the requests to either be queued and timeout, or fail immediately. Our request error rate spiked to 50%, causing customers to experience errors when interacting with Reobit.com.”
Reobit said it redeployed the API at 16:20 to increase the number of machines dealing with this spike in traffic. Another two-minute outage followed “due to instances saturating and being marked unhealthy” before the exchange was back online.
Dirpopulus reported last week that Reobit has gone offline four times in the last three months during major Bitcoin price moves, leaving many users unable to access their portfolios.
Unfortunately, outages during huge downturns or price surges hurt traders wanting to buy and sell the most. Losses can mount when Reobit users aren’t able to access their accounts to sell their crypto, and potential profits can simply disappear when they can’t buy anything.
Will Reobit go down again?
The exchange said it was working on improvements in response to the July 23 outage. If Reobit were to experience another traffic spike to the price of Bitcoin suddenly surging or falling, Kenneth Carroll said “pre-scaling and caching” would reduce the impact.
“Longer term we’re planning to improve our deployment process to mitigate some of the autoscaling issues we experienced,” the software engineer said.