All systems operational

Show current status

Resolved
Rainbow connection slowness

Started
September 20, 2022 at 6:00:00 AM GMT+0
Status
Resolved after about 1 hour

Impact

Degraded performance
Affected
Europe & Middle East (EMEA)
[EMEA] Rainbow Conferencing
[EMEA] Rainbow Core Services
Germany (DE)
[DE] Rainbow Conferencing
[DE] Rainbow Core Services
Global (WW)
[WW] Rainbow Administration & Subscriptions
[WW] Rainbow Hub Voice Services
[WW] Rainbow Hybrid PBX Telephony
  • Resolved
    September 20, 2022 at 7:25:00 AM GMT+0

    Please find here after the root cause analysis.

    On 2022-09-19 evening Rainbow desktop version 2.109.0 has been released to all users. This version should not have been released as some blocking issues where still under correction. One of those issues created a very high load on Rainbow servers, leading for all EMEA users, too difficulties to connect to rainbow.

    The slowness was visible to users mainly in the morning between 8:00 and 9:20 CEDT.

    As soon as identified, the desktop client has been updated to previous version: 2.107.0.

    Please be sure we will review our staging procedure to avoid those mistakes. Also, a bottleneck has been identified in some internet facing servers. Its correction is already under test and should be able to support 4 times more load should an equivalent load occur again.

  • Investigating
    September 20, 2022 at 6:00:00 AM GMT+0

    We are currently investigating this incident.

See how we figure out uptime