Rainbow - Conversations messages impaired / Connection to Rainbow may fail – Incident details

All systems operational

Conversations messages impaired / Connection to Rainbow may fail

Resolved
Partial outage 0
Started 2 months agoLasted about 10 hours

Affected

Global (WW)

Operational from 6:57 AM to 7:14 AM, Major outage from 7:14 AM to 10:15 AM, Partial outage from 10:15 AM to 10:41 AM, Operational from 10:41 AM to 5:07 PM

[WW] Rainbow Hybrid PBX Telephony

Operational from 6:57 AM to 7:14 AM, Major outage from 7:14 AM to 10:15 AM, Partial outage from 10:15 AM to 10:41 AM, Operational from 10:41 AM to 5:07 PM

Europe & Middle East (EMEA)

Degraded performance from 6:57 AM to 7:14 AM, Operational from 6:57 AM to 10:41 AM, Major outage from 7:14 AM to 10:15 AM, Degraded performance from 10:15 AM to 10:41 AM, Partial outage from 10:41 AM to 1:19 PM, Operational from 10:41 AM to 1:19 PM, Partial outage from 1:19 PM to 5:07 PM, Operational from 1:19 PM to 5:07 PM

[EMEA] Rainbow Core Services

Degraded performance from 6:57 AM to 7:14 AM, Major outage from 7:14 AM to 10:15 AM, Degraded performance from 10:15 AM to 10:41 AM, Operational from 10:41 AM to 5:07 PM

[EMEA] Rainbow Hub Voice Services

Operational from 6:57 AM to 10:41 AM, Partial outage from 10:41 AM to 5:07 PM

Asia-Pacific (APAC)

Degraded performance from 6:57 AM to 10:41 AM, Operational from 10:41 AM to 5:07 PM

Updates
  • Resolved
    Resolved

    We implemented a fix on and are currently monitoring the result.

    Some users may have experienced hub telephony issues or slowness when handling conferences or hybrid telephony.

  • Update
    Update

    Dear partners, dear customers,

    We would like to inform you that an incident affecting our Cloud solution occurred this morning, due to a network issue at our IaaS provider, OVH Cloud.

    All collaborative services were restored as of 11:20 AM CEDT, and hybrid telephony services have been fully operational since 12:57 PM.

    At this stage, one last component is still being restored: the Hub "softphone" service.

    Our technical teams remain fully mobilized to restore the situation as quickly as possible.

    We will keep you informed of the evolution of this incident over the coming hours via: https://status.openrainbow.com/

    A follow-up communication including a detailed Root Cause Analysis based on information provided by OVHcloud will be shared once all services have been fully restored.

    We thank you for your understanding and remain at your disposal for any further information.

    Best regards,
    The Rainbow Team

  • Monitoring
    Monitoring

    Hybrid PX Telephony is under restoration. We are monitoring the infrastructure but you should see more services coming up.

    Softphony are still experiencing issues.

  • Update
    Update

    We are continuing to work on a fix for this incident. Collaboration and bubbles are operational and we are working on restoring telephony services

  • Identified
    Identified

    Load is decreasing more and more after restart of some components. More users may be able to login.

  • Update
    Update

    Incident is still under investigation. We have observed a huge load that is not lowering down and as such, we are still investigating step by step.

  • Update
    Update

    Apparently, we observe more than slowness but also errors during connection. Our team is investigating the issue.

  • Investigating
    Investigating

    We are currently investigating an incident impacting access to rainbow bubbles that may be slower than usual.

See how we figure out uptime