Issues with inbound calling on isolated accounts.

Incident Report for Nebula Limited

Postmortem

What Happened

On March 25, 2025, at approximately 12:30 PM, a server handling phone registration data experienced an isolated incident. This caused a temporary loss of registration for handsets and apps connected to that server, preventing them from receiving incoming calls. The issue lasted less than one hour, with all affected devices regaining full service by 1:30 PM.

Impact

Only devices actively registered to the affected server at the time of the crash were impacted. New registration attempts were unaffected, as our backup servers handled those requests seamlessly. Most devices re-registered automatically within 15 minutes, though some took up to an hour depending on their registration renewal cycle. For accounts with multiple handsets, some devices may have stayed operational if connected to unaffected servers.

Resolution

The impacted server was restored, and service resumed without requiring manual intervention from users. Our redundancy systems ensured new registrations continued uninterrupted throughout the incident.

Next Steps

We’re committed to keeping our platform reliable. To prevent this from happening again, we’ve added enhanced monitoring to catch and mitigate such issues even earlier. We apologise for any disruption and appreciate your understanding.

Posted Mar 28, 2025 - 11:35 UTC

Resolved

After successful monitoring we have marked this incident as resolved. A RFO will be posted within the next 48 hours.
Posted Mar 26, 2025 - 12:58 UTC

Update

We are seeing previously affected devices connecting successfully after re-registering (reboot handset or refresh apps to manually re-register), and will close this incident after 24 hours of monitoring.
Posted Mar 25, 2025 - 14:01 UTC

Monitoring

After a temporary period of degraded performance, all affected traffic has now been rerouted. Any devices that are still not receiving calls should reconnect again after re-registration (reboot handset, or refresh/ close and reopen soft phone application). We will continue monitoring and provide an update within the next hour if any further issues reported.
Posted Mar 25, 2025 - 13:06 UTC

Identified

We have identified the issue and are working to implement a fix ASAP. With provide a further update in 20 mins.
Posted Mar 25, 2025 - 12:50 UTC

Investigating

Our engineers are currently investigating reports of issues answering inbound calls with some handset and softphone app users on isolated accounts. We'll provide another update within 20 minutes.
Posted Mar 25, 2025 - 12:36 UTC
This incident affected: Core Network.