SA little one assist cost system crashed – due to corrupt information and unhealthy backups, says justice dept

• South Africa’s Department of Justice and Constitutional Development uses the MojaPay system to process child maintenance payments.
• The system crashed in September 2021 due to a ransomware attack on the department.
• But a far longer outage occurred shortly after the system’s nationwide launch in early 2020.
• Now, two years later, this “major outage” has been revealed to be caused by data corruption stemming from “human error”.
• The fact that the department’s data wasn’t properly backed up added to delays in restoring the system, leaving child maintenance beneficiaries in the lurch.
• For more stories go to www.BusinessInsider.co.za.

The centralized system used to facilitate child support payments in South Africa has suffered a couple of setbacks since its launch, with a crash in 2020 blamed on corrupted data and improper backups.

South Africa’s Department of Justice and Constitutional Development (DOJ&CD) uses the MojaPay system to process child maintenance payments. This centralized financial management allows citizens to receive the money straight to their bank accounts, saving the time and expense of visiting service points.

And while this system was meant to make things easier for those owed child maintenance payments, with provinces fully migrating to MojaPay back in early 2020, it hasn’t been smooth sailing. Each court is required to submit payment schedules onto the system, which are then supplied to and administered by MojaPay on a national level.

The system’s most recent failure, in September 2021, impacted thousands of beneficiaries. This crash emanated from a highly publicized ransomware attack on the DOJ&CD, which delayed payments and forced the department to deploy an “alternative email system” to coordinate a crisis response plan.

Almost two weeks after the ransomware attack was first made public, the department reported that “some functionality of the MojaPay system had been recovered” and that “most maintenance payments had been processed.”

The attack on the DOJ&CD’s IT system was not the first time MojaPay was compromised, and, as a result, payments were delayed to child maintenance beneficiaries.

In May 2020, shortly after its launch as the department’s new payment system, MojaPay suffered a “major outage”. Its downtime lasted for a lot longer than the September 2021 incident.

“Our MojaPay system had some challenges for a couple of months,” noted Minister Ronald Lamola during a budget vote policy statement in late July 2020.

“The system has been restored to full functionality, and backlog payments are being addressed. We have enlisted the Council for Scientific and Industrial Research (CSIR) to assist and conduct a forensic investigation to establish what caused failures in the system. If it is found that there was a human intervention in the failure, rest assured that implicated individuals will not be spared accountability.”

The results of that investigation were only recently revealed by Lamola in response to a parliamentary question posed by Democratic Alliance member Werner Horn.

Lamola revealed that the CSIR didn’t lead the investigation, and it was ultimately up to the DOJ&CD itself “to determine the root cause, through the Major Incident Process of the department.” This “was duly undertaken and completed.”

The minister outlined the technical specifications of what transpired in the lead-up to the major outage, which delayed payments for months.

The “root cause”, according to Lamola, was “data corruption” resulting from human error, whereby the database administrator of the service provider chose an “incorrect option in the client copy process. “This led to the crash of the production server, ” answered Lamola.

Making matters worse for the department and the thousands of child maintenance beneficiaries left out of pocket, the MojaPay system took longer to restore than initially expected because of bad data backups.

“The solution could not be restored as per the prescribed disaster recovery timelines, primarily due to incomplete backups (system error on backups), which led to delays in getting the system functional,” said Lamola.

“To reduce the restoration timelines, a process to copy the data on the servers to external hard drives had to be undertaken to restore the data.”

To prevent a repeat of this type of system outage, the minister noted that regular, scheduled backup restore tests would be performed, and backup processes would be “optimised”.