Space usage on the system has exceeded the critical storage threshold of 85%

  • 0
  • 1
  • Question
  • Updated 1 year ago
  • Answered
I recently added another dSource that caused some spacing issues. All of my dSources are inactive due to this last one being added. I've dropped the dSource and am wondering how to get the system running again?

I see in the message that "Validated sync has been disabled for the dSource. It will be automatically resumed once the system goes below 82%."
Photo of Keino Jason

Keino Jason

  • 82 Points 75 badge 2x thumb

Posted 1 year ago

  • 0
  • 1
Photo of Jaclyn Schoof

Jaclyn Schoof, Community Manager

  • 5,092 Points 5k badge 2x thumb
Official Response
Hi Keino,
As stated the system will resume validated sync etc once you are back to 82%, some of the quickest ways to alleviate storage issues can be found in this Knowledge Base article on support.delphix.com.

Here is an explanation of what the different storage level warnings are:
  • When 78% of the total storage quota is reached, a Warning fault is triggered. You can resolve this fault by deleting objects in the Delphix Engine, adding storage, or changing policies, as described in the topics Adding and Expanding Storage DevicesDeleting Objects to Increase Capacity, and Changing Snapshot Retention to Increase Capacity. Additionally, refreshing target databases will clear space the engine uses to track changes over time for each DB.

  • When 80% of the total storage quota is reached, performance may decrease as the ZFS file system begins to leverage an algorithm which prioritizes space-savings over performance for storing data.

  • When 85% of the total storage quota is reached, a Critical fault is triggered, and the Delphix Engine will enter Maintenance Mode. When this occurs:
    • All pending link, sync, refresh, and provisioning processes will be cancelled, and no new operations can be initiated
    • Scheduled replication processes will first check for capacity on the target engine and hold data (replication currently in progress to the engine will not be halted)
    • Policy operations such as SnapSync, snapshot, and refresh are suspended for all platforms
    • dSources stop pulling in new changes. LogSync is suspended for all Oracle and PostgreSQL dSources. Validated sync is disabled for SQL Server dSources.
    • No virtual database (VDB) snapshots can be taken
  • When 95% of the total storage quota is reached, a second Critical fault is triggered for SQL Server. All SQL Server VDBs stop in order to maintain data integrity.

To take the system out of Maintenance Mode, increase the storage capacity. You can do this by adding storage devices, deleting objects, or changing policy settings. When you have increased the storage capacity, the system will automatically exit Maintenance Mode.

  • When the system falls below 95% of the total storage quota, you can manually start SQL Server VDBs that had stopped
  • When the system falls below 90% of the total storage quota, SQL Server VDBs that had stopped will automatically start
  • When the system falls below 82% of the total storage quota:
    • New link, sync, refresh and provisioning operations are allowed
    • Policy operations such as SnapSync, Snapshot, and Refresh resume for all platforms
    • dSources start pulling in new changes from their corresponding data sources. LogSync is resumed for Oracle and PostgreSQL dSources. Validated sync is enabled for SQL Server dSources.