Md Mominul Islam | Software and Data Enginnering | SQL Server, .NET, Power BI, Azure Blog

while(!(succeed=try()));

LinkedIn Portfolio Banner

Latest

Home Top Ad

Responsive Ads Here

Wednesday, September 10, 2025

SQL Server Out of Memory Errors: Prevention and Solutions

 

SQL Server Out of Memory Errors: Prevention and Solutions

Out of memory (OOM) errors in SQL Server can bring business operations to a halt, manifesting as error 701 ("There is insufficient system memory to run this query") or sudden query failures under heavy load. In environments like financial trading platforms or e-commerce databases, these errors can lead to transaction rollbacks, delayed reports, and lost revenue—imagine a stock exchange system crashing during market volatility or an online store failing during a holiday rush. SQL Server 2025 addresses these with enhanced features like tempdb space resource governance, which prevents runaway workloads from exhausting tempdb space, improved memory grant feedback with percentile calculations, and automatic memory leak detection via new feature switches. These build on Intelligent Query Processing (IQP) to dynamically adjust memory usage.

This blog post provides a step-by-step guide to preventing and resolving OOM errors through configuration, indexing, query tuning, and optimization. We'll include T-SQL examples, real-life business scenarios, pros and cons, and practical applications to help DBAs and developers maintain robust systems.

Understanding Out of Memory Errors

OOM errors occur when SQL Server can't allocate enough memory for operations like query execution, sorting, or hashing. SQL Server manages memory via the buffer pool (for data pages), procedure cache (for plans), and workspace memory (for queries). In 2025, features like percentile memory grant feedback persist adjustments in Query Store, reducing over-grants that lead to pressure. Common symptoms include high PAGEIOLATCH waits, spills to tempdb, or outright failures, often in memory-constrained VMs or shared servers.

Common Causes

  • Overcommitted Server Memory: Unlimited max server memory lets SQL consume all RAM, starving the OS.
  • Excessive Query Memory Grants: Large sorts or joins request too much memory.
  • Inefficient Indexes: Table scans load unnecessary data into memory.
  • TempDB Contention: Spills from under-granted queries exhaust tempdb.
  • Memory Leaks or Fragmentation: Rare, but 2025's AutoMemoryLeakDetection helps identify them.

Step-by-Step Prevention and Solutions

1. Configure Server Memory Settings

Properly limiting SQL Server's memory prevents it from monopolizing system resources.

  • Step 1: Assess System RAM: Use Task Manager or DMVs to check total RAM.
    sql
    SELECT 
        physical_memory_in_use_kb / 1024 AS MemoryUsedMB,
        locked_page_allocations_kb / 1024 AS LockedPagesMB
    FROM sys.dm_os_process_memory;
  • Step 2: Set Max Server Memory: Leave 10-20% for OS. For a 16GB system:
    sql
    EXEC sp_configure 'show advanced options', 1;
    RECONFIGURE;
    EXEC sp_configure 'max server memory (MB)', 12288;  -- 12GB
    RECONFIGURE;
  • Step 3: Enable Lock Pages in Memory: Grant the SQL service account this Windows policy to prevent paging.
  • Step 4: Monitor with 2025 Features: Use new switches like CalculateMemoryPressureSignal for proactive alerts.

Pros: Stabilizes the system, reduces OOM by 30-50%; easy to implement. Cons: Under-setting can cause internal pressure; requires testing under load. Real-Life and Business Usage: In a SaaS analytics firm, configuring max memory prevented OOM during client spikes, ensuring 99.9% uptime and retaining contracts worth millions. Businesses use this to scale VMs cost-effectively, avoiding over-provisioning.

2. Optimize Memory Grants with IQP

SQL Server 2025's memory grant feedback adjusts grants based on past executions to avoid spills or over-allocation.

  • Step 1: Enable Query Store and Feedback:
    sql
    ALTER DATABASE YourDatabase SET QUERY_STORE = ON;
    ALTER DATABASE YourDatabase SET COMPATIBILITY_LEVEL = 170;  -- 2025 level
  • Step 2: Identify Problem Queries:
    sql
    SELECT 
        qsq.query_id,
        qt.query_sql_text,
        rs.avg_granted_memory_kb,
        rs.avg_used_memory_kb,
        rs.avg_spill_kb  -- From 2025 enhancements
    FROM sys.query_store_query qsq
    INNER JOIN sys.query_store_query_text qt ON qsq.query_text_id = qt.query_text_id
    INNER JOIN sys.query_store_runtime_stats rs ON qsq.query_id = rs.query_id
    WHERE rs.avg_spill_kb > 0
    ORDER BY rs.avg_spill_kb DESC;
  • Step 3: Apply Feedback: Percentile mode (default in 2025) uses history for better grants. Force a plan if needed:
    sql
    EXEC sys.sp_query_store_force_plan @query_id = 123, @plan_id = 456;
  • Step 4: Disable if Over-Adjusting:
    sql
    ALTER DATABASE SCOPED CONFIGURATION SET MEMORY_GRANT_FEEDBACK_PERCENTILE_GRANT = OFF;

Pros: Reduces spills by 20-40%, persists across restarts; automatic in 2025. Cons: Initial executions may spill; requires Query Store space. Real-Life and Business Usage: A banking app used feedback to handle volatile queries during trading hours, cutting OOM incidents and ensuring compliance with millisecond SLAs, saving potential fines.

3. Indexing to Reduce Memory Needs

Proper indexes minimize data loaded into memory, cutting grants.

  • Step 1: Find Missing Indexes:
    sql
    SELECT 
        migs.avg_total_user_cost * migs.avg_user_impact * (migs.user_seeks + migs.user_scans) AS improvement_measure,
        mid.statement AS TableName,
        mid.equality_columns,
        mid.inequality_columns,
        mid.included_columns
    FROM sys.dm_db_missing_index_group_stats migs
    INNER JOIN sys.dm_db_missing_index_groups mig ON migs.group_handle = mig.index_group_handle
    INNER JOIN sys.dm_db_missing_index_details mid ON mig.index_handle = mid.index_handle
    ORDER BY improvement_measure DESC;
  • Step 2: Create Indexes: For a sales query SELECT * FROM Sales WHERE Date > '2025-01-01' ORDER BY Amount;:
    sql
    CREATE NONCLUSTERED INDEX IX_Sales_Date_Amount 
    ON Sales(Date, Amount) INCLUDE (CustomerID);
  • Step 3: Use Columnstore for Analytics: In 2025, ordered columnstore reduces memory for scans:
    sql
    CREATE CLUSTERED COLUMNSTORE INDEX CCI_Sales ON Sales WITH (ORDER (Date));
  • Step 4: Maintain Indexes: Rebuild fragmented ones:
    sql
    ALTER INDEX ALL ON Sales REBUILD;

Pros: Lowers memory usage by 30-60%; speeds queries. Cons: Increases storage and write overhead. Real-Life and Business Usage: In retail inventory systems, columnstore indexes prevented OOM in reporting, enabling real-time stock analysis and reducing overstock costs by 15%.

4. Query Tuning

Rewrite queries to use less memory.

  • Step 1: Analyze Plans: Use SSMS to spot high-memory operators like Hash Match.
  • Step 2: Rewrite for Efficiency: Replace cursors with set-based ops. Example:
    sql
    -- Inefficient:
    DECLARE @Cursor CURSOR; -- High memory
    -- Efficient:
    UPDATE Sales SET Status = 'Processed' WHERE ID IN (SELECT ID FROM TempTable);
  • Step 3: Use Hints: Limit memory:
    sql
    SELECT * FROM LargeTable OPTION (MAX_GRANT_PERCENT = 10);
  • Step 4: Batch Processing: For large ops:
    sql
    WHILE @@ROWCOUNT > 0
    BEGIN
        DELETE TOP (10000) FROM LargeTable WHERE Condition = 1;
    END

Pros: Reduces grants without config changes; immediate impact. Cons: Requires code reviews; over-hints can degrade performance. Real-Life and Business Usage: A healthcare database tuned ETL queries to avoid OOM, ensuring timely patient data syncing and compliance with HIPAA.

5. TempDB Optimization

TempDB spills cause OOM; 2025 adds governance.

  • Step 1: Configure Multiple Files: Equal to CPU cores.
    sql
    ALTER DATABASE tempdb ADD FILE (NAME = 'tempdev2', FILENAME = 'D:\TempDB\tempdev2.ndf', SIZE = 1GB);
  • Step 2: Enable tmpfs (Linux): For RAM-backed tempdb.
  • Step 3: Set Space Governance: Percent-based limits in 2025.
    sql
    -- Use Resource Governor for similar control
    CREATE RESOURCE POOL TempDBPool WITH (MAX_MEMORY_PERCENT = 20);
  • Step 4: Monitor:
    sql
    SELECT * FROM sys.dm_db_file_space_usage WHERE database_id = 2;

Pros: Prevents tempdb exhaustion; 2025 features cut spills. Cons: RAM-backed needs sufficient memory; governance adds overhead. Real-Life and Business Usage: In BI tools for a logistics firm, tempdb governance handled large sorts, enabling real-time tracking and cutting delivery delays.

6. Use Resource Governor and Monitoring

Cap resources per workload.

  • Step 1: Enable Governor:
    sql
    ALTER RESOURCE GOVERNOR RECONFIGURE;
  • Step 2: Create Pools/Groups:
    sql
    CREATE RESOURCE POOL ReportingPool WITH (MAX_MEMORY_PERCENT = 30);
    CREATE WORKLOAD GROUP ReportingGroup USING ReportingPool;
  • Step 3: Monitor DMVs:
    sql
    SELECT * FROM sys.dm_os_performance_counters WHERE object_name LIKE '%Memory%';
  • Step 4: Use 2025 Monitoring: Enable MemobjStats for detailed tracking.

Pros: Isolates workloads, prevents OOM cascades. Cons: Complex setup; misconfiguration limits performance. Real-Life and Business Usage: A telecom company used it to prioritize calls over reports, maintaining service during peaks and boosting customer satisfaction.

No comments:

Post a Comment

Thanks for your valuable comment...........
Md. Mominul Islam