🎓 Lesson 11
D5
Historian Sizing & Compression Strategy
Historian sizing is choosing how much storage and processing power a SCADA system needs to reliably save and retrieve operational data over time.
🎯 Learning Objectives
- ✓ Calculate required storage capacity for a given tag count, sampling rate, and retention period
- ✓ Design an optimal compression strategy (e.g., deadband vs. time-based) based on process criticality and data fidelity requirements
- ✓ Analyze trade-offs between compression ratio, data fidelity, and historian query performance
- ✓ Apply ISA-18.2 and ISA-95 guidelines to justify historian configuration decisions
📖 Why This Matters
In mining and blasting operations, historians store critical data like blast hole pressures, detonation timing, vibration signatures, and environmental sensor readings. Under-sizing leads to data truncation during high-frequency events (e.g., seismic monitoring), risking non-compliance with MSHA or ISO 45001 audit trails. Over-sizing wastes capital and complicates maintenance. Getting historian sizing right is foundational—not just for alarms and reporting, but for forensic analysis after incidents, blast optimization modeling, and AI-driven predictive maintenance.
📘 Core Principles
Historian sizing rests on four interdependent pillars: (1) Data acquisition profile—number of tags, update rates (e.g., 1 Hz for ambient sensors vs. 1000 Hz for piezoelectric blast monitors), and value change frequency; (2) Compression methodology—lossless (e.g., delta encoding) vs. lossy (e.g., deadband, time-slice, or algorithmic compression like OPC UA’s Aggregate Functions); (3) Retention policy—driven by operational needs (e.g., 90 days for daily blast logs), regulatory mandates (e.g., MSHA Part 46 requires 1 year of training & incident records), and business analytics (e.g., 5-year trend analysis for pit wall stability); and (4) Query & redundancy requirements—concurrent user load, historical trending speed, and failover replication bandwidth. Compression strategy directly governs storage growth rate and reconstruction accuracy—especially for transient, high-amplitude events like blast waves.
📐 Storage Capacity Estimation
The fundamental storage estimation formula accounts for raw data volume, compression ratio, metadata overhead, and redundancy. It enables engineers to project disk space and inform hardware procurement and lifecycle planning.
Annual Raw Storage Requirement
S_raw = Σ(N_analog × R_analog × B_analog + N_digital × R_digital × B_digital) × T_sec/yearCalculates uncompressed annual storage volume in bytes, prior to compression or overhead.
Variables:
| Symbol | Name | Unit | Description |
|---|---|---|---|
| N_analog | Number of analog tags | count | Total analog measurement points (e.g., pressure, voltage, temperature) |
| R_analog | Analog sampling rate | Hz | Frequency of value acquisition per second |
| B_analog | Bytes per analog sample | bytes | Typically 2 for INT16, 4 for FLOAT32, 8 for DOUBLE |
| N_digital | Number of digital tags | count | Binary or status points (e.g., detonator armed, vent open) |
| R_digital | Digital sampling rate | Hz | Acquisition frequency for discrete signals |
| B_digital | Bytes per digital sample | bytes | Usually 0.125 (1 bit packed), but may be 1–4 bytes depending on protocol |
| T_sec/year | Seconds per year | s | 31,536,000 (non-leap year) |
Typical Ranges:
Medium-scale open-pit mine (1,000–2,500 tags): 30–120 GB/year raw
High-frequency blast monitoring (≥100 channels @ 1 kHz): 1.5–5 TB/year raw
💡 Worked Example
Problem: A surface mine deploys 1,200 I/O tags: 800 analog (16-bit), 400 digital (1-bit). Analog tags sample at 2 Hz; digital tags at 1 Hz. Target retention = 3 years. System uses 4:1 average compression ratio (including metadata and redundancy overhead).
1.
Step 1: Compute analog bytes/year = 1,200 tags × 2 samples/sec × 8,6400 sec/day × 365 days × 2 bytes/sample = ~60.5 GB/year
2.
Step 2: Compute digital bytes/year = 400 tags × 1 sample/sec × 86,400 × 365 × 0.125 bytes/sample ≈ 1.3 GB/year
3.
Step 3: Total raw = 61.8 GB/year → ×3 years = 185.4 GB raw. Apply 4:1 compression: 185.4 ÷ 4 = 46.4 GB. Add 20% for indexes, redundancy, and growth buffer: 46.4 × 1.2 = 55.7 GB.
4.
Step 4: Verify against typical range: For 1,200-tag mining SCADA, industry practice targets 40–70 GB/3 years post-compression — result falls within safe range.
Answer:
The result is 55.7 GB, which falls within the safe range of 40–70 GB.
🏗️ Real-World Application
At Newmont’s Boddington Mine (Western Australia), engineers sized their OSIsoft PI Historian for 12,500 tags—including 320 high-speed seismometers (2 kHz sampling during blasts) and 1,800 blast initiation circuit monitors. They applied adaptive deadband compression (±0.25% for pressure, ±1 ms for timing) combined with time-compression for quiescent periods. This reduced annual storage from 4.2 TB raw to 1.1 TB usable—enabling 7-year retention while maintaining sub-10ms timestamp fidelity for blast sequence forensics. The design passed MSHA audit validation and supported real-time fragmentation prediction models.