🖥️ Server Hardware Reference Guide
HP ProLiant BL Series · HP C7000 Blade Chassis · Dell PowerEdge · IBM MegaRAID · L3 Deep Technical Q&A
HP Smart Array, IBM MegaRAID & HP ILO
RAID controller management, Smart Array commands, IBM MegaRAID storcli/MegaCli, and HP iLO out-of-band management
HP HPSSACLI / HPACUCLI — Smart Array Commands
RAID 0 (Striping) — data split across all disks, best performance, NO redundancy. If one disk fails, ALL data lost. Use for: temp/scratch data, caches. RAID 1 (Mirroring) — exact copy on two disks. Can lose one disk. Reads are fast (both disks serve reads). Use for: OS drives, boot volumes. RAID 5 (Distributed Parity) — data + parity distributed across 3+ disks. Can lose 1 disk. Good balance of performance/redundancy. Use for: data volumes. RAID 6 — like RAID 5 but can lose 2 disks (double parity). For production critical storage.
- 1Check status:
hpssacli ctrl slot=0 pd all show status— look for "Predictive Failure" or "Failed" - 2Get drive location:
hpssacli ctrl slot=0 pd 2I:1:6 show detail— note Port, Box, Bay - 3Blink the LED:
hpssacli ctrl slot=0 pd 2I:1:6 modify led=on— physically identify the drive - 4Hot-swap the drive (if RAID allows)
- 5Turn off LED:
hpssacli ctrl slot=0 pd 2I:1:6 modify led=off - 6Rescan:
hpssacli rescan - 7Monitor rebuild:
hpssacli ctrl slot=0 ld all show status
IBM MegaRAID storcli & MegaCli Commands
Preserved cache is write-back cache stored in the controller's battery-backed memory when a virtual drive goes offline unexpectedly (e.g., power loss, disk failure). The controller preserves this cache to prevent data loss hoping the drives come back. If the drives are gone permanently (replaced), the controller keeps this orphaned cache and may block operations. You then run storcli64 /c0/v11 delete preservedcache to clear it. Only do this when you're certain the data isn't needed — it's a destructive operation.
HP ILO — Out-of-Band Management
ILO (Integrated Lights-Out) is HP's out-of-band management technology. It's a dedicated management processor with its own network port that allows admins to manage servers even when the OS is down, crashed, or the server is powered off. Features: remote console (KVM), remote power on/off/reset, hardware health monitoring, firmware updates, virtual media mounting, BIOS configuration. IBM equivalent is IMM (Integrated Management Module). Dell calls it iDRAC. Essential for remote data centers where physical access is impractical.
HP BladeSystem C7000 & BL Series
c7000 chassis components, Onboard Administrator, Virtual Connect, iLO deep dive, blade recovery, firmware updates
Describe the HP BladeSystem c7000 chassis architecture — all major hardware components.
- Onboard Administrator (OA) — dual redundant management modules (slots 1 & 2). Provides web UI, CLI, SNMP, REST API. Manages power, cooling, firmware, and inter-module communication.
- Blade Server Bays (1–16) — house BL460c, BL680c, BL2x220c. Half-height blades = 1 bay; full-height = 2 bays.
- Power Supply Bays (1–6) — up to six hot-plug PSUs. Minimum 4 for N+1 redundancy under full load.
- Fan Bays (1–10) — ten hot-plug redundant fans with zone-based cooling. Active Cool fans auto-adjust from thermal sensors.
- Interconnect Module Bays (1–8) — network (FlexFabric, VC) and SAN (FC, SAS) modules. Bays 1–2 = LOM connected; Bays 3–8 = mezzanine card connected.
- Insight Display — front LCD for chassis status, IP config, and UID control.
- Midplane — backplane connecting blades to interconnect modules and power distribution.
show enclosure info | show server list | poweroff server 3 | iLO reset 5 | show interconnect list | show powerWhat is the HP Onboard Administrator (OA)? What can and cannot be configured through it?
- Power on/off/reset individual blades, set power capping per bay
- Configure interconnect modules
- Update blade iLO firmware remotely
- Configure LDAP/RADIUS for OA authentication
- Configure enclosure SNMP traps
- Manage Virtual Connect from OA CLI
- Configure UID LEDs, download chassis-wide logs
- Configure rack topology
- OS-level management (that is iLO's job)
- Configure BIOS/boot order (done via iLO or iLO RESTful API)
- Access individual disk controllers (done via Smart Array in iLO)
- Manage VM workloads
Explain the Virtual Connect (VC) module in c7000. What problem does it solve and how do you manage it?
- Abstraction: VC presents virtual MAC addresses and WWNs to blades. Physical upstream connections are shared. When a blade is replaced, the same virtual MAC/WWN is reassigned — server identity preserved without switch reconfiguration.
- Uplink aggregation: VC modules aggregate blade traffic to a small number of uplinks to the core network.
- Server Profiles: define which networks and fabrics a blade can access. Profiles contain virtual MACs, WWNs, network assignments, and bandwidth policies.
- Management: HP OneView or VC Manager GUI, OA CLI (
vc managercommands), or REST API. - Common issue: Server profile conflict when re-seating blades — resolve by reassigning or cloning the profile.
What is HP iLO and how is it different from the OA? What can you do in iLO that you cannot do in OA?
| Capability | iLO | OA |
|---|---|---|
| Management scope | Individual server (OS + hardware level) | Entire chassis (power, cooling, interconnects) |
| OS independence | Yes — works when OS is down | Yes — chassis level |
| Virtual console (KVM) | ✅ Yes | ❌ No |
| BIOS configuration | ✅ Yes | ❌ No |
| RAID management | ✅ via Smart Array | ❌ No |
| Virtual media (ISO mount) | ✅ Yes | ❌ No |
| iLO versions | iLO 4 (Gen8/9), iLO 5 (Gen10), iLO 6 (Gen11) | N/A |
| API | Redfish (iLO 5+) | REST/SNMP |
Walk me through remotely recovering a blade server with a hung kernel (OS unresponsive).
- Access iLO web UI or SSH:
ssh Administrator@<iLO-IP>. If iLO unreachable, access via OA CLI →iLO connect <blade-number> - Check IML: verify iLO System Health for hardware faults (memory ECC errors, processor faults, storage alerts)
- Virtual Console: launch HTML5 iLO remote console — if hung kernel, you'll see the Linux panic output or frozen screen
- Attempt graceful shutdown first: iLO Power & Thermal → Momentary Press (sends ACPI power button signal). Wait 60 seconds.
- If no response: Force Power Off (hard power cycle). Document as unclean shutdown.
- After reboot: use virtual console to monitor POST and OS boot. If boot fails, mount ISO via iLO Virtual Media to boot from rescue image.
- Post-recovery: check IML for hardware alerts, check OS logs (
/var/log/messages,dmesg) after recovery.
How do you update iLO firmware, BIOS, and Smart Array firmware on HP BL blades in production?
| Method | Description | Requires Reboot? |
|---|---|---|
| HP SPP (Service Pack for ProLiant) | Bootable ISO containing all firmware/drivers. Mount via iLO Virtual Media, run interactive/automated update. Best for full stack updates. | Yes |
| HP iLO Amplifier / OneView | Centralized firmware management. Detects versions, applies baseline. Can update iLO firmware online (no reboot for iLO-only in iLO 4+). | iLO: No; BIOS: Yes |
| OA CLI push | FIRMWARE UPDATE <bay> — updates iLO firmware to all blades from OA without booting OS. | No (iLO only) |
| HP Smart Update Manager (SUM) | Run on Linux, deploys RPM packages for online firmware update. Updates BIOS, NIC, Smart Array online. | Yes (BIOS) |
| Satellite + firmware RPMs | HP provides firmware as RPMs via HPE Software Delivery Repository. Deploy via package management. | Yes |
Dell PowerEdge — iDRAC, PERC & Lifecycle Controller
iDRAC vs iLO comparison, racadm CLI commands, Lifecycle Controller, PERC RAID management
Compare HP iLO and Dell iDRAC — functional equivalents and key differences.
| Feature | HP iLO | Dell iDRAC |
|---|---|---|
| Management Controller | iLO (Integrated Lights-Out) | iDRAC (Integrated Dell Remote Access Controller) |
| Current Generation | iLO 6 (Gen11) | iDRAC9 (14G/15G), iDRAC10 (16G) |
| Chassis Management | OA (Onboard Administrator) | CMC (M-series), OME (rack) |
| API Standard | Redfish (iLO 5+), REST API | Redfish, WSMAN, REST API |
| Remote Console | IRC Java/HTML5 | Virtual Console Java/HTML5, iDRAC Direct |
| Firmware Management | HPE OneView, SPP, SUM | iDRAC, OMSA, Lifecycle Controller |
| Storage Management | HP SSA / SSACLI | OMSA / racadm / PERC / storcli |
| OS Deployment | iLO Virtual Media + PXE | iDRAC Virtual Media + Lifecycle Controller |
| Free License | iLO Standard (basic) | iDRAC Basic (limited), Enterprise (full) |
What is racadm and how do you use it for remote Dell server management? Give key commands.
What is the Dell Lifecycle Controller and how is it used in production?
- OS Deployment: deploy OS from ISO via virtual media or network share without a PXE server. Configures drivers automatically.
- Firmware Update: update all server firmware (BIOS, iDRAC, PERC, NIC, PSU) from a network share or attached media in one pass.
- Hardware Configuration: configure BIOS settings, RAID arrays, network adapter settings, and boot order without booting the OS.
- Part Replacement: when replacing a system board or PERC controller, LC restores previous configuration from backup — critical for blade swap scenarios.
- System Erase: securely wipe all data and configuration — used for decommissioning.
- SupportAssist: collect hardware diagnostics for Dell support.
racadm get BIOS.* to view and set BIOS settings.How do you manage a Dell PERC RAID controller? Health check, failed drive replacement, storage expansion.
Hardware Troubleshooting Scenarios
5 real-world scenarios — power loss, POST failure, VC network outage, iDRAC unreachable, temperature alerts
You receive a P1 alert that an HP c7000 blade enclosure has lost power. 14 of 16 blades are offline. Two blades show "insufficient power" in OA. The data center reports no power failure at the PDU level. You have OA CLI and iLO access.
- What is your first diagnostic step via OA CLI?
- How do you determine if it is a PSU failure or a power budget issue?
- What OA commands do you use to check power status and PSU health?
- If two PSUs have failed, what is the recovery procedure?
- How do you bring the 14 offline blades back online safely?
SSH to OA IP→show enclosure power(overall power status) →show powersupply all(lists all 6 PSU slots with status, input voltage, output wattage, health) →show server list(power state of all 16 bays)- c7000 requires minimum 4 PSUs for N+1 under full load — 2 failures may trip power budget protection.
- If PSUs failed: identify failed units (status = Failed or Unknown), dispatch data center to physically inspect/replace.
- Power budget:
show enclosure power— if total blade demand exceeds available supply, OA enforces power capping. Solution: reduce power cap on non-critical blades or bring PSUs back online. - Safe restart: power on blades in priority order —
poweron server <bay>— stagger by 30 seconds to avoid inrush current spike. - After recovery: check IML on each blade for hardware faults logged during unclean shutdown.
- Rebooting all 16 blades simultaneously — risks exceeding power budget and causing repeat failure
- Not checking PSU input voltage — may indicate upstream PDU issue the DC team missed
A critical application blade (BL460c Gen9) does not respond after a scheduled maintenance reboot. iLO shows the server is powered on but the virtual console shows a black screen. OA shows the blade as "Powered On — Health: Degraded".
- How do you access the server when the console shows a black screen?
- What is your step-by-step diagnostic process via iLO?
- What does "Degraded" health in OA typically indicate?
- How do you check for POST failure codes?
- What is your escalation path if iLO itself is unresponsive?
- Try iLO virtual console via both Java IWS and HTML5 — sometimes one works when the other does not. Also try iLO text console:
ssh to iLO → start /system1/oemhp_vsp1 - Check IML: iLO > Information > Integrated Management Log — look for critical alerts (DIMM fault, processor fault, POST error code).
- "Degraded" usually means: failed DIMM (ECC uncorrectable), failed CPU, overtemperature event, or failed storage controller.
- POST error codes display on the virtual console. On Gen9, check iLO Physical Activity Log for POST-phase errors. HP POST error codes are documented in the server maintenance guide.
- If black screen persists: perform iLO reset (does NOT reboot the server): OA CLI:
iLO reset <bay>. Wait 90 seconds. - Escalation: if hardware fault confirmed (e.g., DIMM failure), open HPE support case with IML export and plan blade swap maintenance window.
All 8 blades connected to VC module in interconnect bay 1 have lost network connectivity simultaneously. Blades in bay 2 are fine. Servers respond to iLO management but have no data network.
- How do you determine if this is a VC module failure, uplink failure, or server profile issue?
- What OA and VC CLI commands do you use for diagnosis?
- How do you check uplink status on the VC module?
- What is the recovery procedure if the VC module has failed?
- How do you verify server profile assignments during the outage?
- OA CLI:
show interconnect list— shows status of all 8 interconnect bays, health, and link state.show interconnect <bay1>— port statistics, link status, firmware version. - Connect to VC Manager:
vc manager login→show ports,show networks,show uplinksets— check if uplinks are down. - Check VC uplinks: if uplinks to core switch are down, blades lose connectivity but VC module itself may be healthy. Verify on core switch side — are VC trunk ports showing link?
- If VC module failed: hot-swap replacement is possible in c7000. VC module in bay 1 can be replaced while bay 2 continues. Server profiles are stored in OA, not the VC module — they restore on new module insertion.
- VC module firmware mismatch after replacement: new module may need firmware update to match bay 2 — do this before re-activating.
After applying OS patches on a Dell R740 server, iDRAC becomes unreachable on the network. The OS is running fine. The iDRAC was accessible before the patch. You have console access to the OS.
- What could cause iDRAC to lose network connectivity after an OS patch?
- How do you diagnose and recover iDRAC network settings from within the OS?
- What racadm commands do you use locally to reset iDRAC network configuration?
- How do you reset iDRAC to factory defaults if configuration is corrupted?
- What should you check to prevent this from recurring?
- Root causes: OS network config change affected shared NIC used by iDRAC; iDRAC firmware update in patch triggered reset of NIC config; patching disabled iDRAC Service Module (ISM) which manages shared NIC arbitration.
- From OS (local racadm):
racadm getniccfg— verify current iDRAC IP settings. Reset:racadm setniccfg -s <static-IP> <mask> <gateway> - If ISM was affected:
systemctl status dcismeng→ restart if stopped:systemctl restart dcismeng - If config corrupted:
racadm racreset— soft reset iDRAC (~2 minutes, does NOT power cycle server). Or:racadm factorydefaults— WARNING: resets all config including credentials. - Prevention: exclude iDRAC shared NIC driver packages from OS patching scope, or switch to dedicated iDRAC NIC port to eliminate shared NIC dependency.
- Attempting iDRAC factory reset without knowing the recovery credentials — locks you out permanently
You receive a critical alert: a Dell R640 in production reports inlet temperature at 35°C (threshold 30°C) and CPU temperature at 85°C. Two fans are reported as degraded in iDRAC. Server is still running.
- What is your immediate triage process?
- How do you check fan and temperature status via iDRAC and racadm?
- What Dell thermal management options can you use?
- At what point do you proactively shutdown the server to prevent hardware damage?
- What is the root cause investigation process?
racadm -r <IP> getsensorinfo | grep -i temp— get all temperature readings.racadm -r <IP> getsensorinfo | grep -i fan— check fan RPMs.- iDRAC GUI: Hardware > Fans — shows all fan RPMs and status. Identify the two degraded fans (bay location, current RPM vs expected).
- Dell thermal management: iDRAC > Configuration > Thermal — switch to Maximum Cooling mode (forces all fans to max RPM) as emergency measure while degraded fans are replaced.
- IPMI force max fan speed:
ipmitool -I lanplus -H <iDRAC-IP> -U root -P <pass> raw 0x30 0x30 0x01 0x00 - Shutdown threshold: if inlet temp exceeds 40°C or any CPU exceeds 95°C, proactive graceful shutdown is warranted. Do not wait for iDRAC auto-shutdown at critical threshold.
- Root cause: failed fans (bearing failure), blocked airflow (cable management, empty bay without filler panel), cooling system failure (CRACs/CRAHs), elevated DC ambient temperature.
Quick Reference — HP OA/iLO & Dell iDRAC/racadm Cheat Sheet
Complete command reference for HP OA CLI, iLO CLI, SSACLI, Dell racadm, and storcli
Quick Reference — HP OA/iLO & Dell iDRAC/racadm Command Cheat Sheet
| Task | HP OA / iLO Command |
|---|---|
| Show all blades status | show server list (OA CLI) |
| Power on blade in bay 3 | poweron server 3 (OA CLI) |
| Reset iLO on bay 5 | iLO reset 5 (OA CLI) |
| Show PSU status | show powersupply all (OA CLI) |
| Show enclosure power | show enclosure power (OA CLI) |
| Show interconnect modules | show interconnect list (OA CLI) |
| iLO SSH login | ssh Administrator@<iLO-IP> |
| Show IML via iLO CLI | show /system1/log1 (iLO CLI) |
| Start virtual console | start /system1/oemhp_vsp1 (iLO CLI) |
| Show RAID via SSACLI | ssacli ctrl slot=0 ld all show |
| Show physical disks | ssacli ctrl slot=0 pd all show status |
| Rebuild status | ssacli ctrl slot=0 ld 1 show detail | grep -i progress |
| Task | racadm / storcli Command |
|---|---|
| Get system info | racadm -r <IP> -u root -p <pass> getsysinfo |
| Get all sensor data | racadm -r <IP> -u root -p <pass> getsensorinfo |
| Graceful shutdown | racadm -r <IP> -u root -p <pass> serveraction graceshutdown |
| Hard reset | racadm -r <IP> -u root -p <pass> serveraction hardreset |
| Get hardware event log | racadm -r <IP> -u root -p <pass> getsel |
| Get iDRAC NIC config | racadm -r <IP> -u root -p <pass> getniccfg |
| Soft reset iDRAC | racadm -r <IP> -u root -p <pass> racreset |
| Show RAID virtual disks | storcli /c0 /vall show |
| Show all physical disks | storcli /c0 /eall/sall show |
| Check rebuild progress | storcli /c0/v0 show | grep Progress |
| Add hot spare | storcli /c0/e<enc>/s<slot> add hotsparedrive |
| Run consistency check | storcli /c0/v0 start cc |
Oracle SPARC Server Hardware — Complete Study Guide
T Series · M Series · S Series · V Series · SuperCluster — Architecture, Features, Administration & L3 Interview Guide
Oracle SPARC Server Portfolio — Overview & Series Comparison
Oracle SPARC servers are enterprise-grade systems built around the SPARC (Scalable Processor ARChitecture) processor family. Unlike x86, SPARC is RISC (Reduced Instruction Set Computing) architecture, designed for extreme reliability, RAS (Reliability, Availability, Serviceability), and mission-critical workloads. All SPARC servers run Oracle Solaris.
| Series | Processor | Market Segment | Key Strength | Typical Workloads |
|---|---|---|---|---|
| T Series (CMT) | SPARC T4 → T8 | Mid-range / Scale-out | Thread-dense — massive parallelism (256 threads in T8) | Web servers, Java apps, database frontends, cloud workloads |
| M Series (SMP) | SPARC M7 → M12 | High-end / Mission-critical | Extreme RAS, large SMP, huge RAM capacity (up to 32TB) | Core banking, ERP, Oracle DB, mission-critical OLTP |
| S Series (Superserver) | SPARC S7 | Entry-level / Department | Cost-effective, Software in Silicon, in-memory security | Dev/test, SMB workloads, departmental database |
| V Series (x86 + SPARC) | Varies | Virtual/Cloud | Engineered system integration, VM consolidation | Mixed Oracle workload consolidation |
| SuperCluster | T/M Series | Engineered System | Full-stack optimisation (HW+OS+DB+Middleware) | Oracle Exadata alternative — Oracle DB + middleware |
☀️ T Series — SPARC CMT (Chip Multithreading) Servers
T Series Architecture — CMT & Thread Density
CMT was designed to handle thousands of concurrent requests by keeping the CPU pipeline full with many threads, tolerating memory latency by switching threads during cache misses.
| Processor | Cores/Socket | Threads/Core | Max Threads/Socket | Max Clock Speed | L3 Cache |
|---|---|---|---|---|---|
| SPARC T4 | 8 | 8 | 64 | 3.0 GHz | 4 MB/core |
| SPARC T5 | 16 | 8 | 128 | 3.6 GHz | 8 MB/core |
| SPARC T7 | 32 | 8 | 256 | 4.13 GHz | 256 MB total |
| SPARC T8 | 32 | 8 | 256 | 4.5 GHz | 256 MB total |
T Series Server Models:
- T4-1 / T5-1 / T7-1 / T8-1 — Single socket, 1U/2U rack server. Entry-level CMT.
- T4-2 / T5-2 / T7-2 / T8-2 — Dual socket, 2U rack server. Most common deployment.
- T4-4 / T5-4 / T7-4 — 4-socket server. 512 threads in a T5-4.
- T4-1B / T5-1B / T7-1B — Blade server form factor for Oracle SPARC T4 Blade Server System.
- T5-8 / T7-4 — 8-socket/4-socket max scale CMT configurations.
T Series Key Features — Software in Silicon & RAS
Software in Silicon — hardware-accelerated capabilities built into the SPARC T7/T8 processor die:
| Silicon Feature | Description | Benefit vs x86 |
|---|---|---|
| Crypto acceleration (DAX) | AES-NI equivalent but broader: RSA, SHA, DES, 3DES, RC4, MD5 all in hardware | Zero CPU overhead for TLS, VPN, database encryption |
| Database in Silicon | SQL operators (scan, filter, decompression) accelerated in hardware | Oracle DB queries run 2-10x faster with no software change |
| Dedup/Compression | ZFS inline compression/deduplication in hardware | Doubles effective storage, zero CPU penalty |
| Silicon Secured Memory (SSM) | Hardware memory tagging — detects buffer overflows at nanosecond speed | Catches exploits like HeartBleed in hardware, not software |
| Palomar (Query Acceleration) | Offloads Oracle DB column store scanning to the processor | Direct Oracle DB performance improvement built into CPU |
RAS Features (Reliability, Availability, Serviceability):
- Chipkill ECC — corrects multi-bit memory errors. Goes beyond standard single-bit ECC correction.
- Hot-plug DIMMs (M Series) — replace memory modules without downtime.
- CPU/Core retirement — faulty cores are automatically retired and marked offline by the hypervisor. System continues operating.
- Memory mirroring — duplicate all memory writes to a mirror for immediate failover.
- Memory sparing — a DIMM row is held in reserve; activated automatically when a DIMM begins failing.
- Predictive self-healing (ASR) — Automatic System Recovery. Faulty components are detected before failure and automatically taken offline.
- FMA (Fault Management Architecture) — Solaris fault management framework. Every hardware fault generates a structured fault event (UUID) with automated diagnosis and suggested repair action.
T Series Administration — ILOM, OBP & Key Commands
⬛ M Series — Mission-Critical SMP Servers
M Series Architecture — SMP & Extreme RAS
The M Series is Oracle's flagship SMP (Symmetric Multiprocessing) server line — designed for absolute maximum reliability for mission-critical workloads like core banking, Oracle databases, and ERP. The M12 supports up to 32 TB of RAM and 384 cores.
| Model | Processor | Max Sockets | Max Cores | Max RAM | Max I/O Slots | Form Factor |
|---|---|---|---|---|---|---|
| SPARC M3000 | SPARC64 VII+ | 1 | 4 | 64 GB | — | Rack |
| SPARC M4000 | SPARC64 VII+ | 2 | 8 | 128 GB | 10 PCIe | Rack |
| SPARC M5000 | SPARC64 VII+ | 4 | 16 | 256 GB | 20 PCIe | Rack |
| SPARC M8000 | SPARC64 VII+ | 8 | 32 | 512 GB | 40 PCIe | Rack |
| SPARC M9000 | SPARC64 VII+ | 16/32 | 64/128 | 2 TB | 160 PCIe | Cabinet |
| SPARC M6-32 | SPARC M6 | 32 | 384 | 32 TB | — | Rack |
| SPARC M7-8 | SPARC M7 | 8 | 256 | 32 TB | — | Rack |
| SPARC M8-8 | SPARC M8 | 8 | 256 | 48 TB | — | Rack |
| SPARC M12-2S | SPARC M12 | 4 | 192 | 32 TB | — | Rack |
M Series RAS Features — Dynamic Domains & Memory Protection
M Series RAS features go far beyond T Series — designed for five-nines (99.999%) uptime:
- Dynamic Domains (DR — Dynamic Reconfiguration) — partition the server into independent logical domains, each with its own OS instance. Add or remove CPUs, memory, and I/O from a domain while it is running.
- Dynamic CPU/Memory Addition — hot-add CPU boards and memory boards to a running system. No downtime.
- CPU Locking — lock specific CPU cores to a domain to prevent NUMA migration.
- Mirror Memory Mode — entire memory is mirrored. A single DIMM failure causes zero data loss — the mirror takes over immediately.
- Memory Scrubbing — continuous background ECC scrubbing detects and corrects single-bit errors before they accumulate into multi-bit errors.
- XSCF (eXtended System Control Facility) — M Series management controller (equivalent to ILOM in T Series, but more powerful). Manages hardware partitioning, domains, environmental monitoring.
- Field Replaceable Units (FRUs) — virtually every component is a FRU: CPU boards, memory boards, I/O boards, fans, PSUs, backplanes — all hot-swappable.
- Predictive Failure Analysis (PFA) — monitors component health trends and predicts failures before they occur.
Dynamic Reconfiguration (DR) — CPU/Memory/IO Hot Operations
DR is one of the most interview-tested M Series features. It allows zero-downtime hardware changes on a live production system.
1. Verify board is not running critical processes:
cfgadm -al SB02. Check memory usage — ensure remaining memory can absorb workload:
prtconf | grep Memory3. Check CPU load — ensure remaining CPUs can handle workload:
mpstat 1 54. Alert application teams before CPU DR — some JVMs do not handle CPU removal gracefully.
🟦 S Series — SPARC S7 Superserver
SPARC S7-2 & S7-2L Architecture & Features
The SPARC S7 (introduced 2016) is Oracle's entry/mid-range SPARC server — designed to bring Software in Silicon to a more cost-effective platform.
| Specification | SPARC S7-2 | SPARC S7-2L (Large Memory) |
|---|---|---|
| Processor | SPARC S7 (8 cores/socket) | SPARC S7 (8 cores/socket) |
| Sockets | 1 or 2 | 1 or 2 |
| Max Cores | 16 | 16 |
| Threads/Core | 8 | 8 |
| Max Threads | 128 | 128 |
| Clock Speed | 4.27 GHz | 4.27 GHz |
| Max RAM | 512 GB | 1 TB |
| DIMM Slots | 16 | 32 (extended memory board) |
| PCIe Slots | 8 | 8 |
| Form Factor | 2U Rack | 2U Rack |
| Storage | 8 × 2.5" SAS/SATA/SSD | 8 × 2.5" SAS/SATA/SSD |
| Management | ILOM 3.2+ | ILOM 3.2+ |
Key SPARC S7 Features:
- Software in Silicon — full DAX (Data Analytics Accelerator), cryptographic acceleration, Silicon Secured Memory (SSM), and Query Acceleration from T7/M7 generation, available at entry-level price.
- In-Memory Security — Silicon Secured Memory (SSM) detects buffer overflows, heap sprays, and use-after-free attacks in hardware. Enabled with a single kernel parameter change.
- Oracle Solaris Zones — S7 is well-suited for zone-dense deployments due to its thread count. A 2-socket S7-2 with 128 threads can run 20-40 OS zones efficiently.
- Oracle VM for SPARC (LDoms) — S7 supports LDoms for hardware virtualisation. Create guest domains for OS isolation.
- PCIe 3.0 — high-bandwidth I/O for SAN connectivity, 10/25/40 GbE networking.
Silicon Secured Memory (SSM) — Hardware Security Feature
SSM is one of the most distinctive SPARC S7/T7/T8/M7/M8 features. It uses hardware memory tagging to detect memory corruption attacks in the processor itself, before software-level security can even respond.
| Attack Type | Without SSM | With SSM |
|---|---|---|
| Buffer overflow | Exploitable — writes beyond buffer boundary undetected | Hardware detects the overflow at the instruction boundary, raises fault |
| HeartBleed-style | Application reads beyond allocated memory — leaks data | Reads outside tagged region generate hardware exception immediately |
| Use-after-free | Freed memory still accessible — dangling pointer | Freed region's tag invalidated — subsequent access triggers hardware fault |
| Return-oriented programming (ROP) | Stack spray undetected | Stack region tagged — unexpected execution path triggers detection |
| Heap spray | Heap corruption undetected | Heap allocations tagged — cross-region access detected in hardware |
🟩 V Series — Oracle SPARC V & Virtual Platforms
Oracle SPARC V Series — Engineered Virtualisation Platforms
The V Series covers Oracle's engineered virtualisation and hybrid platforms that combine SPARC hardware with Oracle software stack integration.
| Platform | Type | Key Characteristic |
|---|---|---|
| SPARC T4-4 (V-scale) | 4-socket T Series | Virtual-dense — 512 hardware threads for zone/LDom consolidation |
| Oracle VM Server for SPARC (LDoms) | Hypervisor (Type-1) | Logical Domains — hardware partitioning on SPARC CMT servers |
| Oracle Solaris Zones | OS Virtualisation | Lightweight namespace containers within a single Solaris instance |
| Oracle SuperCluster T8-8 | Engineered System | SPARC T8 + ZS7-2 storage + InfiniBand — Oracle DB+middleware optimised |
| Oracle SuperCluster M8 | Engineered System | SPARC M8-8 + ZS7-2 + InfiniBand — mission-critical Oracle stack |
Oracle VM Server for SPARC (LDoms) — Architecture:
- Control Domain — the primary domain that hosts the LDom Manager software. Manages all other domains. Always domain 0.
- Service Domain — provides virtual I/O services (virtual disk, virtual network) to guest domains. Often combined with the control domain.
- I/O Domain — a domain that has direct (PCIe passthrough) access to a physical I/O device. Can share that device as a virtual device to other domains.
- Guest Domain — a virtual machine. Consumes virtual CPUs (vCPUs), virtual memory, and virtual I/O. Can run a full independent Solaris instance.
- vCPU allocation — LDom vCPUs are allocated as SPARC hardware thread strands. A T8-1 (256 threads) can run 8 guest domains × 32 vCPUs each.
🔬 SPARC Processor Generations — Technical Deep Dive
SPARC Processor Generations — Feature Evolution
| Generation | Used In | Key Innovation | Notable Feature |
|---|---|---|---|
| SPARC64 VI | M3000–M4000 | Chip-multithreading on M Series | First M-class CMT |
| SPARC64 VII | M4000–M9000 | 4-core, 2-thread HT | Double floating point throughput |
| SPARC64 VII+ | M3000–M9000 refresh | Higher clock speed | ECC improvements |
| SPARC T3 | T3-x Series | 16-core CMT, 128 threads/socket | On-chip 10GbE |
| SPARC T4 | T4-x, T4-1B | 8-core, 8-thread, superscalar | Out-of-order execution + CMT (hybrid) |
| SPARC T5 | T5-2, T5-4, T5-8 | 16-core, 128 threads/socket | 2× T4 in one package |
| SPARC M5 | M5-32 | 32-core SMP, 512 threads | First Software in Silicon preview |
| SPARC M6 | M6-32 | 12-core, 96 threads/chip | 32 sockets = 384 cores |
| SPARC M7/T7 | M7-8, T7-2, T7-4 | 32-core, 256 threads/socket | Full Software in Silicon (DAX, SSM, Crypto) |
| SPARC M8/T8 | M8-8, T8-2, T8-4 | 32-core, 256 threads, 4.5GHz | 30% faster DAX v2, enhanced SSM |
| SPARC S7 | S7-2, S7-2L | 8-core, 64 threads/socket | Software in Silicon at entry-level |
| SPARC M12 (Fujitsu SPARC XII) | M12-2S | 12-core, 4.25GHz | Designed for in-memory DB (SAP HANA) |
SPARC NUMA Architecture & Memory Topology
All modern multi-socket SPARC servers are NUMA (Non-Uniform Memory Access). Understanding NUMA topology is critical for performance tuning.
- NUMA node — each socket (processor) has its own local memory bank. Memory attached to Socket 0 is "local" to Socket 0 and "remote" to Socket 1.
- NUMA latency — local memory access: ~50ns. Remote memory access: 100-200ns. Binding Oracle DB processes to local NUMA nodes reduces latency significantly.
- On-chip interconnect — SPARC T7/T8 uses on-chip coherent interconnect to link sockets. No separate northbridge chip as in older x86 designs.
- Coherent Interconnect — cache coherence is maintained across all sockets by the hardware. All sockets see a single unified memory address space.
🔧 Common SPARC Administration Tasks
SPARC Boot Management — OBP, Solaris Boot & Troubleshooting
svc:/network/ssh:default. This is a key differentiator from Linux.Solaris FMA — Fault Management Architecture
FMA is Solaris's automated hardware and software fault management framework. It is one of the most L3-tested SPARC topics because it integrates hardware and software fault diagnosis.
- DE (Diagnosis Engine) — analyses fault telemetry and produces a structured fault event with UUID, diagnosis, and suggested repair actions.
- FRU (Field Replaceable Unit) — the specific hardware component implicated in a fault. FMA identifies the exact FRU to replace.
- FMRI — Fault Management Resource Identifier — uniquely identifies a resource (service, device, or hardware component) in FMA.
- Suspect list — FMA provides a ranked list of components that may have caused a fault, with percentage certainty.
- Retire — FMA retires faulty CPU cores, memory pages, or other components to continue system operation while repairs are scheduled.
fmdump -ev | tail -100 to check for FMA fault events before the reboot. Then check /var/adm/messages, then ILOM/XSCF event logs. FMA should have generated a UUID for any hardware fault that caused the reboot.ZFS on SPARC — Hardware Accelerated Features
ZFS on SPARC servers benefits directly from Software in Silicon — compression and deduplication are hardware-accelerated, not software. This makes ZFS practical where it would impose too much CPU overhead on x86.
🎯 SPARC Interview Q&A — L3 Level Questions
Architecture & Comparison Questions
M Series (SMP) is designed for mission-critical reliability and very large workloads — up to 32TB RAM, Dynamic Reconfiguration, memory mirroring, and XSCF hardware management. Ideal for core banking systems, Oracle Database instances requiring 10TB+ memory, ERP systems that cannot afford downtime. Price is significantly higher.
Recommendation framework: If the workload is multi-threaded and I can afford some single-thread latency → T Series. If the workload requires maximum RAS, huge memory, and I need hardware-level DR → M Series. If budget is limited but I need Software in Silicon → S Series.
On x86, AES-NI provides hardware encryption, but ZFS compression is purely software, and there is no equivalent to SSM or DAX. The gap is most visible in Oracle Database workloads where DAX accelerates SQL column store scanning by offloading filter/scan operations to the processor itself.
XSCF (eXtended System Control Facility): Used on M Series (M3000 through M9000 with SPARC64 processors). It is more powerful than ILOM because it manages hardware domains, dynamic reconfiguration, and FRU management at the chassis level. Commands like
showboards, addboard, showdomainstatus are XSCF-only.M7-8 and M8-8 use a newer management architecture combining ILOM-style interfaces with domain management — interviewers may probe on this distinction.
Zones are OS-level virtualisation — all zones share the same Solaris kernel. They provide namespace isolation (process space, network stack, filesystem view) but not kernel isolation. More lightweight (200+ zones on a T8-2 are feasible). Zones can access host resources dynamically (capped CPU, fair share scheduling).
Use case guidance: LDoms when you need kernel isolation, different Solaris versions per domain, or when a domain needs direct PCIe passthrough. Zones when you need lightweight multi-tenancy, application isolation, and very fast provisioning (a zone creates in seconds).
Operational & Troubleshooting Questions
ssh root@<ILOM-IP> → show /SP/logs/event/list. Look for hardware alerts (temperature, power, memory) logged before the reboot.Step 2 — FMA events:
fmdump -ev | tail -200 on Solaris. Look for fault UUIDs generated before the reboot. fmadm faulty to see if any component is currently flagged.Step 3 — Solaris system log:
tail -500 /var/adm/messages. Look for panic messages, kernel errors, or driver faults immediately before the timestamp of the reboot.Step 4 — Crash dump: Check if a crash dump was generated:
ls -la /var/crash/$(uname -n)/. If yes: mdb -k /var/crash/$(uname -n)/unix.0 /var/crash/$(uname -n)/vmcore.0 → ::stack to see the panic stack trace.Step 5 — Power/Thermal: If ILOM shows temperature or power alerts, investigate the physical environment (DC cooling, PSU failure).
Step 6 — SMF services:
svcs -x after reboot to see if any services failed to start and may indicate a software cause.Step 2 — Verify Solaris detects the disk:
cfgadm -al to see attachment points, or format to list disks, or devfsadm to trigger device rescan → format.Step 3 — Identify new disk path:
ls /dev/dsk/ | sort to find new device node (e.g., c1t5d0).Step 4 — ZFS pool expansion:
• To add a new mirror vdev to a raidz/mirror pool:
zpool add data mirror c1t5d0 c1t6d0• To replace a failed disk in an existing mirror:
zpool replace data c1t0d0 c1t5d0• To expand pool by adding a spare:
zpool add data spare c1t5d0Step 5 — Monitor:
zpool status data — watch resilvering progress for replace operations.Step 6 — After resilvering:
zpool status data should show ONLINE for all vdevs.The process: (1) The SPARC core generates a hardware error trap. (2) FMA's cpumem-diagnosis engine analyses the error, determines severity, and generates a fault UUID. (3) If the core is unreliable, FMA calls
psradm -f <cpu-id> to take all hardware threads on that core offline. (4) The OS migrates all threads off the retired core transparently. (5) The system continues running with reduced CPU capacity. (6) fmadm faulty shows the retired core with fault UUID. (7) After hardware repair (typically replacing the CPU board/module), fmadm repaired <UUID> and psradm -n <cpu-id> bring it back online.This is fundamentally different from x86 — a faulty x86 core typically causes a system panic. SPARC continues running.
Step 1 — Download firmware: obtain the latest firmware bundle from My Oracle Support (MOS) or Oracle Software Delivery Cloud. Firmware is delivered as a .pkg file or .zip.
Step 2 — Upload to ILOM: ILOM web UI → Administration → Firmware Upgrade → upload .pkg file. Or via ILOM CLI:
→ load -source tftp://<server>/firmware.pkg /SP/firmwareStep 3 — ILOM-only update: select "Preserve existing configuration" and "Delay BIOS update" options. ILOM firmware updates immediately; host BIOS update is deferred to next reboot.
Step 4 — Verify:
→ show /SP/firmware_versionStep 5 — Schedule BIOS update: plan a maintenance window for the host reboot to apply deferred BIOS/OBP firmware.
Real-World Scenario Questions
fmadm faulty -s → check if fault is correctable (CE — Correctable Error) or uncorrectable (UE). CE can be tolerated temporarily; UE requires immediate action.Step 2 — Identify affected DIMM:
fmdump -V -u <UUID> → look for the FRU location code (e.g., /SYS/MB/MEM_GRP0/DIMM0).Step 3 — Check if page retirement is active: FMA may have already retired the faulty memory page and the database is running on remaining memory.
Step 4 — Plan DR (Dynamic Reconfiguration): M7-8 supports hot-memory operations. Check if the DIMM can be taken offline without rebooting:
cfgadm -al | grep mem → check if the memory board (not just DIMM) can be unconfigured.Step 5 — If CE and page retired: schedule memory board replacement during the next maintenance window. Monitor with
fmstat -m cpumem-diagnosis for increasing error rate.Step 6 — If UE or escalating: arrange emergency Oracle Support visit for hot DIMM replacement (M7-8 supports hot-add memory boards). Coordinate with DBA to migrate database instance to a RAC node during hardware service.
Step 7 — After replacement:
fmadm repaired <UUID> → verify with fmadm faulty (should show empty or resolved).ldm list-util on target. (2) Network connectivity between source and target ILOM. (3) Both servers running same or compatible LDom Manager version: ldm --version. (4) Verify live migration is supported (both servers must have same SPARC generation).Migration command:
ldm migrate-domain -n guestA ldoms@<target-ILOM-IP>The
-n flag means non-interactive (no OS suspend) — this is a live migration with the guest OS running.Migration process: LDom Manager iteratively copies memory pages from source to target while the guest keeps running. Remaining dirty pages are copied. At the final moment, the guest is suspended for a few milliseconds (typically <1 second), final state is transferred, and it resumes on the target. Guest OS and applications see no interruption (TCP connections remain established).
Monitor migration:
ldm list on both servers — the guest shows "migrating" state.Post-migration:
ldm list on target — guest shows "active". Verify application health. Remove guest definition from source: it is automatically removed on successful migration.SPARC Quick Reference — Key Commands Cheat Sheet
| Task | Command / Location |
|---|---|
| Access ILOM CLI | ssh root@<ILOM-IP> |
| Access XSCF CLI (M Series) | ssh platadm@<XSCF-IP> |
| Show system health (ILOM) | → show /System |
| Show faults (ILOM) | → show -level all /SP/faultmgmt/shell |
| Show hardware config (XSCF) | XSCF> showhardconf |
| Show domain status (XSCF) | XSCF> showdomainstatus -a |
| OBP boot verbose | ok> boot disk0 -v |
| OBP scan SCSI devices | ok> probe-scsi-all |
| OBP single user mode | ok> boot disk0 -s |
| Show CPU topology | psrinfo -pv |
| Show memory config | prtconf | grep Memory |
| Show full hardware | prtdiag -v |
| Show FRU serials | prtfru |
| Show active faults | fmadm faulty |
| Fault detail | fmdump -V -u <UUID> |
| FMA statistics | fmstat |
| Mark fault repaired | fmadm repaired <UUID> |
| Retire a CPU core | psradm -f <cpu-id> |
| Bring CPU back online | psradm -n <cpu-id> |
| DR — show boards | cfgadm -al |
| DR — unconfigure board | cfgadm -c unconfigure SB0 |
| DR — configure new board | cfgadm -c configure SB2 |
| Show NUMA topology | lgrpinfo -T |
| Show LDom list | ldm list |
| LDom live migrate | ldm migrate-domain -n guestA ldoms@<target> |
| LDom resource utilisation | ldm list-util |
| ZFS pool status | zpool status |
| ZFS pool scrub | zpool scrub <pool> |
| Solaris boot environments | beadm list |
| SMF service status | svcs -x |