Hello,
I've got one SAP system on Oracle - single Oracle instance without ASM. It has average (user calls average based on 2 months period) 17 000 user calls per second and daily around 2 million dialog steps + 1,5 million of RFC+UPD+BGD steps together.
We are following SAP rules regarding DB parameters and tuning expensive SQL statements (in top 20 selects sorted by DB time we don't have any
select with more than 5 blocks read by each row, I'm aware that it is not only one criteria, but it is to show that system is being tuned (not that all
are tuned but all of top resource consuming SQLs from cursor cache looks fine) from expensive SQLs perspective. I’m aware that very often expensive SQLs causing very often unnecessary load.
Our storage vendor did checks and claims that there are no hot spots on storage and it has a free resources to handle current load twice (IOPs). I would like to know where this configuration could have bottlenecks. Some times (especially when load on db is higher than usual) I see queue and wait times on dm* devices. Configuration looks in this way:
Linux SLES 11 SP1. LUNs configured from OS to storage:
- Oracle parameters align with SAP recommendations (as mentioned above)
- Filesystem ext3 with mount options: rw,noatime,nodiratime,barrier=0 (5 sapdata LUN/fs - 2TB each)
- LVM:
lvm2, Metadata Areas 1, Clustered yes, PE Size 4.00 MB - Linux multipathing is used with configuration:
features='1 queue_if_no_path' hwhandler='1 alua' wp=rw policy='round-robin 0' - Then lower:
FC network (2 cards in server, 2 FC switches, 2 Controllers in storage array, storage array 160+ spindle disks + 12 ssd for tiering - no significant waits visible from storage array statistics).
Please let me know if you have experience with similar load and Oracle on Linux architecture and what configuration you have or what you would improve for above (please give a hints basing upon your experience)
Thanks in advance,
Marek