You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-11 - 14:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Sep 11, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
111442239163,24sleep20-21swapper/207:06:122
113352234162,59sleep00-21swapper/007:08:390
110872223194,19sleep30-21swapper/307:05:293
112322205168,24sleep10-21swapper/107:07:201
3263221490,5sleep31154099cyclictest09:10:023
1958621450,3sleep11152899cyclictest11:05:171
22332420,4sleep232-21ksoftirqd/210:25:222
1152399323,13cyclictest1816-21munin-run08:00:010
1154099312,16cyclictest0-21swapper/310:20:213
1153399310,3cyclictest0-21swapper/207:55:232
1152399311,14cyclictest8061-21munin-run11:55:010
231352290,4sleep11152899cyclictest10:00:161
1152399293,13cyclictest6834-21munin-run09:25:010
30302280,4sleep230-21rcuc/209:15:142
176222280,3sleep01152399cyclictest11:00:230
1154099254,12cyclictest88950irq/132-enp2s0-10:40:213
1152899258,10cyclictest0-21swapper/112:06:151
1152899251,16cyclictest0-21swapper/110:15:151
1152399251,13cyclictest23456-21munin-run12:30:000
1154099242,9cyclictest15240-21users09:40:263
1152899249,13cyclictest9524-21kworker/u8:107:59:491
1152899243,5cyclictest16448-40dnf07:18:251
1152899241,6cyclictest21678-21expr12:25:111
1152899241,17cyclictest0-21swapper/110:22:571
1152899241,16cyclictest0-21swapper/110:28:381
1152899241,11cyclictest1064-21users11:35:261
176252231,18sleep217732-21load07:20:182
176252231,18sleep217732-21load07:20:182
1154099232,11cyclictest25262-21/usr/sbin/munin11:20:153
1154099231,7cyclictest579-21systemd-journal11:45:013
11540992316,4cyclictest24654-21users11:15:253
1153399235,3cyclictest13627-21grep10:50:262
1153399235,3cyclictest13627-21grep10:50:262
1152899237,9cyclictest0-21swapper/112:38:141
1152899234,12cyclictest0-21swapper/109:28:171
1152899232,15cyclictest0-21swapper/108:58:391
1152899232,15cyclictest0-21swapper/108:39:581
11528992317,3cyclictest12541-21irqstats07:10:181
1152899231,16cyclictest0-21swapper/109:43:241
1152899231,12cyclictest0-21swapper/111:50:211
1152899231,11cyclictest0-21swapper/109:11:551
1152899230,16cyclictest0-21swapper/111:13:161
1152399238,13cyclictest0-21swapper/010:20:020
11523992318,3cyclictest3920-21smart_sda09:15:230
1152399231,10cyclictest9361-21kworker/0:311:10:070
1154099226,5cyclictest0-21swapper/309:39:473
1154099226,5cyclictest0-21swapper/309:39:473
1154099225,3cyclictest87350irq/126-enp1s0-10:58:473
1154099221,11cyclictest3028-21grep08:00:183
1154099221,11cyclictest0-21swapper/312:00:163
1154099221,11cyclictest0-21swapper/312:00:153
1153399223,14cyclictest32358-21cut10:20:222
1153399221,11cyclictest0-21swapper/210:03:522
1153399221,10cyclictest0-21swapper/209:35:272
1153399221,10cyclictest0-21swapper/209:35:272
1153399220,5cyclictest17739-21kworker/u8:110:05:482
1152899225,14cyclictest3392-21if_enp1s009:15:161
1152899222,15cyclictest0-21swapper/111:44:511
11528992214,3cyclictest0-21swapper/111:00:251
11528992214,3cyclictest0-21swapper/110:48:031
11528992213,3cyclictest0-21swapper/109:08:481
1152899221,18cyclictest12896-21cat09:35:251
1152899221,18cyclictest12896-21cat09:35:251
1152899221,18cyclictest10097-21unixbench-2d08:15:241
1152899221,12cyclictest0-21swapper/110:30:491
1152899221,11cyclictest0-21swapper/111:22:561
1152899221,11cyclictest0-21swapper/109:21:291
1152899220,16cyclictest0-21swapper/110:12:301
267852210,3sleep19-21rcu_preempt07:40:221
1154099219,3cyclictest19017-21/usr/sbin/munin07:25:113
1154099216,4cyclictest0-21swapper/310:38:283
1154099215,2cyclictest87350irq/126-enp1s0-09:47:273
1154099212,5cyclictest8422-21awk08:15:013
1154099212,12cyclictest87350irq/126-enp1s0-11:35:243
1154099211,8cyclictest0-21swapper/310:27:083
1154099211,6cyclictest0-21swapper/311:59:483
1154099211,3cyclictest25579-21sed08:50:243
1154099211,11cyclictest20843-21head12:20:253
1153399215,4cyclictest9254-21/usr/sbin/munin09:30:192
1152899213,11cyclictest0-21swapper/109:55:561
1152899212,17cyclictest0-21swapper/110:56:471
1152899212,13cyclictest87550irq/128-enp1s0-10:35:191
11528992117,3cyclictest0-21swapper/108:13:281
11528992113,3cyclictest0-21swapper/109:47:121
1152899211,2cyclictest15963-21ntp_states12:10:261
11528992112,3cyclictest0-21swapper/111:15:441
1152899211,18cyclictest0-21swapper/108:40:141
1152899211,15cyclictest0-21swapper/112:00:411
1152899211,15cyclictest0-21swapper/112:00:411
1152899211,14cyclictest0-21swapper/112:31:261
1152899211,14cyclictest0-21swapper/112:24:001
1152899211,14cyclictest0-21swapper/109:03:351
1152899211,14cyclictest0-21swapper/108:46:111
1152899211,12cyclictest21-21rcuc/111:25:251
1152899211,11cyclictest0-21swapper/110:05:051
1152899211,10cyclictest0-21swapper/109:30:501
1152399216,3cyclictest23112-21kworker/0:212:33:440
1152399212,9cyclictest29384-21kworker/0:011:41:470
1152399211,8cyclictest1416-21kworker/0:311:46:010
93892201,1sleep00-21swapper/011:55:220
79682200,3sleep27973-21unixbench_multi08:10:252
6552201,3sleep00-21swapper/010:23:470
209192201,2sleep00-21swapper/012:20:510
1154099207,7cyclictest0-21swapper/310:13:073
1154099207,3cyclictest30792-21kworker/u8:110:30:233
1154099204,4cyclictest19540-21grep08:40:013
1154099203,9cyclictest39-21rcuc/309:15:133
1154099203,9cyclictest26216-21/usr/sbin/munin08:55:113
1154099203,7cyclictest0-21swapper/310:15:033
1154099202,4cyclictest22542-21tr10:00:123
1154099202,10cyclictest16465-21/usr/sbin/munin11:00:253
1154099201,6cyclictest0-21swapper/309:21:413
1154099201,13cyclictest0-21swapper/311:25:253
1154099201,11cyclictest22832-21timerandwakeup12:25:243
1154099201,10cyclictest21207-21df_inode11:10:123
1154099201,10cyclictest20620-21strings08:40:213
1154099201,10cyclictest16255-21cat12:10:293
1154099201,10cyclictest0-21swapper/307:24:223
1154099201,10cyclictest0-21swapper/307:24:213
11540992010,2cyclictest13737-21sshd10:51:183
11540992010,2cyclictest13737-21sshd10:51:183
1154099200,6cyclictest0-21swapper/310:45:413
1153399206,3cyclictest87650irq/129-enp1s0-09:00:202
1153399205,8cyclictest28726-21swap11:25:222
1153399205,4cyclictest25192-21ps08:50:212
1153399205,2cyclictest0-21swapper/210:36:312
1153399203,5cyclictest0-21swapper/209:58:542
1153399201,12cyclictest0-21swapper/211:33:052
1152899209,3cyclictest2317-21expr08:00:121
1152899208,3cyclictest4235-21diskstats11:45:131
1152899206,4cyclictest0-21swapper/110:51:251
1152899206,4cyclictest0-21swapper/110:51:251
1152899202,16cyclictest0-21swapper/107:54:061
1152899202,16cyclictest0-21swapper/107:39:111
1152899202,10cyclictest87550irq/128-enp1s0-11:34:231
11528992011,7cyclictest2428-21kworker/u8:208:53:521
1152899201,14cyclictest0-21swapper/112:16:421
1152899201,14cyclictest0-21swapper/110:40:341
1152899201,10cyclictest18878-21cat07:25:011
1152899201,10cyclictest18878-21cat07:25:011
1152899200,15cyclictest0-21swapper/109:50:471
1152399205,11cyclictest87350irq/126-enp1s0-07:30:190
1152399201,11cyclictest18051-21/usr/sbin/munin09:50:320
1152399201,11cyclictest16819-21/usr/sbin/munin07:20:200
1152399201,11cyclictest16819-21/usr/sbin/munin07:20:200
1152399200,9cyclictest19933-21kworker/0:009:59:090
237542190,4sleep339-21rcuc/307:35:153
1154099198,3cyclictest29999-21/usr/sbin/munin07:50:243
1154099197,6cyclictest0-21swapper/309:58:303
1154099194,8cyclictest41-21ksoftirqd/307:17:003
1154099194,4cyclictest30069-21fschecks_count11:30:213
1154099194,12cyclictest5233-21needreboot08:05:193
1154099191,9cyclictest1433-21strings09:10:193
1154099191,9cyclictest0-21swapper/308:45:183
11540991913,3cyclictest4527-21gltestperf11:45:143
1154099191,10cyclictest8042-21needreboot09:25:183
1154099190,7cyclictest0-21swapper/309:52:353
1153399195,3cyclictest1436-21needreboot09:10:182
1153399193,4cyclictest16864-21munin-run12:15:012
1153399192,7cyclictest17336-21cat08:35:012
1153399192,7cyclictest17336-21cat08:35:012
1153399191,8cyclictest0-21swapper/211:06:552
1153399191,7cyclictest14750-21grep07:15:102
1153399191,6cyclictest0-21swapper/210:42:542
1153399191,4cyclictest0-21swapper/212:26:112
11533991912,4cyclictest29651-21grep10:15:142
1152899192,3cyclictest0-21swapper/108:26:011
1152899192,16cyclictest0-21swapper/107:46:321
11528991915,3cyclictest0-21swapper/108:23:431
1152899191,17cyclictest0-21swapper/107:30:521
1152899191,16cyclictest0-21swapper/107:27:061
1152899191,10cyclictest0-21swapper/111:55:161
1152899190,14cyclictest0-21swapper/108:32:341
1152899190,14cyclictest0-21swapper/108:32:331
1152399197,3cyclictest808-21snmpd08:57:450
1152399193,7cyclictest15846-21/usr/sbin/munin09:45:170
1152399191,9cyclictest4834-21grep10:30:250
1152399191,9cyclictest31812-21kworker/0:008:09:040
11523991912,4cyclictest24778-21grep10:05:120
1152399191,10cyclictest808-21snmpd11:23:130
1152399191,10cyclictest27276-21sshd07:41:180
1152399191,10cyclictest14618-21/usr/sbin/munin07:15:140
71312181,2sleep00-21swapper/010:36:120
29532181,2sleep00-21swapper/010:28:570
291932181,3sleep00-21swapper/011:27:090
235772180,3sleep00-21swapper/008:47:240
183672180,3sleep20-21swapper/211:04:112
174292180,1sleep00-21swapper/008:35:120
1154099185,4cyclictest19842-21ps11:05:213
1154099184,12cyclictest87350irq/126-enp1s0-10:05:183
1154099181,5cyclictest0-21swapper/308:30:143
1154099181,5cyclictest0-21swapper/308:30:143
1154099181,4cyclictest12809-21munin-run08:25:013
1154099181,15cyclictest0-21swapper/307:10:263
1154099181,12cyclictest87350irq/126-enp1s0-07:30:183
1154099181,11cyclictest0-21swapper/311:52:003
1154099181,10cyclictest0-21swapper/312:30:213
1154099180,8cyclictest0-21swapper/312:36:003
1153399185,4cyclictest4799-21load11:45:172
1153399184,4cyclictest13828-21pmu-power12:05:212
1153399183,5cyclictest0-21swapper/211:59:122
1153399182,5cyclictest23797-21munin-run08:50:012
1153399181,8cyclictest0-21swapper/209:55:022
1153399181,7cyclictest1-21systemd11:21:172
1153399181,7cyclictest11413-21perl08:20:162
1153399181,7cyclictest0-21swapper/211:16:402
1153399181,7cyclictest0-21swapper/209:26:472
11533991813,3cyclictest20376-21if_enp2s008:40:192
1153399181,12cyclictest87350irq/126-enp1s0-09:48:002
1152899187,3cyclictest0-21swapper/108:05:231
1152399188,3cyclictest16561-21sshd12:11:180
1152399188,3cyclictest10481-21/usr/sbin/munin12:00:140
1152399188,3cyclictest10481-21/usr/sbin/munin12:00:140
1152399187,2cyclictest30089-21kworker/0:109:05:130
1152399184,4cyclictest20393-21kworker/u8:011:15:140
1152399182,9cyclictest25878-21/usr/sbin/munin12:35:120
1152399182,5cyclictest87350irq/126-enp1s0-09:30:180
1152399182,2cyclictest9361-21kworker/0:310:46:370
1152399181,6cyclictest0-21swapper/009:25:130
11523991813,3cyclictest23025-21grep10:00:150
1152399181,11cyclictest0-21swapper/009:12:520
1152399181,10cyclictest0-21swapper/009:38:020
1152399181,10cyclictest0-21swapper/009:38:020
11523991810,3cyclictest7-21ksoftirqd/007:25:340
290412170,2sleep339-21rcuc/307:45:223
193532170,2sleep20-21swapper/208:39:362
1154099174,3cyclictest26598-21needreboot07:40:193
1154099171,9cyclictest29371-21kworker/u8:212:05:113
1154099171,7cyclictest10536-21cat09:30:263
1154099171,5cyclictest9239-21head08:15:153
11540991712,3cyclictest28417-21/usr/sbin/munin09:00:133
11540991711,3cyclictest17349-21expr12:15:133
1154099171,10cyclictest0-21swapper/307:57:043
1153399178,2cyclictest87350irq/126-enp1s0-07:14:142
1153399177,2cyclictest87550irq/128-enp1s0-12:33:082
1153399174,3cyclictest21378-21tr07:30:132
1153399173,4cyclictest0-21swapper/209:40:532
1153399171,9cyclictest11147-21sed12:00:162
1153399171,9cyclictest11147-21sed12:00:162
1153399171,7cyclictest0-21swapper/210:46:572
1153399171,6cyclictest3284-21users11:40:262
1153399171,6cyclictest18079-21strings12:15:192
1153399171,6cyclictest15778-21tail10:55:262
1153399171,6cyclictest0-21swapper/209:20:122
1153399171,5cyclictest4437-21proc_pri10:30:232
1153399171,3cyclictest11511-21cyclictest08:00:222
1153399171,3cyclictest0-21swapper/211:10:122
11533991712,3cyclictest7678-21users11:50:262
11533991711,3cyclictest30971-21dnf09:05:142
11533991711,3cyclictest30102-21tr07:50:122
11533991711,3cyclictest27922-21ntp_states10:10:262
1152399177,3cyclictest808-21snmpd08:24:250
1152399177,2cyclictest808-21snmpd10:58:080
1152399176,3cyclictest17088-21/usr/sbin/munin12:15:230
1152399174,3cyclictest0-21swapper/010:51:540
1152399174,3cyclictest0-21swapper/010:51:540
1152399172,9cyclictest0-21swapper/007:45:210
1152399172,6cyclictest8635-21/usr/sbin/munin08:15:250
1152399172,12cyclictest28417-21/usr/sbin/munin09:00:130
1152399172,12cyclictest23410-21/usr/sbin/munin07:35:160
1152399171,6cyclictest0-21swapper/010:42:060
1152399171,6cyclictest0-21swapper/008:50:140
1152399171,4cyclictest12692-21/usr/sbin/munin12:05:190
11523991712,3cyclictest13648-21acpi09:40:120
11523991711,4cyclictest0-21swapper/008:00:120
11523991711,3cyclictest6772-21df08:10:120
11523991710,4cyclictest810-21sshd10:10:180
104082171,2sleep20-21swapper/208:17:412
11540991611,3cyclictest13034-21/usr/sbin/munin08:25:173
1153399169,4cyclictest28475-21sh07:45:152
1153399168,2cyclictest87350irq/126-enp1s0-08:05:112
1153399162,5cyclictest26793-21memory12:35:182
1153399161,6cyclictest0-21swapper/211:37:122
1153399161,3cyclictest0-21swapper/207:25:122
11533991612,3cyclictest0-21swapper/208:28:042
11533991611,3cyclictest26216-21/usr/sbin/munin08:55:172
11533991611,3cyclictest24331-21wc07:35:202
1153399160,5cyclictest26600-21needreboot07:40:192
1153399160,2cyclictest0-21swapper/212:20:132
1152399165,4cyclictest87350irq/126-enp1s0-11:05:240
1152399165,3cyclictest13034-21/usr/sbin/munin08:25:120
11523991612,3cyclictest0-21swapper/008:31:500
11523991612,3cyclictest0-21swapper/008:31:500
11523991611,3cyclictest31800-21seq07:53:000
11523991611,3cyclictest19703-21cat08:40:140
1152399161,12cyclictest29668-21/usr/sbin/munin11:30:210
11523991610,3cyclictest32187-21df11:35:140
1152399159,4cyclictest12197-21grep07:10:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional