You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-18 - 21:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Mon May 18, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1148724334,0sleep10-21swapper/106:54:381
1125823727,0sleep30-21swapper/306:51:593
1117723625,0sleep20-21swapper/206:51:022
1127523524,0sleep00-21swapper/006:52:120
11613991111,0cyclictest3514-21snmpd10:03:210
339960,0migration/39430-21sh10:13:403
339960,0migration/39283-21sh09:52:413
339960,0migration/36603-21latency11:30:343
339960,0migration/323678-21kernelversion08:40:343
29725060,0irq/27-eth0-tx-31774-21munin-node07:10:300
29725060,0irq/27-eth0-tx-21428-21munin-node08:15:380
29715060,0irq/26-eth0-rx-9676-21diskstats11:35:283
29715060,0irq/26-eth0-rx-21417-21kthreadcore11:40:343
29715060,0irq/26-eth0-rx-15592-21ssh10:55:363
149960,0migration/08868-21gltestperf08:05:300
149960,0migration/07682-21sh12:13:270
149960,0migration/031786-21switchtime11:05:370
149960,0migration/028595-21kthreadcore10:45:330
149960,0migration/01853-21sh09:49:330
116169960,0cyclictest0-21swapper/211:28:232
116139960,0cyclictest0-21swapper/010:51:360
8215050,0irq/24-0000:00:9008-21munin-plugin-st11:35:162
8215050,0irq/24-0000:00:16635-21cat12:00:112
8215050,0irq/24-0000:00:16250-21ssh09:55:392
339950,0migration/38640-21runrttasks11:55:133
339950,0migration/35808-21kthreadcore09:30:343
339950,0migration/328544-21kthreadcore07:55:353
339950,0migration/327216-21munin-node10:05:303
339950,0migration/325484-21unixbench_singl07:25:403
339950,0migration/320727-21cpuspeed_turbos07:50:293
339950,0migration/320407-21kthreadcore09:20:343
339950,0migration/319162-21sshd07:24:133
339950,0migration/319083-21fschecks_time10:00:303
339950,0migration/31901-21latency08:45:353
339950,0migration/318281-21aten_r1power_fr09:00:283
339950,0migration/314697-21munin-node08:30:383
339950,0migration/314078-21kthreadcore09:35:333
339950,0migration/312880-21kthreadcore10:35:323
339950,0migration/312737-21munin-node06:55:333
339950,0migration/310540-21latency_hist09:15:163
339950,0migration/310238-21munin-node11:55:323
29725050,0irq/27-eth0-tx-9096-21munin-node08:30:330
29725050,0irq/27-eth0-tx-8134-21munin-node09:10:382
29725050,0irq/27-eth0-tx-32503-21ssh11:06:542
29725050,0irq/27-eth0-tx-31824-21ssh11:46:492
29725050,0irq/27-eth0-tx-28322-21cut10:45:332
29725050,0irq/27-eth0-tx-27077-21perl10:25:302
29725050,0irq/27-eth0-tx-26150-21munin-node07:30:300
29725050,0irq/27-eth0-tx-18817-21munin-node09:20:302
29725050,0irq/27-eth0-tx-18073-21sed09:00:162
29725050,0irq/27-eth0-tx-12822-21munin-node07:20:360
29725050,0irq/27-eth0-tx-11502-21tr10:35:322
29725050,0irq/27-eth0-tx-0-21swapper/211:55:012
29725050,0irq/27-eth0-tx-0-21swapper/211:40:282
29725050,0irq/27-eth0-tx-0-21swapper/209:07:472
29725050,0irq/27-eth0-tx-0-21swapper/007:55:300
29715050,0irq/26-eth0-rx-7188-21kthreadcore08:25:343
29715050,0irq/26-eth0-rx-6559-21runrttasks12:05:213
29715050,0irq/26-eth0-rx-31520-21open_files11:05:353
29715050,0irq/26-eth0-rx-27393-21iostat_ios10:25:323
29715050,0irq/26-eth0-rx-0-21swapper/312:15:213
29715050,0irq/26-eth0-rx-0-21swapper/312:00:433
29715050,0irq/26-eth0-rx-0-21swapper/311:25:403
29715050,0irq/26-eth0-rx-0-21swapper/311:21:573
29715050,0irq/26-eth0-rx-0-21swapper/311:01:183
29715050,0irq/26-eth0-rx-0-21swapper/310:42:253
29715050,0irq/26-eth0-rx-0-21swapper/310:30:343
29715050,0irq/26-eth0-rx-0-21swapper/310:20:293
29715050,0irq/26-eth0-rx-0-21swapper/309:46:323
29715050,0irq/26-eth0-rx-0-21swapper/309:13:153
29715050,0irq/26-eth0-rx-0-21swapper/309:05:273
29715050,0irq/26-eth0-rx-0-21swapper/308:56:343
29715050,0irq/26-eth0-rx-0-21swapper/308:05:163
259950,0migration/29162-21kthreadcore07:40:332
259950,0migration/28338-21timerandwakeup09:50:402
259950,0migration/28153-21needreboot09:30:362
259950,0migration/220847-21kthreadcore11:00:332
259950,0migration/219034-21kthreadcore07:45:342
259950,0migration/214156-21kthreadcore09:35:332
259950,0migration/210489-21kthreadcore12:15:322
249950,0watchdog/24111-21gltestperf08:50:312
179950,0migration/17667-21munin-node10:30:371
179950,0migration/1330-21latency_hist12:10:161
179950,0migration/129653-21fschecks_time08:45:301
179950,0migration/127956-21idleruntime07:55:321
179950,0migration/124050-21sh10:00:361
179950,0migration/123843-21munin-node10:40:411
179950,0migration/118770-21sh09:20:191
179950,0migration/110638-21fschecks_time10:55:291
169950,0watchdog/14773-21missed_timers07:10:351
149950,0migration/09666-21gltestperf11:55:300
149950,0migration/08586-21kthreadcore07:40:330
149950,0migration/08494-21taskset10:31:250
149950,0migration/08019-21sh11:53:240
149950,0migration/06238-21kthreadcore08:50:340
149950,0migration/04293-21kthreadcore08:25:330
149950,0migration/03734-21kthreadcore09:10:320
149950,0migration/029166-21irqcore08:20:340
149950,0migration/026749-21kthreadcore07:05:340
149950,0migration/026512-21kthreadcore08:40:350
149950,0migration/02589-21fschecks_count10:10:290
149950,0migration/024768-21taskset09:01:440
149950,0migration/019999-21fschecks_count07:25:310
149950,0migration/018603-21kthreadcore08:10:340
149950,0migration/017148-21sh09:16:340
149950,0migration/017077-21kthreadcore08:35:300
149950,0migration/016316-21switchtime09:55:390
149950,0migration/016092-21unixbench-2d08:55:400
149950,0migration/01525-21taskset10:29:490
149950,0migration/014837-21sh11:35:350
149950,0migration/012600-21kthreadcore12:15:330
116169955,0cyclictest0-21swapper/211:22:232
116169950,0cyclictest0-21swapper/212:03:422
116169950,0cyclictest0-21swapper/211:15:182
116169950,0cyclictest0-21swapper/211:11:122
116169950,0cyclictest0-21swapper/210:58:442
116149955,0cyclictest0-21swapper/110:12:351
116149955,0cyclictest0-21swapper/109:32:421
116149950,0cyclictest0-21swapper/112:23:461
116149950,0cyclictest0-21swapper/111:41:491
116149950,0cyclictest0-21swapper/111:24:571
116149950,0cyclictest0-21swapper/110:37:301
116149950,0cyclictest0-21swapper/109:43:541
116149950,0cyclictest0-21swapper/109:29:141
116149950,0cyclictest0-21swapper/109:04:571
116149950,0cyclictest0-21swapper/108:59:071
116139955,0cyclictest0-21swapper/012:00:480
116139955,0cyclictest0-21swapper/011:33:410
116139955,0cyclictest0-21swapper/011:21:420
116139955,0cyclictest0-21swapper/011:10:310
116139955,0cyclictest0-21swapper/011:01:500
116139955,0cyclictest0-21swapper/010:15:200
116139955,0cyclictest0-21swapper/008:00:410
116139950,0cyclictest0-21swapper/009:09:160
339940,0migration/327867-21kthreadcore11:45:333
339940,0migration/325000-21kthreadcore08:15:353
339940,0migration/32324-21kthreadcore07:35:343
339940,0migration/320255-21taskset07:50:163
339940,0migration/312367-21taskset07:17:453
29725040,0irq/27-eth0-tx-2854-21gltestperf10:10:302
29725040,0irq/27-eth0-tx-26507-21munin-node10:05:312
29725040,0irq/27-eth0-tx-18481-21munin-node10:00:382
29725040,0irq/27-eth0-tx-18305-21ssh10:20:262
29725040,0irq/27-eth0-tx-18166-21ssh10:40:232
29725040,0irq/27-eth0-tx-0-21swapper/212:06:212
29725040,0irq/27-eth0-tx-0-21swapper/211:35:352
29725040,0irq/27-eth0-tx-0-21swapper/210:50:222
29725040,0irq/27-eth0-tx-0-21swapper/210:15:212
29725040,0irq/27-eth0-tx-0-21swapper/209:45:262
29725040,0irq/27-eth0-tx-0-21swapper/209:15:322
29715040,0irq/26-eth0-rx-16722-21ssh10:16:283
29715040,0irq/26-eth0-rx-15570-21smartctl08:10:303
29715040,0irq/26-eth0-rx-0-21swapper/312:21:263
29715040,0irq/26-eth0-rx-0-21swapper/312:10:313
29715040,0irq/26-eth0-rx-0-21swapper/311:18:143
29715040,0irq/26-eth0-rx-0-21swapper/311:12:003
29715040,0irq/26-eth0-rx-0-21swapper/310:50:313
29715040,0irq/26-eth0-rx-0-21swapper/310:45:203
29715040,0irq/26-eth0-rx-0-21swapper/309:57:533
29715040,0irq/26-eth0-rx-0-21swapper/309:40:233
29715040,0irq/26-eth0-rx-0-21swapper/309:27:473
29715040,0irq/26-eth0-rx-0-21swapper/308:00:273
29715040,0irq/26-eth0-rx-0-21swapper/307:40:333
259940,0migration/28396-21sh10:31:072
259940,0migration/231676-21kthreadcore07:55:362
259940,0migration/223995-21kthreadcore07:25:352
259940,0migration/21896-21latency08:45:352
259940,0migration/216995-21df_abs12:20:292
259940,0migration/21615-21kthreadcore07:10:342
259940,0migration/211396-21kthreadcore08:30:342
249940,0watchdog/226942-21ssh09:25:232
249940,0watchdog/21315-21hwlatdetect12:10:302
249940,0watchdog/20-21swapper/208:55:352
249940,0watchdog/20-21swapper/208:20:352
179940,0migration/17637-21kthreadcore07:15:331
179940,0migration/14041-21kthreadcore07:35:341
179940,0migration/127038-21cpuspeed_turbos09:45:291
179940,0migration/124839-21kthreadcore07:50:341
179940,0migration/124303-21timerwakeupswit07:00:401
179940,0migration/120927-21kthreadcore12:00:331
179940,0migration/116607-21taskset10:16:081
179940,0migration/111250-21kthreadcore12:15:331
16734240,0sleep318419-21cut08:35:313
169940,0watchdog/19823-21diskmemload11:36:261
169940,0watchdog/18504-21ssh11:33:501
169940,0watchdog/1428-21ssh11:49:581
169940,0watchdog/14112-21gltestperf08:50:311
169940,0watchdog/116216-21fschecks_count08:35:271
169940,0watchdog/1152962sleep111:56:141
169940,0watchdog/10-21swapper/109:50:351
169940,0watchdog/10-21swapper/106:55:351
149940,0migration/09025-21sh09:31:170
149940,0migration/0673-21latency_hist11:50:160
149940,0migration/028638-21kthreadcore12:05:360
149940,0migration/022959-21kthreadcore07:50:330
149940,0migration/020275-21kthreadcore10:40:380
149940,0migration/013604-21kthreadcore06:55:340
116169944,0cyclictest0-21swapper/208:10:412
116169943,0cyclictest3556-21lldpd07:00:312
116169943,0cyclictest3514-21snmpd07:52:482
116169943,0cyclictest3514-21snmpd07:16:182
116169943,0cyclictest3514-21snmpd07:05:182
116169943,0cyclictest3514-21snmpd06:55:182
116169943,0cyclictest27002-21awk07:30:312
116169943,0cyclictest2340-21awk08:00:312
116169943,0cyclictest23343-21awk08:40:332
116169943,0cyclictest22280-21awk08:15:322
116169943,0cyclictest16598-21awk08:35:282
116169942,0cyclictest19705-21awk09:40:312
116149944,0cyclictest0-21swapper/109:36:101
116149943,0cyclictest7395-21cat10:50:361
116149943,0cyclictest3556-21lldpd11:52:031
116149943,0cyclictest3556-21lldpd09:16:021
116149943,0cyclictest3514-21snmpd09:59:491
116149943,0cyclictest3514-21snmpd08:25:181
116149943,0cyclictest3514-21snmpd08:23:181
116149943,0cyclictest3514-21snmpd08:16:181
116149943,0cyclictest3514-21snmpd08:10:181
116149943,0cyclictest3514-21snmpd08:07:181
116149943,0cyclictest3514-21snmpd08:03:181
116149943,0cyclictest3514-21snmpd07:45:181
116149943,0cyclictest3514-21snmpd07:40:181
116149943,0cyclictest3514-21snmpd07:33:181
116149943,0cyclictest3241-21irqbalance11:14:451
116149943,0cyclictest3241-21irqbalance10:46:051
116149943,0cyclictest3048-21grep09:10:291
116149943,0cyclictest26868-21awk11:05:301
116149943,0cyclictest26107-21awk07:05:321
116149943,0cyclictest15505-21ls08:35:161
116139944,0cyclictest0-21swapper/009:41:170
116139944,0cyclictest0-21swapper/009:23:560
116139943,0cyclictest6954-21awk07:15:310
116139943,0cyclictest3556-21lldpd09:27:020
116139943,0cyclictest3514-21snmpd07:35:480
116139943,0cyclictest3241-21irqbalance11:41:240
116139943,0cyclictest3241-21irqbalance10:38:550
116139943,0cyclictest3241-21irqbalance07:45:350
116139943,0cyclictest29926-21awk08:45:320
116139943,0cyclictest26086-21sadc10:25:160
339930,0migration/35135-21kthreadcore08:50:343
29715030,0irq/26-eth0-rx-0-21swapper/308:20:383
29715030,0irq/26-eth0-rx-0-21swapper/307:32:163
29715030,0irq/26-eth0-rx-0-21swapper/307:08:393
29715030,0irq/26-eth0-rx-0-21swapper/307:00:313
259930,0migration/25376-21kthreadcore08:25:342
169930,0watchdog/10-21swapper/107:25:391
116169933,0cyclictest13660-21awk07:20:322
116169932,0cyclictest6113-21sensors07:35:392
116169930,0cyclictest8513-21diskstats08:05:302
116149932,0cyclictest463-21rm12:10:251
116149932,0cyclictest3514-21snmpd11:25:201
116149932,0cyclictest3241-21irqbalance10:21:151
116149932,0cyclictest31475-21kthreadcore10:25:341
116149932,0cyclictest29641-21kthreadcore10:05:341
116149932,0cyclictest26754-21grep09:05:271
116149932,0cyclictest23010-21fschecks_time08:40:311
116149932,0cyclictest21346-21kthreadcore11:00:331
116149932,0cyclictest13705-21awk07:20:321
116149932,0cyclictest13582-21kthreadcore11:15:331
116139933,0cyclictest19998-21kworker/0:110:06:180
116139933,0cyclictest19998-21kworker/0:109:50:260
116139933,0cyclictest19998-21kworker/0:109:35:460
116139933,0cyclictest10902-21kworker/0:210:56:300
116139932,0cyclictest26427-21awk11:25:290
116139932,0cyclictest18565-21aten2.4-expect07:00:290
116139932,0cyclictest17477-21awk12:20:320
116139932,0cyclictest10599-21awk11:15:320
339920,0migration/3314-21iostat_ios07:10:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional