You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-06 - 20:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Wed May 06, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450350,0irq/25-eth10-21swapper/306:53:543
247923423,0sleep00-21swapper/006:56:010
233522925,0sleep10-21swapper/106:54:221
221622824,0sleep20-21swapper/206:52:582
25899976,0cyclictest6125-21kthreadcore06:56:421
339960,0migration/36004-21sh12:17:193
339960,0migration/35758-21sendmail_mailtr11:56:453
339960,0migration/315386-21sh11:44:383
25909960,0cyclictest0-21swapper/211:16:482
25899965,0cyclictest17140-21irqcore12:06:401
149960,0migration/07653-21sh08:58:060
149960,0migration/06563-21sh12:18:500
149960,0migration/031967-21sh10:33:290
149960,0migration/029727-21diskmemload09:11:410
149960,0migration/017417-21sh09:25:550
149960,0migration/014933-21taskset12:24:490
149960,0migration/014432-21sh12:02:350
149960,0migration/012542-21kthreadcore10:41:410
8215050,0irq/24-0000:00:23574-21latency_hist07:36:252
4150,0ktimersoftd/00-21swapper/009:31:340
4150,0ktimersoftd/00-21swapper/009:31:340
339950,0migration/39947-21taskset07:23:063
339950,0migration/38627-21fschecks_time12:01:373
339950,0migration/37431-21kthreadcore07:46:413
339950,0migration/33775-21munin-node07:41:463
339950,0migration/33555-21kthreadcore10:36:403
339950,0migration/33349-21kthreadcore08:31:413
339950,0migration/330016-21timerwakeupswit12:11:463
339950,0migration/329866-21taskset07:37:193
339950,0migration/32874-21aten2.4-expect06:56:373
339950,0migration/328611-21kthreadcore09:11:413
339950,0migration/326816-21munin-node08:51:363
339950,0migration/324957-21unixbench-2d08:21:473
339950,0migration/324846-21aten_r1power_fr11:11:343
339950,0migration/323314-21munin-node09:46:443
339950,0migration/323278-21taskset11:28:343
339950,0migration/322349-21taskset07:08:373
339950,0migration/321489-21kthreadcore07:31:423
339950,0migration/319758-21taskset08:45:243
339950,0migration/318947-21kthreadcore10:26:433
339950,0migration/316037-21sh09:03:293
339950,0migration/314512-21munin-node10:41:433
339950,0migration/31306-21kthreadcore08:06:423
339950,0migration/31142-21taskset10:16:343
339950,0migration/311289-21kthreadcore09:21:413
29725050,0irq/27-eth0-tx-9736-21fschecks_count09:01:372
29725050,0irq/27-eth0-tx-9117-21diskstats11:01:372
29725050,0irq/27-eth0-tx-7727-21ssh10:17:432
29725050,0irq/27-eth0-tx-400-21ssh10:15:172
29725050,0irq/27-eth0-tx-26455-21cut11:11:392
29725050,0irq/27-eth0-tx-17874-21munin-node09:26:432
29725050,0irq/27-eth0-tx-17464-21munin-node10:06:452
29725050,0irq/27-eth0-tx-0-21swapper/210:43:062
29725050,0irq/27-eth0-tx-0-21swapper/210:25:042
29725050,0irq/27-eth0-tx-0-21swapper/209:45:332
29725050,0irq/27-eth0-tx-0-21swapper/209:19:172
29725050,0irq/27-eth0-tx-0-21swapper/209:16:022
29725050,0irq/27-eth0-tx-0-21swapper/008:36:440
29725050,0irq/27-eth0-tx-0-21swapper/007:48:260
29725050,0irq/27-eth0-tx-0-21swapper/007:36:440
29715050,0irq/26-eth0-rx-6754-21latency09:56:423
29715050,0irq/26-eth0-rx-32101-21ssh09:52:293
29715050,0irq/26-eth0-rx-30974-21sed10:51:473
29715050,0irq/26-eth0-rx-30091-21ntp_offset11:31:443
29715050,0irq/26-eth0-rx-16754-21gltestperf12:06:373
29715050,0irq/26-eth0-rx-0-21swapper/312:25:253
29715050,0irq/26-eth0-rx-0-21swapper/311:36:413
29715050,0irq/26-eth0-rx-0-21swapper/311:22:493
29715050,0irq/26-eth0-rx-0-21swapper/311:17:163
29715050,0irq/26-eth0-rx-0-21swapper/311:06:463
29715050,0irq/26-eth0-rx-0-21swapper/310:57:073
29715050,0irq/26-eth0-rx-0-21swapper/310:24:223
29715050,0irq/26-eth0-rx-0-21swapper/310:11:393
29715050,0irq/26-eth0-rx-0-21swapper/310:06:423
29715050,0irq/26-eth0-rx-0-21swapper/309:45:103
29715050,0irq/26-eth0-rx-0-21swapper/309:33:223
29715050,0irq/26-eth0-rx-0-21swapper/309:33:223
29715050,0irq/26-eth0-rx-0-21swapper/309:00:433
27150,0ktimersoftd/20-21swapper/211:32:062
25909955,0cyclictest0-21swapper/211:00:242
25909955,0cyclictest0-21swapper/210:52:032
25909955,0cyclictest0-21swapper/209:32:352
25909955,0cyclictest0-21swapper/209:32:352
25899955,0cyclictest0-21swapper/112:21:481
25899955,0cyclictest0-21swapper/111:18:161
25899955,0cyclictest0-21swapper/111:06:361
25899955,0cyclictest0-21swapper/109:40:591
25899955,0cyclictest0-21swapper/109:13:431
25899954,0cyclictest7673-21awk08:36:391
25899950,0cyclictest0-21swapper/111:43:291
25899950,0cyclictest0-21swapper/110:10:381
25889950,0cyclictest0-21swapper/010:55:170
259950,0migration/29077-21munin-node11:41:372
259950,0migration/24579-21kthreadcore10:36:412
259950,0migration/215363-21kthreadcore08:16:422
259950,0migration/214096-21unixbench_singl12:01:472
259950,0migration/210019-21munin-node09:21:362
249950,0watchdog/212406-21if_eth007:51:392
179950,0migration/124823-21latency_hist10:51:241
179950,0migration/124071-21cpuspeed_turbos07:36:371
169950,0watchdog/125069-21aten_r1power_en10:51:351
149950,0migration/08818-21aten_r1power_po11:01:350
149950,0migration/08674-21sh10:00:040
149950,0migration/08059-21sh10:39:320
149950,0migration/0733-21kthreadcore08:06:410
149950,0migration/05551-21kthreadcore10:56:410
149950,0migration/05240-21kthreadcore09:36:410
149950,0migration/04491-21kthreadcore10:16:410
149950,0migration/03231-21munin-node06:56:380
149950,0migration/031323-21sh11:54:180
149950,0migration/028205-21kthreadcore08:51:410
149950,0migration/027546-21kthreadcore12:11:400
149950,0migration/027159-21kthreadcore10:11:380
149950,0migration/027140-21kthreadcore08:26:410
149950,0migration/026142-21munin-node09:51:360
149950,0migration/02502-21kthreadcore11:56:400
149950,0migration/024411-21kthreadcore07:11:410
149950,0migration/022888-21sh12:08:520
149950,0migration/021917-21kthreadcore09:06:410
149950,0migration/020154-21aten2.4-expect08:46:350
149950,0migration/018750-21missed_timers08:41:430
149950,0migration/018310-21kthreadcore07:06:420
149950,0migration/016366-21munin-node11:46:350
149950,0migration/015161-21kthreadcore07:51:410
149950,0migration/015128-21timerandwakeup10:21:470
149950,0migration/015050-21sh11:43:390
149950,0migration/013992-21kthreadcore07:01:440
149950,0migration/013604-21kthreadcore10:01:420
8215040,0irq/24-0000:00:0-21swapper/207:31:252
339940,0migration/331406-21kthreadcore07:16:413
339940,0migration/328476-21kthreadcore11:51:413
339940,0migration/327738-21kthreadcore08:01:423
339940,0migration/323514-21kthreadcore08:46:413
339940,0migration/317064-21diskmemload11:46:383
339940,0migration/313769-21kthreadcore07:51:413
339940,0migration/311405-21kthreadcore07:01:433
29725040,0irq/27-eth0-tx-1918-21ssh09:36:352
29725040,0irq/27-eth0-tx-16820-21cpuspeed_turbos11:26:362
29725040,0irq/27-eth0-tx-0-21swapper/212:16:512
29725040,0irq/27-eth0-tx-0-21swapper/212:07:022
29725040,0irq/27-eth0-tx-0-21swapper/211:59:032
29725040,0irq/27-eth0-tx-0-21swapper/211:51:502
29725040,0irq/27-eth0-tx-0-21swapper/211:21:372
29725040,0irq/27-eth0-tx-0-21swapper/210:46:242
29725040,0irq/27-eth0-tx-0-21swapper/210:33:192
29725040,0irq/27-eth0-tx-0-21swapper/210:27:102
29725040,0irq/27-eth0-tx-0-21swapper/210:01:302
29725040,0irq/27-eth0-tx-0-21swapper/209:51:262
29725040,0irq/27-eth0-tx-0-21swapper/209:46:442
29725040,0irq/27-eth0-tx-0-21swapper/209:06:272
29725040,0irq/27-eth0-tx-0-21swapper/008:21:450
29715040,0irq/26-eth0-rx-0-21swapper/311:01:413
29715040,0irq/26-eth0-rx-0-21swapper/310:46:413
29715040,0irq/26-eth0-rx-0-21swapper/310:31:363
29715040,0irq/26-eth0-rx-0-21swapper/310:02:583
29715040,0irq/26-eth0-rx-0-21swapper/309:37:173
29715040,0irq/26-eth0-rx-0-21swapper/309:26:343
29715040,0irq/26-eth0-rx-0-21swapper/309:17:283
29715040,0irq/26-eth0-rx-0-21swapper/309:06:413
29715040,0irq/26-eth0-rx-0-21swapper/307:11:363
25909943,0cyclictest5658-21awk07:46:392
25909943,0cyclictest3556-21lldpd09:56:282
25909943,0cyclictest3556-21lldpd08:47:272
25909943,0cyclictest3556-21lldpd08:42:272
25909943,0cyclictest3556-21lldpd08:11:272
25909943,0cyclictest3556-21lldpd06:56:262
25909943,0cyclictest3514-21snmpd08:34:342
25909943,0cyclictest3514-21snmpd07:06:342
25909943,0cyclictest29252-21sadc07:16:252
25909943,0cyclictest17156-21awk11:46:392
25909940,0cyclictest0-21swapper/211:06:372
25899943,0cyclictest9855-21awk10:41:381
25899943,0cyclictest9684-21ps11:21:411
25899943,0cyclictest3556-21lldpd11:56:311
25899943,0cyclictest3556-21lldpd11:38:011
25899943,0cyclictest3556-21lldpd11:34:311
25899943,0cyclictest3556-21lldpd11:04:591
25899943,0cyclictest3556-21lldpd10:31:291
25899943,0cyclictest3556-21lldpd10:26:291
25899943,0cyclictest3556-21lldpd09:59:281
25899943,0cyclictest3556-21lldpd09:53:581
25899943,0cyclictest3556-21lldpd09:26:581
25899943,0cyclictest3556-21lldpd08:31:571
25899943,0cyclictest3556-21lldpd08:26:271
25899943,0cyclictest3556-21lldpd08:21:271
25899943,0cyclictest3556-21lldpd07:41:271
25899943,0cyclictest25232-21awk11:51:381
25899943,0cyclictest2446-21awk09:16:391
25899943,0cyclictest17893-21sadc07:56:251
25899943,0cyclictest17506-21awk11:26:391
25899943,0cyclictest10539-21awk09:41:391
25899943,0cyclictest10196-21awk09:01:391
25889944,0cyclictest0-21swapper/011:37:420
25889944,0cyclictest0-21swapper/009:31:150
25889943,0cyclictest9474-21awk11:21:400
25889943,0cyclictest30391-21awk07:16:400
25889943,0cyclictest25439-21awk11:31:390
25889943,0cyclictest2471-21awk09:16:390
25889943,0cyclictest18997-21awk07:56:380
25889943,0cyclictest17618-21awk11:06:380
25889943,0cyclictest10242-21sadc07:26:250
25889940,0cyclictest0-21swapper/010:46:560
259940,0migration/232550-21taskset11:36:342
259940,0migration/230466-21aten_r1power_cu07:41:352
259940,0migration/228106-21kthreadcore08:26:412
259940,0migration/227557-21hddtemp_smartct08:51:382
259940,0migration/224211-21aten2.4-expect12:11:352
259940,0migration/222788-21kthreadcore08:21:422
259940,0migration/214689-21kthreadcore07:26:422
259940,0migration/214047-21kthreadcore07:01:442
249940,0watchdog/28570-21idleruntime12:21:382
249940,0watchdog/23815-21munin-node07:21:392
249940,0watchdog/234122sleep207:19:512
249940,0watchdog/20-21swapper/208:57:032
249940,0watchdog/20-21swapper/208:06:392
249940,0watchdog/20-21swapper/207:56:392
179940,0migration/1908-21latency_hist08:56:251
179940,0migration/16766-21interrupts08:11:401
179940,0migration/15969-21kthreadcore07:21:411
179940,0migration/15146-21munin-node07:46:371
179940,0migration/14854-21kthreadcore12:16:411
179940,0migration/13356-21kthreadcore10:16:411
179940,0migration/131246-21kthreadcore07:16:411
179940,0migration/131089-21munin-node10:11:411
179940,0migration/131067-21taskset08:04:591
179940,0migration/12989-21kthreadcore10:56:411
179940,0migration/126849-21kthreadcore11:11:391
179940,0migration/125089-21irqcore12:11:391
179940,0migration/124667-21kthreadcore07:11:411
179940,0migration/123449-21kthreadcore08:46:411
179940,0migration/118994-21kthreadcore07:06:421
179940,0migration/117694-21munin-node09:46:341
179940,0migration/117305-21kthreadcore08:16:421
179940,0migration/115078-21kthreadcore08:41:411
179940,0migration/114343-21kthreadcore10:01:421
179940,0migration/11163-21cpuspeed_turbos10:36:361
179940,0migration/111177-21kthreadcore07:01:431
179940,0migration/110851-21kthreadcore12:01:411
179940,0migration/110442-21fschecks_time09:21:381
169940,0watchdog/126783-21if_err_eth109:31:391
169940,0watchdog/126783-21if_err_eth109:31:391
169940,0watchdog/122901-21ssh11:48:061
169940,0watchdog/119290-21ps07:56:391
169940,0watchdog/117546-21aten2.4-expect09:06:351
169940,0watchdog/10-21swapper/110:21:271
149940,0migration/032750-21sh11:16:280
339930,0migration/36195-21fschecks_count08:11:373
29725030,0irq/27-eth0-tx-0-21swapper/008:31:340
29725030,0irq/27-eth0-tx-0-21swapper/007:26:360
29715030,0irq/26-eth0-rx-0-21swapper/308:40:053
29715030,0irq/26-eth0-rx-0-21swapper/308:26:363
29715030,0irq/26-eth0-rx-0-21swapper/308:17:493
29715030,0irq/26-eth0-rx-0-21swapper/307:26:363
25909932,0cyclictest25687-21awk08:01:402
25899932,0cyclictest3556-21lldpd07:26:261
25899932,0cyclictest28149-21grep08:51:411
25889933,0cyclictest21719-21kworker/0:009:01:460
25889933,0cyclictest18244-21kworker/0:211:26:460
25889933,0cyclictest18244-21kworker/0:211:11:380
25889933,0cyclictest18244-21kworker/0:210:26:420
25889932,0cyclictest9654-21munin-node09:41:360
25889932,0cyclictest18784-21kernelversion10:06:410
25889932,0cyclictest17906-21cpuspeed_turbos09:46:350
25889932,0cyclictest17901-21grep07:31:380
25889930,0cyclictest5743-21munin-node08:11:420
25889930,0cyclictest0-21swapper/008:16:380
25889930,0cyclictest0-21swapper/008:01:410
25889930,0cyclictest0-21swapper/007:41:370
249930,0watchdog/20-21swapper/208:36:272
169930,0watchdog/131421-21ps08:06:271
169930,0watchdog/10-21swapper/107:31:351
29715020,0irq/26-eth0-rx-0-21swapper/307:56:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional