You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 19:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon May 25, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
774825441,0sleep30-21swapper/306:54:323
752423424,0sleep10-21swapper/106:51:541
750623026,0sleep00-21swapper/006:51:450
740722622,0sleep20-21swapper/206:50:352
78289960,0cyclictest0-21swapper/210:08:102
78259960,0cyclictest0-21swapper/011:40:060
78259960,0cyclictest0-21swapper/011:06:310
339960,0migration/320704-21sh12:04:183
29715060,0irq/26-eth0-rx-28421-21diskstats07:09:513
29715060,0irq/26-eth0-rx-27814-21sed09:44:553
29715060,0irq/26-eth0-rx-27269-21sensors11:24:583
29715060,0irq/26-eth0-rx-19271-21awk10:59:553
29715060,0irq/26-eth0-rx-18173-21tr11:19:523
29715060,0irq/26-eth0-rx-11019-21ssh11:34:573
149960,0migration/029675-21sh09:48:150
149960,0migration/017192-21kthreadcore11:59:530
149960,0migration/012568-21sh12:19:290
8215050,0irq/24-0000:00:6346-21cat09:14:372
8215050,0irq/24-0000:00:31625-21cat08:49:372
78289955,0cyclictest0-21swapper/212:05:462
78289955,0cyclictest0-21swapper/211:14:422
78289955,0cyclictest0-21swapper/211:14:422
78289955,0cyclictest0-21swapper/210:16:502
78289950,0cyclictest0-21swapper/211:43:142
78289950,0cyclictest0-21swapper/211:25:342
78289950,0cyclictest0-21swapper/207:59:592
78279955,0cyclictest0-21swapper/112:16:271
78279955,0cyclictest0-21swapper/111:33:151
78279955,0cyclictest0-21swapper/110:08:351
78259955,0cyclictest2048-21ssh12:09:540
78259955,0cyclictest0-21swapper/009:54:570
78259955,0cyclictest0-21swapper/009:34:550
78259950,0cyclictest0-21swapper/012:05:480
339950,0migration/37545-21kthreadcore08:54:543
339950,0migration/37091-21kthreadcore10:54:523
339950,0migration/36765-21munin-node09:14:473
339950,0migration/36292-21runrttasks09:34:413
339950,0migration/33959-21sh10:30:143
339950,0migration/331782-21kthreadcore09:49:533
339950,0migration/330523-21latency_hist09:09:373
339950,0migration/328612-21taskset12:08:533
339950,0migration/32650-21taskset07:35:513
339950,0migration/324598-21df_abs08:19:503
339950,0migration/323900-21munin-node09:24:523
339950,0migration/323113-21munin-node10:24:523
339950,0migration/323007-21munin-node10:04:513
339950,0migration/321188-21latency_hist11:44:373
339950,0migration/319703-21switchtime10:39:573
339950,0migration/319082-21kthreadcore06:59:543
339950,0migration/318690-21missed_timers11:39:533
339950,0migration/318332-21hddtemp_smartct08:14:503
339950,0migration/317355-21kthreadcore09:59:523
339950,0migration/317045-21kthreadcore12:19:543
339950,0migration/315170-21if_eth008:59:533
339950,0migration/312403-21taskset11:17:193
339950,0migration/312403-21taskset11:17:193
339950,0migration/310476-21munin-node08:29:553
29725050,0irq/27-eth0-tx-5753-21munin-node10:34:522
29725050,0irq/27-eth0-tx-4216-21ssh10:51:202
29725050,0irq/27-eth0-tx-3405-21ssh11:50:502
29725050,0irq/27-eth0-tx-29749-21munin-node11:09:562
29725050,0irq/27-eth0-tx-28439-21ssh10:25:342
29725050,0irq/27-eth0-tx-20518-21cat09:19:592
29725050,0irq/27-eth0-tx-18266-21load12:19:542
29725050,0irq/27-eth0-tx-17570-21grep10:39:522
29725050,0irq/27-eth0-tx-0-21swapper/210:45:302
29725050,0irq/27-eth0-tx-0-21swapper/210:09:542
29725050,0irq/27-eth0-tx-0-21swapper/209:49:582
29725050,0irq/27-eth0-tx-0-21swapper/209:48:572
29725050,0irq/27-eth0-tx-0-21swapper/008:34:060
29715050,0irq/26-eth0-rx-6117-21df_abs10:34:493
29715050,0irq/26-eth0-rx-2553-21kthreadcore09:29:553
29715050,0irq/26-eth0-rx-23592-21cut09:04:543
29715050,0irq/26-eth0-rx-23487-21munin-node07:54:493
29715050,0irq/26-eth0-rx-20828-21tr08:39:533
29715050,0irq/26-eth0-rx-16154-21cut09:39:533
29715050,0irq/26-eth0-rx-0-21swapper/312:17:453
29715050,0irq/26-eth0-rx-0-21swapper/312:10:433
29715050,0irq/26-eth0-rx-0-21swapper/311:53:313
29715050,0irq/26-eth0-rx-0-21swapper/311:32:173
29715050,0irq/26-eth0-rx-0-21swapper/311:04:493
29715050,0irq/26-eth0-rx-0-21swapper/310:45:113
29715050,0irq/26-eth0-rx-0-21swapper/310:19:183
29715050,0irq/26-eth0-rx-0-21swapper/310:09:483
29715050,0irq/26-eth0-rx-0-21swapper/309:56:543
29715050,0irq/26-eth0-rx-0-21swapper/309:19:533
27150,0ktimersoftd/20-21swapper/211:22:442
27150,0ktimersoftd/20-21swapper/208:33:502
259950,0migration/29611-21kthreadcore10:54:532
259950,0migration/24471-21sh11:33:202
259950,0migration/229700-21gltestperf07:34:522
259950,0migration/22681-21kthreadcore09:29:552
259950,0migration/225233-21kthreadcore11:04:542
259950,0migration/22486-21sendmail_mailst12:09:572
259950,0migration/224445-21kthreadcore07:04:542
259950,0migration/217974-21kthreadcore07:24:532
259950,0migration/216087-21kthreadcore09:39:532
259950,0migration/214160-21latency_hist08:59:372
259950,0migration/213206-21sh09:57:382
259950,0migration/212507-21kthreadcore08:09:552
259950,0migration/211023-21sendmail_mailtr11:34:572
249950,0watchdog/230743-21grep07:09:552
179950,0migration/19050-21kthreadcore10:54:531
179950,0migration/16101-21sh11:34:501
179950,0migration/131807-21kthreadcore07:34:551
179950,0migration/127476-21sh12:05:361
179950,0migration/12483-21munin-node11:49:551
179950,0migration/12468-21munin-node12:09:571
179950,0migration/118845-21sh11:59:581
179950,0migration/116014-21kthreadcore09:19:531
179950,0migration/115419-21munin-node09:39:521
179950,0migration/113496-21sh08:58:221
179950,0migration/112539-21munin-node09:14:581
169950,0watchdog/11760-21kthreadcore10:49:551
149950,0migration/08017-21munin-node07:14:590
149950,0migration/06549-21kthreadcore11:54:520
149950,0migration/06309-21latency_hist09:14:370
149950,0migration/0577-21kthreadcore11:09:520
149950,0migration/05580-21kthreadcore07:39:510
149950,0migration/05043-21latency_hist08:29:370
149950,0migration/032205-21kthreadcore11:49:520
149950,0migration/031573-21kthreadcore10:49:540
149950,0migration/030948-21timerwakeupswit08:44:590
149950,0migration/030687-21aten_r1power_fr09:09:470
149950,0migration/029984-21latency_hist10:29:370
149950,0migration/029902-21sh09:28:220
149950,0migration/02859-21missed_timers07:59:570
149950,0migration/027332-21unixbench-2d11:44:590
149950,0migration/02652-21kthreadcore09:49:540
149950,0migration/025158-21kthreadcore09:04:550
149950,0migration/025048-21kthreadcore07:54:530
149950,0migration/025010-21kthreadcore10:44:540
149950,0migration/024267-21aten_r1power_en08:19:490
149950,0migration/024028-21latency_hist08:19:370
149950,0migration/022233-21latency_hist07:29:370
149950,0migration/017818-21kthreadcore07:24:530
149950,0migration/016223-21kthreadcore10:19:520
149950,0migration/015639-21kthreadcore10:39:520
149950,0migration/015052-21kthreadcore11:19:510
149950,0migration/013696-21hwlatdetect12:19:500
149950,0migration/013587-21kthreadcore08:34:540
149950,0migration/012621-21sh10:57:120
78289944,0cyclictest0-21swapper/209:09:402
78289943,0cyclictest5655-21awk12:14:502
78289943,0cyclictest3514-21snmpd10:31:512
78289943,0cyclictest3514-21snmpd08:55:512
78289943,0cyclictest3514-21snmpd08:40:362
78289943,0cyclictest24245-21awk07:54:522
78279944,0cyclictest0-21swapper/111:56:161
78279944,0cyclictest0-21swapper/110:27:211
78279944,0cyclictest0-21swapper/109:13:241
78279944,0cyclictest0-21swapper/108:25:001
78279943,0cyclictest7019-21awk09:54:511
78279943,0cyclictest6335-21awk11:14:501
78279943,0cyclictest6335-21awk11:14:501
78279943,0cyclictest3514-21snmpd11:43:221
78279943,0cyclictest3514-21snmpd10:19:511
78279943,0cyclictest3514-21snmpd10:11:061
78279943,0cyclictest3514-21snmpd08:32:511
78279943,0cyclictest3514-21snmpd07:32:201
78279943,0cyclictest3514-21snmpd07:22:191
78279943,0cyclictest3514-21snmpd07:10:341
78279943,0cyclictest3241-21irqbalance11:09:451
78279943,0cyclictest3241-21irqbalance11:08:251
78279943,0cyclictest3241-21irqbalance10:16:351
78279943,0cyclictest3241-21irqbalance09:46:051
78279943,0cyclictest3241-21irqbalance07:39:451
78279943,0cyclictest3130-21awk07:14:521
78279943,0cyclictest25984-21awk08:44:511
78279943,0cyclictest25092-21awk08:19:521
78279943,0cyclictest24197-21awk07:54:521
78279943,0cyclictest19437-21awk08:39:511
78279943,0cyclictest16565-21awk07:24:521
78279943,0cyclictest15674-21awk06:59:521
78279943,0cyclictest12671-21awk08:34:521
78279943,0cyclictest11805-21awk08:09:531
78259944,0cyclictest0-21swapper/011:31:260
78259944,0cyclictest0-21swapper/010:26:480
78259944,0cyclictest0-21swapper/010:09:270
78259944,0cyclictest0-21swapper/009:30:120
78259943,0cyclictest3556-21lldpd11:15:280
78259943,0cyclictest3556-21lldpd11:15:280
78259943,0cyclictest3556-21lldpd11:01:280
78259943,0cyclictest3514-21snmpd08:42:510
78259943,0cyclictest3241-21irqbalance10:11:450
78259943,0cyclictest3241-21irqbalance09:42:550
78259943,0cyclictest32053-21kworker/0:209:22:380
78259943,0cyclictest29868-21awk07:34:530
78259943,0cyclictest23027-21sadc07:54:370
78259943,0cyclictest15154-21awk08:59:530
78259942,0cyclictest32053-21kworker/0:207:10:390
339940,0migration/38683-21fschecks_time06:54:503
339940,0migration/36616-21kthreadcore08:04:513
339940,0migration/332282-21kthreadcore07:59:553
339940,0migration/316621-21taskset07:49:413
29725040,0irq/27-eth0-tx-22395-21ssh09:04:462
29725040,0irq/27-eth0-tx-21270-21ssh11:44:402
29725040,0irq/27-eth0-tx-15568-21kthreadcore10:59:522
29725040,0irq/27-eth0-tx-10617-21awk11:54:542
29725040,0irq/27-eth0-tx-0-21swapper/211:59:502
29725040,0irq/27-eth0-tx-0-21swapper/210:02:412
29725040,0irq/27-eth0-tx-0-21swapper/209:25:142
29725040,0irq/27-eth0-tx-0-21swapper/008:54:500
29725040,0irq/27-eth0-tx-0-21swapper/008:04:540
29725040,0irq/27-eth0-tx-0-21swapper/007:45:000
29715040,0irq/26-eth0-rx-25317-21cpu08:44:493
29715040,0irq/26-eth0-rx-0-21swapper/311:54:563
29715040,0irq/26-eth0-rx-0-21swapper/311:11:013
29715040,0irq/26-eth0-rx-0-21swapper/310:49:533
29715040,0irq/26-eth0-rx-0-21swapper/310:22:183
29715040,0irq/26-eth0-rx-0-21swapper/307:04:483
259940,0migration/28220-21kthreadcore09:34:542
259940,0migration/27194-21kthreadcore08:04:512
259940,0migration/223005-21fschecks_time07:29:502
259940,0migration/22255-21kthreadcore08:24:532
259940,0migration/215152-21irqcore10:19:522
259940,0migration/21193-21kthreadcore08:49:542
249940,0watchdog/29910-21if_eth007:19:512
249940,0watchdog/229000-21kthreadcore08:19:552
249940,0watchdog/216242-21kthreadcore08:34:552
249940,0watchdog/20-21swapper/207:39:512
179940,0migration/17234-21kthreadcore10:34:531
179940,0migration/130413-21taskset09:49:421
179940,0migration/128562-21timerwakeupswit09:24:591
179940,0migration/12532-21kthreadcore08:49:541
179940,0migration/124179-21kthreadcore07:04:541
179940,0migration/122228-21gltestperf11:24:491
179940,0migration/117232-21kthreadcore10:39:521
179940,0migration/116617-21kthreadcore09:59:521
169940,0watchdog/16286-21kthreadcore08:04:511
169940,0watchdog/131262-21cut09:29:511
169940,0watchdog/130122-21aten_r1power_cu10:29:471
169940,0watchdog/128096-21ssh10:45:151
169940,0watchdog/124941-21cat09:04:551
169940,0watchdog/122998-21ntp_kernel_err08:14:551
169940,0watchdog/121284-21ssh11:44:421
169940,0watchdog/119849-21ssh11:20:381
169940,0watchdog/119628-21timerandwakeup10:59:581
169940,0watchdog/1193112sleep112:20:501
169940,0watchdog/10-21swapper/109:34:551
149940,0migration/021854-21sh10:04:230
78289933,0cyclictest0-21swapper/206:55:112
78289932,0cyclictest3514-21snmpd07:46:202
78289932,0cyclictest2169-21cat07:14:372
78289932,0cyclictest18819-21cut07:49:532
78289932,0cyclictest15601-21awk06:59:522
78279933,0cyclictest8911-21awk06:54:511
78279932,0cyclictest30740-21ls07:59:521
78279932,0cyclictest14328-21basename08:59:491
78279932,0cyclictest12055-21grep07:44:541
78259933,0cyclictest32053-21kworker/0:208:49:480
78259933,0cyclictest32053-21kworker/0:207:19:470
78259933,0cyclictest32053-21kworker/0:207:04:470
78259933,0cyclictest32053-21kworker/0:206:59:390
78259933,0cyclictest32053-21kworker/0:206:54:490
78259933,0cyclictest30709-21kworker/0:111:34:380
78259933,0cyclictest30709-21kworker/0:111:24:380
78259933,0cyclictest15183-21kworker/0:010:34:420
78259933,0cyclictest15183-21kworker/0:010:14:420
29725030,0irq/27-eth0-tx-0-21swapper/008:09:380
29715030,0irq/26-eth0-rx-0-21swapper/308:50:283
29715030,0irq/26-eth0-rx-0-21swapper/308:34:493
29715030,0irq/26-eth0-rx-0-21swapper/308:09:493
29715030,0irq/26-eth0-rx-0-21swapper/307:44:473
29715030,0irq/26-eth0-rx-0-21swapper/307:39:453
29715030,0irq/26-eth0-rx-0-21swapper/307:29:473
29715030,0irq/26-eth0-rx-0-21swapper/307:24:483
29715030,0irq/26-eth0-rx-0-21swapper/307:21:143
29715030,0irq/26-eth0-rx-0-21swapper/307:17:383
249930,0watchdog/20-21swapper/208:14:552
179930,0migration/120922-21kthreadcore07:49:541
29715020,0irq/26-eth0-rx-0-21swapper/308:24:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional