You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-20 - 15:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri Mar 20, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450350,0irq/25-eth10-21swapper/306:55:513
1276323524,0sleep00-21swapper/007:00:490
1265523526,0sleep10-21swapper/106:59:341
1268323324,0sleep20-21swapper/206:59:552
128339976,0cyclictest17632-21ssh09:02:031
339960,0migration/326265-21sh10:29:303
339960,0migration/324976-21sh11:07:383
339960,0migration/323639-21sh12:26:413
339960,0migration/319889-21irqrtprio10:46:053
339960,0migration/318158-21sh09:21:503
339960,0migration/317751-21sh11:25:353
339960,0migration/317210-21sh10:21:293
29715060,0irq/26-eth0-rx-1271-21ssh11:14:363
149960,0migration/05458-21kthreadcore09:56:100
149960,0migration/032679-21sh11:53:140
149960,0migration/027193-21sh09:50:430
149960,0migration/018990-21irqcore11:46:050
149960,0migration/01822-21latency_hist11:15:500
149960,0migration/017910-21sh10:44:240
149960,0migration/016732-21idleruntime08:16:040
149960,0migration/015809-21sh12:01:240
128339966,0cyclictest10974-21diskmemload12:05:531
128339960,0cyclictest0-21swapper/112:16:461
128339960,0cyclictest0-21swapper/110:21:401
128339960,0cyclictest0-21swapper/110:14:161
339950,0migration/39995-21ntp_offset08:56:113
339950,0migration/39353-21aten_r1power_fr08:11:013
339950,0migration/38133-21smart_sdb10:56:113
339950,0migration/38043-21kthreadcore10:16:063
339950,0migration/33395-21runrttasks09:35:563
339950,0migration/331418-21irqrtprio08:51:063
339950,0migration/330166-21kthreadcore09:11:063
339950,0migration/329189-21kthreadcore12:11:063
339950,0migration/328925-21kthreadcore09:51:073
339950,0migration/327483-21kthreadcore11:31:053
339950,0migration/326696-21taskset07:31:343
339950,0migration/325052-21kthreadcore09:26:083
339950,0migration/324306-21taskset11:26:213
339950,0migration/324238-21fschecks_time08:46:043
339950,0migration/317362-21fschecks_count08:41:043
339950,0migration/317020-21taskset11:02:383
339950,0migration/316793-21kthreadcore07:01:073
339950,0migration/314723-21latency12:21:063
339950,0migration/310898-21irqcore11:41:063
339950,0migration/310690-21irqcore12:01:063
339950,0migration/310638-21runrttasks10:40:513
29725050,0irq/27-eth0-tx-7075-21ssh11:56:062
29725050,0irq/27-eth0-tx-7075-21ssh11:56:062
29725050,0irq/27-eth0-tx-29432-21cut11:51:082
29725050,0irq/27-eth0-tx-28158-21taskset10:11:042
29725050,0irq/27-eth0-tx-23541-21munin-node08:46:090
29725050,0irq/27-eth0-tx-19333-21munin-node09:06:132
29725050,0irq/27-eth0-tx-18115-21munin-node11:26:132
29725050,0irq/27-eth0-tx-13998-21munin-node07:26:030
29725050,0irq/27-eth0-tx-0-21swapper/212:21:552
29725050,0irq/27-eth0-tx-0-21swapper/210:20:362
29725050,0irq/27-eth0-tx-0-21swapper/209:36:142
29725050,0irq/27-eth0-tx-0-21swapper/209:27:312
29725050,0irq/27-eth0-tx-0-21swapper/209:23:502
29725050,0irq/27-eth0-tx-0-21swapper/209:03:382
29725050,0irq/27-eth0-tx-0-21swapper/007:16:020
29715050,0irq/26-eth0-rx-4402-21cat07:41:053
29715050,0irq/26-eth0-rx-29800-21kthreadcore10:51:053
29715050,0irq/26-eth0-rx-28870-21interrupts09:31:063
29715050,0irq/26-eth0-rx-23649-21unixbench_singl12:06:113
29715050,0irq/26-eth0-rx-2321-21if_eth011:56:023
29715050,0irq/26-eth0-rx-2321-21if_eth011:56:023
29715050,0irq/26-eth0-rx-17721-21sh10:02:173
29715050,0irq/26-eth0-rx-0-21swapper/311:55:093
29715050,0irq/26-eth0-rx-0-21swapper/311:46:063
29715050,0irq/26-eth0-rx-0-21swapper/311:36:353
29715050,0irq/26-eth0-rx-0-21swapper/311:16:083
29715050,0irq/26-eth0-rx-0-21swapper/310:38:473
29715050,0irq/26-eth0-rx-0-21swapper/310:06:083
29715050,0irq/26-eth0-rx-0-21swapper/310:00:243
29715050,0irq/26-eth0-rx-0-21swapper/309:06:053
259950,0migration/29786-21cpuspeed_turbos12:01:012
259950,0migration/28939-21kthreadcore08:56:092
259950,0migration/28565-21timerandwakeup10:36:122
259950,0migration/24794-21kthreadcore11:36:062
259950,0migration/24244-21idleruntime09:16:042
259950,0migration/227597-21aten_r1power_en09:11:012
259950,0migration/226481-21munin-node12:11:042
259950,0migration/225907-21sh11:10:202
259950,0migration/224324-21sh11:47:162
259950,0migration/218548-21munin-node07:01:102
259950,0migration/216820-21latency_hist08:40:502
259950,0migration/216502-21munin-node10:21:082
249950,0watchdog/227997-21perl07:36:052
249950,0watchdog/221691-21cut07:56:012
19150,0ktimersoftd/10-21swapper/111:13:131
179950,0migration/18451-21sendmail_mailtr10:36:111
179950,0migration/18127-21taskset11:57:321
179950,0migration/18127-21taskset11:57:321
179950,0migration/132726-21kthreadcore09:51:091
179950,0migration/13113-21idleruntime10:56:031
179950,0migration/124427-21sendmail_mailqu10:46:101
169950,0watchdog/126456-21aten_r1power_vo07:11:011
149950,0migration/09179-21kthreadcore08:56:090
149950,0migration/08208-21taskset12:18:250
149950,0migration/07682-21taskset07:41:280
149950,0migration/04767-21kthreadcore10:36:060
149950,0migration/03974-21fschecks_count08:31:020
149950,0migration/032557-21sh12:13:380
149950,0migration/03128-21taskset09:35:500
149950,0migration/030390-21kthreadcore10:11:060
149950,0migration/02813-21irqcore11:36:050
149950,0migration/02701-21sh09:55:270
149950,0migration/025832-21sh11:30:490
149950,0migration/020264-21kthreadcore12:06:050
149950,0migration/018454-21munin-node11:06:020
149950,0migration/016955-21unixbench-2d10:21:120
149950,0migration/015566-21taskset12:21:250
149950,0migration/015134-21kthreadcore09:01:080
149950,0migration/013997-21munin-node07:01:050
128349955,0cyclictest0-21swapper/210:56:232
128349950,0cyclictest0-21swapper/212:08:262
128349950,0cyclictest0-21swapper/210:33:502
128349950,0cyclictest0-21swapper/210:02:562
128349950,0cyclictest0-21swapper/209:56:192
128339955,0cyclictest0-21swapper/111:34:311
128339955,0cyclictest0-21swapper/107:46:131
128339950,0cyclictest0-21swapper/112:28:261
128339950,0cyclictest0-21swapper/111:02:301
128339950,0cyclictest0-21swapper/110:08:261
128339950,0cyclictest0-21swapper/108:52:171
128329950,0cyclictest0-21swapper/012:26:010
128329950,0cyclictest0-21swapper/011:19:480
128329950,0cyclictest0-21swapper/010:16:250
128329950,0cyclictest0-21swapper/008:12:200
8215040,0irq/24-0000:00:0-21swapper/210:45:502
339940,0migration/38529-21kthreadcore08:31:073
339940,0migration/37026-21kthreadcore09:16:073
339940,0migration/328689-21kthreadcore07:36:083
339940,0migration/328512-21cpuspeed_turbos08:01:023
339940,0migration/322197-21kthreadcore07:06:063
339940,0migration/32167-21diskmemload10:14:373
339940,0migration/310724-21fschecks_count08:36:023
29725040,0irq/27-eth0-tx-0-21swapper/212:15:542
29725040,0irq/27-eth0-tx-0-21swapper/211:41:012
29725040,0irq/27-eth0-tx-0-21swapper/211:20:582
29725040,0irq/27-eth0-tx-0-21swapper/211:17:102
29725040,0irq/27-eth0-tx-0-21swapper/211:00:592
29725040,0irq/27-eth0-tx-0-21swapper/210:51:012
29725040,0irq/27-eth0-tx-0-21swapper/210:41:172
29725040,0irq/27-eth0-tx-0-21swapper/210:25:582
29725040,0irq/27-eth0-tx-0-21swapper/209:51:072
29725040,0irq/27-eth0-tx-0-21swapper/209:30:542
29725040,0irq/27-eth0-tx-0-21swapper/007:55:190
29725040,0irq/27-eth0-tx-0-21swapper/007:08:180
29715040,0irq/26-eth0-rx-20034-21sshd07:26:263
29715040,0irq/26-eth0-rx-0-21swapper/312:16:013
29715040,0irq/26-eth0-rx-0-21swapper/310:31:053
29715040,0irq/26-eth0-rx-0-21swapper/309:46:063
29715040,0irq/26-eth0-rx-0-21swapper/309:43:273
29715040,0irq/26-eth0-rx-0-21swapper/309:01:103
259940,0migration/230627-21kthreadcore08:26:072
259940,0migration/227989-21kthreadcore11:11:072
259940,0migration/225871-21kthreadcore08:46:072
259940,0migration/225807-21kthreadcore08:21:082
259940,0migration/215631-21fschecks_time07:51:042
259940,0migration/213526-21kthreadcore09:41:052
249940,0watchdog/27975-21needreboot08:06:092
249940,0watchdog/227483-21grep07:11:052
249940,0watchdog/20-21swapper/207:46:052
249940,0watchdog/20-21swapper/207:26:012
179940,0migration/17263-21kthreadcore09:36:071
179940,0migration/16786-21kthreadcore11:16:081
179940,0migration/15937-21kthreadcore08:31:061
179940,0migration/13774-21irqrtprio08:06:061
179940,0migration/128460-21kthreadcore12:11:061
179940,0migration/126041-21kthreadcore08:21:081
179940,0migration/124983-21processes09:06:091
179940,0migration/124129-21fschecks_time08:46:041
179940,0migration/123267-21taskset08:45:261
179940,0migration/122267-21kthreadcore07:31:071
179940,0migration/121317-21kthreadcore09:46:061
179940,0migration/118446-21fschecks_time11:46:021
179940,0migration/117779-21taskset10:44:031
179940,0migration/11704-21taskset11:35:571
179940,0migration/115557-21runrttasks08:14:171
179940,0migration/114153-21kthreadcore09:41:061
179940,0migration/11218-21missed_timers08:01:081
179940,0migration/110160-21cpuspeed_turbos11:41:021
169940,0watchdog/18312-21grep08:56:091
169940,0watchdog/14047-21irqcore10:16:041
169940,0watchdog/13618-21kthreadcore07:41:051
169940,0watchdog/132255-21open_inodes10:51:081
169940,0watchdog/129235-21kthreadcore11:51:081
169940,0watchdog/125494-21ssh10:27:161
169940,0watchdog/125238-21grep09:26:091
169940,0watchdog/11716-21ssh09:12:131
169940,0watchdog/116438-21ssh12:03:161
169940,0watchdog/113999-21aten_r1power_cu07:26:011
169940,0watchdog/10-21swapper/111:06:081
1565240,0sleep10-21swapper/107:16:071
149940,0migration/09395-21irqcore07:46:060
149940,0migration/05153-21kthreadcore08:06:070
149940,0migration/031891-21kthreadcore08:51:070
149940,0migration/028503-21cpuspeed_turbos08:01:020
149940,0migration/02422-21aten_r1power_vo10:56:010
149940,0migration/021229-21kthreadcore10:26:070
149940,0migration/018810-21sh09:44:340
149940,0migration/012139-21kthreadcore11:41:070
128349943,0cyclictest3556-21lldpd11:30:512
128349943,0cyclictest3556-21lldpd08:32:502
128349943,0cyclictest3514-21snmpd08:13:562
128349943,0cyclictest3514-21snmpd07:42:562
128349943,0cyclictest3514-21snmpd07:21:412
128349943,0cyclictest3514-21snmpd07:08:412
128349943,0cyclictest31059-21awk08:51:052
128349943,0cyclictest20376-21awk09:46:042
128349943,0cyclictest19241-21munin-node10:06:132
128349943,0cyclictest16838-21awk08:16:052
128349943,0cyclictest1470-21awk07:16:052
128349940,0cyclictest0-21swapper/212:26:062
128339944,0cyclictest0-21swapper/110:33:491
128339944,0cyclictest0-21swapper/109:32:381
128339943,0cyclictest4206-21ps09:56:091
128339943,0cyclictest3514-21snmpd11:20:581
128339943,0cyclictest3514-21snmpd09:22:421
128339943,0cyclictest3514-21snmpd07:20:561
128339943,0cyclictest30160-21awk08:26:051
128339943,0cyclictest22794-21grep07:56:061
128339943,0cyclictest19004-21awk11:26:051
128339943,0cyclictest12064-21awk10:01:051
128329944,0cyclictest0-21swapper/011:57:380
128329944,0cyclictest0-21swapper/011:57:380
128329944,0cyclictest0-21swapper/009:40:470
128329944,0cyclictest0-21swapper/009:26:260
128329943,0cyclictest3514-21snmpd10:49:580
128329943,0cyclictest3514-21snmpd09:17:120
128329943,0cyclictest3241-21irqbalance11:22:380
128329943,0cyclictest3241-21irqbalance11:01:480
128329943,0cyclictest27845-21awk10:31:030
128329943,0cyclictest12594-21awk09:21:050
339930,0migration/322731-21sh08:21:023
29715030,0irq/26-eth0-rx-0-21swapper/308:26:053
29715030,0irq/26-eth0-rx-0-21swapper/308:18:253
29715030,0irq/26-eth0-rx-0-21swapper/307:56:013
29715030,0irq/26-eth0-rx-0-21swapper/307:46:023
29715030,0irq/26-eth0-rx-0-21swapper/307:16:043
29715030,0irq/26-eth0-rx-0-21swapper/307:11:253
259930,0migration/228190-21latency_hist08:00:502
169930,0watchdog/110974-21diskmemload09:15:531
169930,0watchdog/10-21swapper/108:16:011
169930,0watchdog/10-21swapper/107:51:051
169930,0watchdog/10-21swapper/107:06:011
128349933,0cyclictest11202-21awk08:36:052
128339933,0cyclictest0-21swapper/108:39:211
128339932,0cyclictest18670-21sensors07:01:111
128339932,0cyclictest15119-21sensors12:21:101
128339931,0cyclictest0-21swapper/107:36:081
128329933,0cyclictest20099-21awk10:06:040
128329933,0cyclictest10123-21kworker/0:111:30:590
128329933,0cyclictest10123-21kworker/0:110:51:050
128329933,0cyclictest10123-21kworker/0:108:41:040
128329933,0cyclictest10123-21kworker/0:108:38:160
128329933,0cyclictest10123-21kworker/0:108:26:340
128329933,0cyclictest10123-21kworker/0:107:20:580
128329933,0cyclictest10123-21kworker/0:107:11:000
128329932,0cyclictest3241-21irqbalance10:01:090
128329932,0cyclictest28490-21awk09:11:040
128329932,0cyclictest22602-21awk07:56:050
128329932,0cyclictest22559-21wc08:20:500
128329932,0cyclictest20257-21awk09:06:040
128329930,0cyclictest27302-21munin-node07:36:040
128329930,0cyclictest20651-21munin-node07:31:050
35120,0ktimersoftd/320669-21munin-node07:51:133
29715020,0irq/26-eth0-rx-0-21swapper/308:06:053
29715020,0irq/26-eth0-rx-0-21swapper/307:21:303
249920,0watchdog/20-21swapper/207:31:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional