You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 06:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Mon Aug 03, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450550,0irq/25-eth10-21swapper/318:44:543
1206923622,0sleep00-21swapper/018:45:230
1205023523,0sleep20-21swapper/218:45:092
1226623425,0sleep10-21swapper/118:47:441
29725080,0irq/27-eth0-tx-0-21swapper/221:03:252
339960,0migration/331231-21sh23:02:293
339960,0migration/330790-21timerandwakeup22:18:203
339960,0migration/325634-21munin-node19:18:163
339960,0migration/321919-21kthreadcore21:33:183
339960,0migration/321265-21kthreadcore18:53:163
339960,0migration/316867-21sh21:29:593
339960,0migration/315461-21sh22:09:123
29725060,0irq/27-eth0-tx-12578-21munin-node18:48:140
29725060,0irq/27-eth0-tx-11106-21cpu20:48:110
29725060,0irq/27-eth0-tx-10736-21munin-node21:08:122
29715060,0irq/26-eth0-rx-19871-21sensors_temp23:53:213
29715060,0irq/26-eth0-rx-11394-21grep22:28:153
29715060,0irq/26-eth0-rx-0-21swapper/322:48:163
27160,0ktimersoftd/20-21swapper/223:23:442
149960,0migration/06229-21taskset22:44:010
339950,0migration/39226-21munin-node20:18:233
339950,0migration/36110-21kthreadcore19:28:173
339950,0migration/35708-21cpuspeed_turbos00:08:103
339950,0migration/35136-21kthreadcore21:03:193
339950,0migration/33898-21kthreadcore22:03:173
339950,0migration/332295-21aten_r1power_fr22:43:093
339950,0migration/332234-21open_files21:18:183
339950,0migration/33147-21munin-node19:53:143
339950,0migration/330870-21kthreadcore19:48:163
339950,0migration/32732-21fschecks_count21:23:123
339950,0migration/326218-21gltestperf21:58:123
339950,0migration/325952-21kthreadcore23:38:163
339950,0migration/32494-21kthreadcore20:38:173
339950,0migration/324780-21unixbench_multi21:13:233
339950,0migration/321307-21kthreadcore20:28:173
339950,0migration/315541-21cpuspeed_turbos23:13:103
339950,0migration/314809-21sh23:12:343
339950,0migration/314136-21fschecks_time00:13:143
339950,0migration/313223-21runrttasks23:29:363
339950,0migration/311845-21munin-node23:48:183
339950,0migration/311826-21munin-node20:48:143
29725050,0irq/27-eth0-tx-32438-21ssh22:01:002
29725050,0irq/27-eth0-tx-27421-21kthreadcore21:38:162
29725050,0irq/27-eth0-tx-24996-21kernelversion22:58:152
29725050,0irq/27-eth0-tx-24949-21ssh21:13:352
29725050,0irq/27-eth0-tx-0-21swapper/223:49:012
29725050,0irq/27-eth0-tx-0-21swapper/222:35:142
29725050,0irq/27-eth0-tx-0-21swapper/222:09:392
29725050,0irq/27-eth0-tx-0-21swapper/222:03:172
29725050,0irq/27-eth0-tx-0-21swapper/018:57:240
29715050,0irq/26-eth0-rx-4305-21ssh00:05:183
29715050,0irq/26-eth0-rx-3944-21grep22:23:153
29715050,0irq/26-eth0-rx-31688-21pmu-power21:38:203
29715050,0irq/26-eth0-rx-17938-21kthreadcore23:33:153
29715050,0irq/26-eth0-rx-1747-21kthreadcore23:23:163
29715050,0irq/26-eth0-rx-15764-21munin-node22:53:203
29715050,0irq/26-eth0-rx-0-21swapper/323:18:573
29715050,0irq/26-eth0-rx-0-21swapper/323:04:143
29715050,0irq/26-eth0-rx-0-21swapper/322:39:313
29715050,0irq/26-eth0-rx-0-21swapper/322:37:123
29715050,0irq/26-eth0-rx-0-21swapper/322:15:263
29715050,0irq/26-eth0-rx-0-21swapper/321:49:323
29715050,0irq/26-eth0-rx-0-21swapper/321:46:313
29715050,0irq/26-eth0-rx-0-21swapper/321:09:203
29715050,0irq/26-eth0-rx-0-21swapper/319:08:163
27150,0ktimersoftd/20-21swapper/222:30:362
259950,0migration/29381-21latency_hist21:47:582
259950,0migration/28737-21if_eth019:33:162
259950,0migration/2632-21sh20:58:422
259950,0migration/23704-21sh00:03:282
259950,0migration/23144-21kthreadcore23:03:162
259950,0migration/219712-21kthreadcore19:38:182
179950,0migration/18085-21unixbench-2d19:53:211
179950,0migration/17570-21kthreadcore21:23:171
179950,0migration/1705-21fschecks_time19:03:131
179950,0migration/16917-21hwlatdetect23:48:121
179950,0migration/14129-21munin-node20:18:151
179950,0migration/128145-21cpuspeed_turbos19:48:111
179950,0migration/125239-21munin-node22:18:101
179950,0migration/12416-21latency_hist21:02:591
179950,0migration/120261-21sh23:53:281
179950,0migration/12006-21irqcore22:03:161
179950,0migration/118165-21taskset21:12:091
179950,0migration/115942-21gltestperf23:13:111
169950,0watchdog/17125-21fschecks_count23:28:121
149950,0migration/09014-21aten_r1power_cu22:08:090
149950,0migration/08712-21taskset21:24:200
149950,0migration/08096-21kthreadcore00:08:150
149950,0migration/04768-21kthreadcore19:03:180
149950,0migration/04597-21kthreadcore19:53:150
149950,0migration/031978-21latency_hist22:42:580
149950,0migration/030210-21kthreadcore20:13:170
149950,0migration/030203-21cpuspeed_turbos00:03:100
149950,0migration/029853-21kthreadcore21:58:160
149950,0migration/027382-21taskset19:45:320
149950,0migration/027338-21kthreadcore18:58:150
149950,0migration/025277-21kthreadcore20:33:170
149950,0migration/024882-21munin-node22:58:140
149950,0migration/02077-21irqcore22:03:160
149950,0migration/020729-21taskset19:40:310
149950,0migration/018851-21sh20:53:100
149950,0migration/016690-21kthreadcore23:53:160
149950,0migration/016640-21kthreadcore23:13:140
149950,0migration/01531-21irqcore22:23:140
149950,0migration/015100-21kthreadcore00:13:170
149950,0migration/01318-21timerwakeupswit19:48:220
149950,0migration/01213-21sh21:42:410
149950,0migration/012016-21latency23:28:170
149950,0migration/011625-21kthreadcore23:08:150
149950,0migration/011290-21irqcore21:28:150
149950,0migration/010011-21irqcore19:58:140
123689955,0cyclictest0-21swapper/221:53:242
123689955,0cyclictest0-21swapper/221:35:162
123689955,0cyclictest0-21swapper/221:31:192
123689953,0cyclictest31406-21awk23:43:152
123689950,0cyclictest0-21swapper/220:57:452
123689950,0cyclictest0-21swapper/200:10:432
339940,0migration/330269-21kthreadcore20:58:163
339940,0migration/317892-21kthreadcore20:03:163
29725040,0irq/27-eth0-tx-870-21fschecks_count22:23:112
29725040,0irq/27-eth0-tx-7833-21munin-node22:48:212
29725040,0irq/27-eth0-tx-3104-21munin-node20:18:210
29725040,0irq/27-eth0-tx-26476-21cut23:18:142
29725040,0irq/27-eth0-tx-0-21swapper/223:33:452
29725040,0irq/27-eth0-tx-0-21swapper/223:13:042
29725040,0irq/27-eth0-tx-0-21swapper/223:08:352
29725040,0irq/27-eth0-tx-0-21swapper/222:53:142
29725040,0irq/27-eth0-tx-0-21swapper/222:37:582
29725040,0irq/27-eth0-tx-0-21swapper/222:18:192
29725040,0irq/27-eth0-tx-0-21swapper/221:23:032
29725040,0irq/27-eth0-tx-0-21swapper/220:50:002
29725040,0irq/27-eth0-tx-0-21swapper/020:23:200
29725040,0irq/27-eth0-tx-0-21swapper/020:08:120
29725040,0irq/27-eth0-tx-0-21swapper/019:28:140
29725040,0irq/27-eth0-tx-0-21swapper/019:18:180
29725040,0irq/27-eth0-tx-0-21swapper/019:13:180
29715040,0irq/26-eth0-rx-22791-21smartctl23:58:103
29715040,0irq/26-eth0-rx-0-21swapper/323:45:383
29715040,0irq/26-eth0-rx-0-21swapper/321:53:153
29715040,0irq/26-eth0-rx-0-21swapper/320:53:213
29715040,0irq/26-eth0-rx-0-21swapper/320:09:323
29715040,0irq/26-eth0-rx-0-21swapper/319:38:213
29715040,0irq/26-eth0-rx-0-21swapper/319:13:153
259940,0migration/24923-21hddtemp_smartct20:43:132
259940,0migration/24833-21kthreadcore19:28:162
259940,0migration/232505-21switchtime21:18:202
259940,0migration/229186-21kthreadcore19:48:162
259940,0migration/22590-21fschecks_count19:53:122
259940,0migration/224147-21kthreadcore23:58:132
259940,0migration/224070-21kthreadcore19:18:142
259940,0migration/215663-21unixbench_singl21:48:212
259940,0migration/214858-21kthreadcore18:48:172
249940,0watchdog/27123-21ssh22:46:402
249940,0watchdog/230826-21kthreadcore20:13:172
249940,0watchdog/224337-21kthreadcore18:53:172
249940,0watchdog/218960-21ntp_states19:13:172
249940,0watchdog/215477-21processes20:23:212
249940,0watchdog/20-21swapper/223:28:162
249940,0watchdog/20-21swapper/220:08:172
249940,0watchdog/20-21swapper/219:23:172
249940,0watchdog/20-21swapper/219:08:172
22814240,0sleep022820-21ssh22:56:090
179940,0migration/14955-21kthreadcore19:28:161
179940,0migration/131102-21kthreadcore19:23:171
179940,0migration/130626-21kthreadcore21:38:171
179940,0migration/130534-21kthreadcore21:18:151
179940,0migration/129890-21kthreadcore20:58:161
179940,0migration/123394-21kthreadcore20:53:161
179940,0migration/121653-21taskset20:04:341
179940,0migration/121477-21kthreadcore18:53:161
179940,0migration/120679-21fschecks_count19:18:101
179940,0migration/120576-21sshd19:38:531
179940,0migration/117583-21kthreadcore00:13:181
179940,0migration/116292-21fschecks_count22:53:121
179940,0migration/11576-21kthreadcore00:03:161
179940,0migration/113823-21cpuspeed_turbos19:13:101
179940,0migration/113284-21netstat19:33:191
179940,0migration/111614-21kthreadcore21:48:151
179940,0migration/111185-21kthreadcore22:08:151
179940,0migration/110161-21kthreadcore22:48:161
169940,0watchdog/16943-21ssh23:07:521
169940,0watchdog/1574-21munin-node22:43:121
169940,0watchdog/1290492sleep100:01:121
169940,0watchdog/119468-21kthreadcore21:33:171
149940,0migration/03636-21iostat21:03:180
149940,0migration/031413-21kthreadcore20:58:160
149940,0migration/022787-21sh23:58:100
149940,0migration/021809-21kthreadcore21:33:170
149940,0migration/020375-21kthreadcore22:33:180
149940,0migration/018550-21kthreadcore20:28:160
123689944,0cyclictest0-21swapper/219:57:592
123689943,0cyclictest4130-21awk20:18:152
123689943,0cyclictest3556-21lldpd20:33:182
123689943,0cyclictest3556-21lldpd20:03:482
123689943,0cyclictest3556-21lldpd19:03:482
123689943,0cyclictest3514-21snmpd23:53:112
123689943,0cyclictest3514-21snmpd22:13:262
123689943,0cyclictest3514-21snmpd20:28:552
123689943,0cyclictest3514-21snmpd19:45:552
123689943,0cyclictest23497-21awk23:38:132
123689943,0cyclictest14388-21awk00:13:152
123679943,0cyclictest5037-21awk20:43:141
123679943,0cyclictest3556-21lldpd22:31:491
123679943,0cyclictest3514-21snmpd23:08:111
123679943,0cyclictest3514-21snmpd22:13:111
123679943,0cyclictest3514-21snmpd21:54:561
123679943,0cyclictest3514-21snmpd21:43:561
123679943,0cyclictest3514-21snmpd18:58:401
123679943,0cyclictest3514-21snmpd18:48:251
123679943,0cyclictest3514-21snmpd00:08:111
123679943,0cyclictest3241-21irqbalance23:33:361
123679943,0cyclictest3241-21irqbalance22:39:171
123679943,0cyclictest3241-21irqbalance21:29:071
123679943,0cyclictest3241-21irqbalance20:38:171
123679943,0cyclictest3241-21irqbalance20:23:171
123679943,0cyclictest3241-21irqbalance20:13:171
123679943,0cyclictest3241-21irqbalance19:08:281
123679943,0cyclictest31959-21awk23:23:131
123679943,0cyclictest31481-21awk23:43:151
123679943,0cyclictest24568-21awk22:58:131
123679943,0cyclictest24028-21awk23:18:121
123679943,0cyclictest24014-21awk20:33:151
123679943,0cyclictest23523-21awk23:38:141
123679943,0cyclictest23173-21awk20:08:161
123679943,0cyclictest19725-21awk21:13:151
123679943,0cyclictest17405-21awk20:28:141
123679943,0cyclictest1379-21awk22:23:131
123679943,0cyclictest11827-21awk20:48:141
123669944,0cyclictest0-21swapper/023:18:180
123669944,0cyclictest0-21swapper/020:38:160
123669943,0cyclictest9334-21awk22:28:130
123669943,0cyclictest6968-21awk23:48:120
123669943,0cyclictest3556-21lldpd20:05:480
123669943,0cyclictest3556-21lldpd19:34:180
123669943,0cyclictest3241-21irqbalance23:25:560
123669943,0cyclictest3241-21irqbalance22:20:170
123669943,0cyclictest3241-21irqbalance21:44:470
123669943,0cyclictest31457-21awk23:43:150
123669943,0cyclictest26843-21munin-node19:23:130
123669943,0cyclictest23413-21awk23:38:120
123669943,0cyclictest17903-21awk22:13:130
35130,0ktimersoftd/30-21swapper/319:23:233
29715030,0irq/26-eth0-rx-0-21swapper/320:47:263
29715030,0irq/26-eth0-rx-0-21swapper/320:33:123
29715030,0irq/26-eth0-rx-0-21swapper/320:23:243
29715030,0irq/26-eth0-rx-0-21swapper/320:14:113
29715030,0irq/26-eth0-rx-0-21swapper/319:59:043
29715030,0irq/26-eth0-rx-0-21swapper/319:43:113
29715030,0irq/26-eth0-rx-0-21swapper/319:33:123
29715030,0irq/26-eth0-rx-0-21swapper/319:07:523
29715030,0irq/26-eth0-rx-0-21swapper/319:01:373
29715030,0irq/26-eth0-rx-0-21swapper/318:48:113
169930,0watchdog/10-21swapper/122:33:161
169930,0watchdog/10-21swapper/121:58:571
169930,0watchdog/10-21swapper/119:58:171
123689932,0cyclictest26741-21awk18:58:142
123679932,0cyclictest26844-21sensors19:43:211
123669933,0cyclictest18875-21kworker/0:123:33:150
123669933,0cyclictest18875-21kworker/0:123:03:000
123669933,0cyclictest18875-21kworker/0:122:48:100
123669933,0cyclictest18875-21kworker/0:121:48:100
123669933,0cyclictest18875-21kworker/0:121:18:000
123669933,0cyclictest18875-21kworker/0:121:08:060
123669933,0cyclictest18875-21kworker/0:120:43:020
123669933,0cyclictest18875-21kworker/0:119:08:020
123669933,0cyclictest18474-21awk21:53:130
123669931,0cyclictest22262-21tr21:13:170
123689921,0cyclictest0-21swapper/220:38:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional