You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-13 - 03:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Mon Jul 13, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
194824948,0sleep00-21swapper/018:47:270
184423725,0sleep20-21swapper/218:46:132
304450350,0irq/25-eth10-21swapper/318:45:403
2808023524,0sleep10-21swapper/118:45:001
339960,0migration/332431-21runrttasks23:49:563
339960,0migration/331300-21sh23:06:103
339960,0migration/323873-21sh23:43:303
339960,0migration/323409-21sh22:21:153
339960,0migration/323118-21sh00:01:343
339960,0migration/323118-21sh00:01:343
339960,0migration/323113-21kthreadcore20:40:143
339960,0migration/315310-21sh21:35:273
29715060,0irq/26-eth0-rx-7839-21ssh21:32:073
29715060,0irq/26-eth0-rx-6124-21sa120:29:593
22319960,0cyclictest24351-21ssh21:43:151
149960,0migration/07960-21sh23:54:140
149960,0migration/018843-21timerwakeupswit20:35:240
149960,0migration/018011-21idleruntime21:40:120
149960,0migration/015490-21sh21:35:570
8215050,0irq/24-0000:00:17008-21munin-plugin-st21:19:592
4150,0ktimersoftd/00-21swapper/022:57:280
339950,0migration/39518-21kthreadcore19:40:153
339950,0migration/39486-21fschecks_time22:15:133
339950,0migration/39256-21fschecks_count22:55:123
339950,0migration/36457-21unixbench-2d23:50:233
339950,0migration/33803-21taskset19:36:143
339950,0migration/33425-21aten_r1power_cu19:15:103
339950,0migration/330858-21timerandwakeup22:45:213
339950,0migration/330140-21kthreadcore21:45:163
339950,0migration/329242-21kthreadcore21:05:173
339950,0migration/32822-21kthreadcore21:50:173
339950,0migration/327646-21kthreadcore23:25:153
339950,0migration/327030-21kthreadcore00:05:153
339950,0migration/325037-21latency_hist22:04:593
339950,0migration/323604-21timerandwakeup19:50:243
339950,0migration/32275-21kthreadcore00:10:163
339950,0migration/319733-21kthreadcore23:20:173
339950,0migration/318282-21munin-node21:20:153
339950,0migration/316856-21runrttasks21:00:043
339950,0migration/316846-21taskset21:19:343
339950,0migration/31524-21gltestperf22:10:143
339950,0migration/314992-21taskset20:55:523
339950,0migration/313450-21munin-node20:35:133
339950,0migration/311269-21kthreadcore00:15:163
339950,0migration/310844-21kthreadcore22:35:153
339950,0migration/310350-21kthreadcore23:55:153
29725050,0irq/27-eth0-tx-23428-21munin-node19:30:210
29725050,0irq/27-eth0-tx-0-21swapper/020:20:100
29725050,0irq/27-eth0-tx-0-21swapper/019:29:350
29715050,0irq/26-eth0-rx-6357-21ps22:50:213
29715050,0irq/26-eth0-rx-22785-21pmu-power21:40:193
29715050,0irq/26-eth0-rx-0-21swapper/323:36:273
29715050,0irq/26-eth0-rx-0-21swapper/323:16:053
29715050,0irq/26-eth0-rx-0-21swapper/323:00:493
29715050,0irq/26-eth0-rx-0-21swapper/322:40:223
29715050,0irq/26-eth0-rx-0-21swapper/322:00:173
29715050,0irq/26-eth0-rx-0-21swapper/321:59:113
29715050,0irq/26-eth0-rx-0-21swapper/321:10:163
29715050,0irq/26-eth0-rx-0-21swapper/319:20:163
29715050,0irq/26-eth0-rx-0-21swapper/318:50:123
249950,0watchdog/220860-21tr22:00:172
22329954,0cyclictest6403-21awk20:05:142
22329954,0cyclictest0-21swapper/222:08:172
22329953,0cyclictest3241-21irqbalance21:36:272
22319955,0cyclictest0-21swapper/123:59:191
22319955,0cyclictest0-21swapper/123:07:551
22319955,0cyclictest0-21swapper/122:48:121
22319955,0cyclictest0-21swapper/122:25:301
22319955,0cyclictest0-21swapper/122:22:401
22319955,0cyclictest0-21swapper/121:09:311
22319950,0cyclictest301-21diskmemload23:22:441
22319950,0cyclictest0-21swapper/123:53:191
22319950,0cyclictest0-21swapper/123:29:021
22319950,0cyclictest0-21swapper/122:57:211
22319950,0cyclictest0-21swapper/121:26:041
22309955,0cyclictest0-21swapper/023:38:180
22309950,0cyclictest0-21swapper/023:03:090
22309950,0cyclictest0-21swapper/021:25:520
22309950,0cyclictest0-21swapper/020:31:100
19150,0ktimersoftd/10-21swapper/122:04:131
179950,0migration/16798-21sh23:10:321
179950,0migration/125541-21munin-node23:45:131
179950,0migration/122837-21sh21:20:191
179950,0migration/121876-21runrttasks19:00:551
179950,0migration/117795-21sh19:25:141
179950,0migration/114322-21pmu-power23:15:201
169950,0watchdog/11620-21cat00:10:161
169950,0watchdog/114259-21grep21:35:171
149950,0migration/08205-21sh20:54:370
149950,0migration/0803-21taskset22:50:120
149950,0migration/06208-21latency22:10:180
149950,0migration/031297-21unixbench_multi21:45:220
149950,0migration/030588-21munin-node23:25:210
149950,0migration/030419-21kernelversion19:10:150
149950,0migration/028375-21kthreadcore23:05:160
149950,0migration/027899-21munin-node19:05:190
149950,0migration/026738-21kthreadcore22:05:140
149950,0migration/022772-21kthreadcore20:15:170
149950,0migration/02272-21kthreadcore22:30:150
149950,0migration/022689-21kthreadcore20:40:130
149950,0migration/022509-21kthreadcore19:50:180
149950,0migration/02186-21kthreadcore23:30:150
149950,0migration/021173-21kthreadcore21:20:170
149950,0migration/020737-21kthreadcore00:00:170
149950,0migration/020737-21kthreadcore00:00:170
149950,0migration/019650-21kthreadcore23:40:180
149950,0migration/014412-21munin-node20:55:200
149950,0migration/014088-21kthreadcore19:45:180
149950,0migration/014009-21munin-node23:55:180
149950,0migration/01362-21idleruntime23:10:130
149950,0migration/013410-21kthreadcore21:55:140
149950,0migration/012532-21fschecks_count20:10:130
149950,0migration/011288-21kthreadcore22:15:160
149950,0migration/010801-21needreboot20:05:170
8215040,0irq/24-0000:00:15592-21perf18:59:592
8215040,0irq/24-0000:00:12816-21latency_hist20:34:592
339940,0migration/3353-21kthreadcore19:10:163
339940,0migration/330559-21kthreadcore20:20:183
339940,0migration/328246-21kthreadcore21:25:163
339940,0migration/325375-21kthreadcore19:05:173
339940,0migration/323297-21kthreadcore20:15:173
339940,0migration/313605-21kthreadcore20:10:163
29725040,0irq/27-eth0-tx-31276-21ssh22:25:472
29725040,0irq/27-eth0-tx-301-21diskmemload22:59:592
29725040,0irq/27-eth0-tx-16646-21munin-node23:40:142
29725040,0irq/27-eth0-tx-15138-21ssh22:36:022
29725040,0irq/27-eth0-tx-0-21swapper/223:25:322
29725040,0irq/27-eth0-tx-0-21swapper/222:30:062
29725040,0irq/27-eth0-tx-0-21swapper/221:55:232
29725040,0irq/27-eth0-tx-0-21swapper/221:12:202
29725040,0irq/27-eth0-tx-0-21swapper/221:02:002
29725040,0irq/27-eth0-tx-0-21swapper/220:56:292
29725040,0irq/27-eth0-tx-0-21swapper/220:54:302
29725040,0irq/27-eth0-tx-0-21swapper/200:01:032
29725040,0irq/27-eth0-tx-0-21swapper/200:01:032
29725040,0irq/27-eth0-tx-0-21swapper/020:00:180
29725040,0irq/27-eth0-tx-0-21swapper/019:55:240
29715040,0irq/26-eth0-rx-0-21swapper/323:30:153
29715040,0irq/26-eth0-rx-0-21swapper/323:10:103
29715040,0irq/26-eth0-rx-0-21swapper/322:30:573
29715040,0irq/26-eth0-rx-0-21swapper/322:25:123
29715040,0irq/26-eth0-rx-0-21swapper/320:50:173
29715040,0irq/26-eth0-rx-0-21swapper/319:55:483
29715040,0irq/26-eth0-rx-0-21swapper/319:25:153
259940,0migration/222395-21ntp_kernel_err22:40:182
249940,0watchdog/29501-21perl22:15:132
249940,0watchdog/25601-21ps18:50:182
249940,0watchdog/2351-21munin-node23:30:162
249940,0watchdog/227822-21kthreadcore23:45:162
249940,0watchdog/219565-21kthreadcore19:50:172
249940,0watchdog/216520-21perl19:00:132
249940,0watchdog/213005-21kthreadcore00:15:162
249940,0watchdog/20-21swapper/223:50:162
249940,0watchdog/20-21swapper/222:21:092
249940,0watchdog/20-21swapper/222:10:172
249940,0watchdog/20-21swapper/219:35:172
22329943,0cyclictest5323-21awk19:40:122
22329943,0cyclictest425-21awk20:25:152
22329943,0cyclictest3514-21snmpd23:22:072
22329943,0cyclictest3514-21snmpd23:15:072
22329943,0cyclictest3514-21snmpd21:07:222
22329943,0cyclictest3514-21snmpd20:45:072
22329943,0cyclictest3514-21snmpd20:40:372
22329943,0cyclictest3514-21snmpd20:30:072
22329943,0cyclictest3514-21snmpd20:21:222
22329943,0cyclictest3514-21snmpd19:58:522
22329943,0cyclictest3514-21snmpd19:30:072
22329943,0cyclictest3514-21snmpd19:23:522
22329943,0cyclictest3514-21snmpd19:15:372
22329943,0cyclictest3514-21snmpd19:11:072
22329943,0cyclictest3241-21irqbalance23:35:072
22329943,0cyclictest3241-21irqbalance23:11:272
22329943,0cyclictest3241-21irqbalance22:55:072
22329943,0cyclictest3241-21irqbalance22:50:072
22329943,0cyclictest3241-21irqbalance22:45:072
22329943,0cyclictest3241-21irqbalance21:30:072
22329943,0cyclictest3241-21irqbalance21:25:572
22329943,0cyclictest3241-21irqbalance21:20:072
22329943,0cyclictest3241-21irqbalance00:14:462
22329943,0cyclictest26259-21awk21:45:142
22329943,0cyclictest25844-21awk23:05:142
22329943,0cyclictest25592-21awk00:05:142
22329943,0cyclictest1809-21awk21:50:152
22329943,0cyclictest18046-21awk21:40:122
22329943,0cyclictest17728-21awk19:25:142
22329943,0cyclictest13010-21awk20:10:152
22329941,0cyclictest0-21swapper/223:55:102
22319944,0cyclictest0-21swapper/122:18:251
22319943,0cyclictest7267-21awk20:30:151
22319943,0cyclictest3514-21snmpd20:00:371
22319943,0cyclictest3514-21snmpd19:06:221
22319943,0cyclictest3514-21snmpd00:05:371
22319943,0cyclictest3241-21irqbalance21:15:371
22319943,0cyclictest17811-21awk23:00:141
22319943,0cyclictest17794-21awk21:00:131
22319943,0cyclictest12947-21awk20:10:141
22309944,0cyclictest0-21swapper/022:03:420
22309943,0cyclictest9678-21awk22:35:130
22309943,0cyclictest3514-21snmpd19:02:220
22309943,0cyclictest3514-21snmpd18:55:520
22309943,0cyclictest27313-21awk20:45:150
22309943,0cyclictest26041-21awk21:05:150
22309943,0cyclictest1945-21awk21:30:140
179940,0migration/17386-21kthreadcore19:40:141
179940,0migration/13505-21kthreadcore22:30:151
179940,0migration/13103-21kthreadcore21:30:161
179940,0migration/1302-21sh23:30:001
179940,0migration/126907-21kthreadcore22:05:141
179940,0migration/123937-21fschecks_count19:30:131
179940,0migration/116966-21aten_r1power_po22:40:101
179940,0migration/112523-21kthreadcore00:15:161
179940,0migration/111760-21ksmtuned20:09:081
179940,0migration/110916-21kthreadcore21:55:131
179940,0migration/110602-21kthreadcore22:35:151
169940,0watchdog/19212-21ssh23:35:121
169940,0watchdog/16592-21port_sshpriv21:50:211
169940,0watchdog/14716-21kthreadcore20:50:171
169940,0watchdog/132709-21ssh21:49:091
169940,0watchdog/131613-21tr19:35:171
169940,0watchdog/125085-21perl19:55:131
169940,0watchdog/124414-21python20:15:211
169940,0watchdog/123833-21ssh00:03:361
169940,0watchdog/123833-21ssh00:03:361
169940,0watchdog/118174-21diskstats19:50:131
169940,0watchdog/116002-21ssh20:58:291
169940,0watchdog/111742sleep122:10:131
169940,0watchdog/10-21swapper/122:50:171
169940,0watchdog/10-21swapper/120:40:131
169940,0watchdog/10-21swapper/120:20:171
149940,0migration/07304-21diskmemload00:12:330
149940,0migration/05965-21kthreadcore19:40:140
149940,0migration/03520-21munin-node19:15:110
149940,0migration/032049-21runrttasks22:28:010
149940,0migration/026937-21kthreadcore23:45:150
149940,0migration/016774-21sh22:39:590
149940,0migration/013669-21kthreadcore19:20:170
339930,0migration/35806-21diskstats20:05:113
339930,0migration/326579-21runrttasks20:45:123
29715030,0irq/26-eth0-rx-0-21swapper/320:31:543
29715030,0irq/26-eth0-rx-0-21swapper/319:48:113
29715030,0irq/26-eth0-rx-0-21swapper/319:30:133
29715030,0irq/26-eth0-rx-0-21swapper/319:01:363
29715030,0irq/26-eth0-rx-0-21swapper/318:55:113
249930,0watchdog/20-21swapper/220:15:132
249930,0watchdog/20-21swapper/219:45:172
22329932,0cyclictest3514-21snmpd19:07:372
22329932,0cyclictest32009-21awk20:00:142
22319933,0cyclictest3556-21lldpd20:25:041
22319933,0cyclictest0-21swapper/119:15:221
22319932,0cyclictest6079-21cut18:50:181
22319932,0cyclictest29195-21basename19:10:101
22319932,0cyclictest17612-21grep20:35:181
22319932,0cyclictest17516-21awk23:40:141
22319932,0cyclictest11055-21awk19:20:151
22309933,0cyclictest9446-21kworker/0:123:20:080
22309933,0cyclictest9446-21kworker/0:123:15:220
22309933,0cyclictest9446-21kworker/0:100:15:010
22309933,0cyclictest31333-21kworker/0:221:15:020
22309933,0cyclictest31333-21kworker/0:221:10:000
22309933,0cyclictest31333-21kworker/0:221:00:020
22309933,0cyclictest31333-21kworker/0:220:25:020
22309933,0cyclictest31333-21kworker/0:219:35:050
22309933,0cyclictest31333-21kworker/0:218:50:050
22309933,0cyclictest14935-21kworker/0:022:45:020
22309933,0cyclictest14935-21kworker/0:022:20:220
22309932,0cyclictest26912-21kthreadcore00:05:150
22309932,0cyclictest1055-21egrep21:50:120
169930,0watchdog/10-21swapper/121:11:051
169930,0watchdog/10-21swapper/120:45:171
169930,0watchdog/10-21swapper/119:45:171
169930,0watchdog/10-21swapper/118:55:211
35120,0ktimersoftd/331108-21aten_r1power_cu20:00:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional