You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-14 - 05:24
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Fri Nov 14, 2025 00:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
213532195179,11sleep10-21swapper/119:08:521
211922193176,12sleep30-21swapper/319:06:493
211772185169,12sleep00-21swapper/019:06:370
211022171155,12sleep20-21swapper/219:05:402
104902520,0sleep00-21swapper/023:15:420
281642470,0sleep00-21swapper/022:35:440
308972390,0sleep2301ktimersoftd/222:54:062
169072100,0sleep10-21swapper/121:41:231
24435280,0sleep10-21swapper/121:29:351
216059981,1cyclictest3-21ksoftirqd/022:55:160
216119970,6cyclictest5810-21perl19:40:162
216059973,1cyclictest3-21ksoftirqd/023:14:330
216059973,1cyclictest3-21ksoftirqd/021:33:210
216059972,1cyclictest3-21ksoftirqd/022:32:220
216059972,1cyclictest3-21ksoftirqd/022:27:210
216059972,1cyclictest3-21ksoftirqd/021:13:180
216059972,1cyclictest3-21ksoftirqd/000:03:200
216059971,3cyclictest27240-21sendmail-msp19:20:000
216059971,1cyclictest3-21ksoftirqd/023:40:210
216059971,1cyclictest3-21ksoftirqd/023:23:550
216059971,1cyclictest0-21swapper/023:25:170
216059971,1cyclictest0-21swapper/022:20:140
216059971,1cyclictest0-21swapper/021:40:240
216059971,1cyclictest0-21swapper/021:05:120
216059970,1cyclictest3-21ksoftirqd/023:50:010
216059970,1cyclictest17228-21diskmemload00:13:350
216059970,1cyclictest0-21swapper/022:40:220
216059970,1cyclictest0-21swapper/020:45:150
216169964,2cyclictest16753-21seq23:19:433
216169960,5cyclictest14995-21/usr/sbin/munin21:05:193
216119960,6cyclictest0-21swapper/222:47:192
216119960,4cyclictest0-21swapper/200:15:112
216059963,1cyclictest3-21ksoftirqd/000:20:200
216059962,1cyclictest3-21ksoftirqd/022:50:180
216059962,1cyclictest3-21ksoftirqd/021:45:450
216059962,1cyclictest3-21ksoftirqd/000:25:200
216059961,1cyclictest3-21ksoftirqd/023:58:230
216059961,1cyclictest3-21ksoftirqd/023:46:320
216059961,1cyclictest3-21ksoftirqd/023:35:180
216059961,1cyclictest3-21ksoftirqd/023:30:150
216059961,1cyclictest3-21ksoftirqd/023:07:200
216059961,1cyclictest3-21ksoftirqd/023:00:260
216059961,1cyclictest3-21ksoftirqd/022:45:100
216059961,1cyclictest3-21ksoftirqd/022:05:500
216059961,1cyclictest3-21ksoftirqd/022:00:140
216059961,1cyclictest3-21ksoftirqd/021:55:520
216059961,1cyclictest3-21ksoftirqd/021:50:170
216059961,1cyclictest3-21ksoftirqd/021:35:030
216059961,1cyclictest3-21ksoftirqd/021:20:390
216059961,1cyclictest3-21ksoftirqd/021:15:040
216059961,1cyclictest3-21ksoftirqd/020:20:120
216059961,1cyclictest3-21ksoftirqd/019:50:140
216059961,1cyclictest17228-21diskmemload22:15:320
216059961,1cyclictest0-21swapper/021:25:110
216059961,0cyclictest0-21swapper/020:35:240
216059960,2cyclictest0-21swapper/000:35:120
216059960,1cyclictest27970-21strings00:30:200
216059960,1cyclictest0-21swapper/022:11:440
216059960,1cyclictest0-21swapper/020:50:140
216059960,1cyclictest0-21swapper/020:15:150
216059960,1cyclictest0-21swapper/019:45:220
216059960,1cyclictest0-21swapper/019:40:120
216059960,1cyclictest0-21swapper/019:10:120
216059960,1cyclictest0-21swapper/000:07:110
216059960,0cyclictest81rcu_preempt19:30:170
1825260,0sleep30-21swapper/319:30:233
216169955,0cyclictest3510-21kworker/3:121:47:243
216169955,0cyclictest30386-21kworker/3:223:28:083
216169955,0cyclictest30386-21kworker/3:222:32:093
216169955,0cyclictest0-21swapper/322:37:233
216169953,1cyclictest15647-21ssh21:25:113
216169950,5cyclictest0-21swapper/321:57:303
216169950,5cyclictest0-21swapper/321:36:173
216169950,5cyclictest0-21swapper/320:10:333
216169950,4cyclictest28733-21perl22:20:143
216169950,4cyclictest0-21swapper/322:25:113
216119955,0cyclictest0-21swapper/221:55:402
216119955,0cyclictest0-21swapper/200:39:392
216119952,3cyclictest714-21snmpd23:35:442
216119952,3cyclictest16523-21ssh23:02:582
216119951,2cyclictest0-21swapper/220:10:162
216119950,5cyclictest0-21swapper/223:54:582
216119950,5cyclictest0-21swapper/222:07:262
216119950,5cyclictest0-21swapper/221:54:572
216119950,5cyclictest0-21swapper/221:27:532
216119950,4cyclictest0-21swapper/222:35:142
216119950,4cyclictest0-21swapper/221:40:142
216099951,3cyclictest0-21swapper/100:15:111
216099951,2cyclictest0-21swapper/122:20:131
216099951,2cyclictest0-21swapper/121:55:121
216099951,2cyclictest0-21swapper/120:50:131
216059951,2cyclictest0-21swapper/020:10:160
216059951,1cyclictest3-21ksoftirqd/020:30:510
216059951,1cyclictest17228-21diskmemload00:15:020
216059951,0cyclictest0-21swapper/019:20:180
216059950,2cyclictest3-21ksoftirqd/019:35:000
216059950,1cyclictest3-21ksoftirqd/021:00:200
216059950,1cyclictest0-21swapper/020:00:110
216059950,1cyclictest0-21swapper/019:55:140
18000250,0sleep340-21ksoftirqd/300:09:233
931240,0sleep10-21swapper/100:00:151
216169944,0cyclictest3510-21kworker/3:120:29:003
216169944,0cyclictest3510-21kworker/3:120:17:483
216169944,0cyclictest31870-21ssh00:15:353
216169944,0cyclictest0-21swapper/322:18:553
216169942,1cyclictest9835-21cron19:50:013
216169941,1cyclictest0-21swapper/320:50:143
216169941,1cyclictest0-21swapper/320:40:133
216169940,4cyclictest5421-21ssh23:30:153
216169940,4cyclictest14464-21memory21:40:203
216169940,4cyclictest0-21swapper/323:45:023
216169940,4cyclictest0-21swapper/323:35:143
216169940,4cyclictest0-21swapper/323:05:133
216169940,4cyclictest0-21swapper/323:00:123
216169940,4cyclictest0-21swapper/322:50:143
216169940,4cyclictest0-21swapper/322:45:153
216169940,4cyclictest0-21swapper/321:15:203
216169940,4cyclictest0-21swapper/320:20:123
216169940,4cyclictest0-21swapper/319:55:123
216169940,4cyclictest0-21swapper/300:30:143
216169940,4cyclictest0-21swapper/300:25:163
216169940,4cyclictest0-21swapper/300:00:123
216169940,3cyclictest8879-21ntp_states19:45:173
216169940,3cyclictest8306-21/usr/sbin/munin00:20:203
216169940,3cyclictest765-21/usr/sbin/munin21:50:133
216169940,3cyclictest714-21snmpd22:05:203
216169940,3cyclictest31196-21ntp_states19:25:183
216169940,3cyclictest24263-21df19:15:003
216169940,3cyclictest22894-21ls23:55:153
216169940,3cyclictest20471-21sh22:00:073
216169940,3cyclictest20298-21sh23:20:383
216169940,3cyclictest17559-21cat21:10:013
216169940,3cyclictest15296-21cat21:25:013
216169940,3cyclictest13571-21if_eth423:50:173
216169940,1cyclictest4318-21grep00:35:143
216169940,1cyclictest19582-21/usr/sbin/munin00:10:213
216119944,0cyclictest31-21ksoftirqd/200:00:182
216119944,0cyclictest0-21swapper/223:21:302
216119944,0cyclictest0-21swapper/222:28:062
216119942,2cyclictest714-21snmpd19:58:312
216119942,2cyclictest1343-21kworker/2:023:26:482
216119942,2cyclictest1343-21kworker/2:021:00:532
216119942,1cyclictest24577-21sshd20:18:412
216119941,3cyclictest11593-21ssh22:11:182
216119941,2cyclictest0-21swapper/221:10:132
216119940,4cyclictest291rcuc/219:35:012
216119940,4cyclictest28206-21ssh23:57:472
216119940,4cyclictest0-21swapper/223:45:132
216119940,4cyclictest0-21swapper/223:40:272
216119940,4cyclictest0-21swapper/223:07:162
216119940,4cyclictest0-21swapper/222:42:212
216119940,4cyclictest0-21swapper/222:30:122
216119940,4cyclictest0-21swapper/221:45:122
216119940,4cyclictest0-21swapper/221:37:552
216119940,4cyclictest0-21swapper/221:30:142
216119940,4cyclictest0-21swapper/221:20:132
216119940,4cyclictest0-21swapper/221:15:122
216119940,4cyclictest0-21swapper/220:00:112
216119940,4cyclictest0-21swapper/219:25:112
216119940,3cyclictest6851-21ntp_states23:30:212
216119940,3cyclictest6688-21ssh00:20:052
216119940,3cyclictest32600-21chrt22:55:032
216119940,3cyclictest31964-21ssh22:21:352
216119940,3cyclictest31785-21ssh23:10:192
216119940,3cyclictest26362-21ssh00:30:042
216119940,3cyclictest24269-21egrep19:15:012
216119940,3cyclictest20606-21/usr/sbin/munin22:00:172
216119940,3cyclictest19918-21cat20:10:012
216119940,3cyclictest18980-21/usr/sbin/munin22:15:152
216119940,3cyclictest11983-21ssh23:16:412
216119940,3cyclictest0-21swapper/221:05:132
216119940,3cyclictest0-21swapper/200:12:042
216119940,1cyclictest0-21swapper/220:20:152
216099944,0cyclictest0-21swapper/123:23:511
216099944,0cyclictest0-21swapper/122:51:421
216099944,0cyclictest0-21swapper/122:36:071
216099944,0cyclictest0-21swapper/122:13:511
216099944,0cyclictest0-21swapper/100:28:101
216099944,0cyclictest0-21swapper/100:22:561
216099942,1cyclictest0-21swapper/121:45:111
216099942,1cyclictest0-21swapper/121:00:141
216099941,2cyclictest0-21swapper/123:40:151
216099941,2cyclictest0-21swapper/122:40:121
216099941,2cyclictest0-21swapper/119:55:111
216099941,2cyclictest0-21swapper/119:20:141
216099941,1cyclictest0-21swapper/119:40:131
216099940,4cyclictest0-21swapper/121:20:541
216099940,2cyclictest0-21swapper/123:00:111
216099940,0cyclictest0-21swapper/123:50:461
216059941,1cyclictest0-21swapper/020:25:120
216059941,1cyclictest0-21swapper/019:25:120
216059940,3cyclictest0-21swapper/020:55:110
216059940,1cyclictest3-21ksoftirqd/020:40:140
216059940,1cyclictest3-21ksoftirqd/020:05:150
216169932,1cyclictest714-21snmpd19:35:103
216169932,1cyclictest24885-21ssh23:40:013
216169931,1cyclictest25705-21apt-get21:30:143
216169930,3cyclictest5143-21apt-get19:40:123
216169930,3cyclictest418-21sh20:35:163
216169930,3cyclictest23113-21ps19:10:183
216169930,3cyclictest16026-21irqstats20:00:163
216169930,3cyclictest0-21swapper/322:55:043
216169930,3cyclictest0-21swapper/322:40:153
216169930,3cyclictest0-21swapper/321:00:163
216169930,3cyclictest0-21swapper/320:55:133
216169930,3cyclictest0-21swapper/320:45:143
216169930,3cyclictest0-21swapper/320:30:143
216169930,3cyclictest0-21swapper/320:05:143
216169930,3cyclictest0-21swapper/319:20:133
216169930,2cyclictest0-21swapper/323:10:013
216169930,1cyclictest7917-21ssh22:10:083
216119933,0cyclictest1343-21kworker/2:020:30:092
216119933,0cyclictest1343-21kworker/2:019:20:122
216119932,1cyclictest7929-21tr19:45:122
216119931,1cyclictest4903-21apt-get20:45:142
216119930,3cyclictest469-21diskstats19:30:142
216119930,3cyclictest0-21swapper/220:55:122
216119930,3cyclictest0-21swapper/220:50:102
216119930,3cyclictest0-21swapper/220:40:142
216119930,3cyclictest0-21swapper/220:35:162
216119930,3cyclictest0-21swapper/220:25:122
216119930,3cyclictest0-21swapper/219:50:132
216119930,3cyclictest0-21swapper/219:10:132
216119930,3cyclictest0-21swapper/200:25:062
216119930,3cyclictest0-21swapper/200:05:022
216099933,0cyclictest0-21swapper/123:35:031
216099931,1cyclictest195452chrt00:10:131
216099931,1cyclictest0-21swapper/123:55:131
216099931,1cyclictest0-21swapper/123:45:131
216099931,1cyclictest0-21swapper/123:25:151
216099931,1cyclictest0-21swapper/123:15:131
216099931,1cyclictest0-21swapper/123:10:011
216099931,1cyclictest0-21swapper/123:05:121
216099931,1cyclictest0-21swapper/122:05:131
216099931,1cyclictest0-21swapper/121:10:141
216099931,1cyclictest0-21swapper/121:05:131
216099931,1cyclictest0-21swapper/120:40:141
216099931,1cyclictest0-21swapper/120:35:161
216099930,2cyclictest777-21taskset00:33:141
216099930,2cyclictest199382sleep122:31:491
216099930,2cyclictest0-21swapper/122:25:121
216099930,2cyclictest0-21swapper/121:35:111
216099930,2cyclictest0-21swapper/121:15:121
216099930,2cyclictest0-21swapper/120:45:151
216099930,2cyclictest0-21swapper/120:20:111
216099930,2cyclictest0-21swapper/119:25:121
216099921,1cyclictest0-21swapper/122:15:111
216099921,1cyclictest0-21swapper/122:00:141
216099921,1cyclictest0-21swapper/121:50:141
216099921,1cyclictest0-21swapper/120:55:111
216099921,1cyclictest0-21swapper/120:25:111
216099921,1cyclictest0-21swapper/120:15:151
216099921,1cyclictest0-21swapper/120:05:151
216099921,1cyclictest0-21swapper/119:50:141
216099921,1cyclictest0-21swapper/119:45:121
216099921,1cyclictest0-21swapper/119:35:011
216099921,1cyclictest0-21swapper/119:15:141
216099921,1cyclictest0-21swapper/119:10:131
216099920,1cyclictest5174-21chrt23:30:051
216099920,1cyclictest4050-21chrt00:35:101
216099920,1cyclictest254062sleep121:30:031
216099920,1cyclictest2517-21chrt22:55:231
216099920,1cyclictest15041-21chrt20:00:101
216099920,1cyclictest12998-21chrt22:45:141
216099920,1cyclictest0-21swapper/120:30:131
216099920,1cyclictest0-21swapper/120:10:161
216099920,1cyclictest0-21swapper/119:30:121
216099920,1cyclictest0-21swapper/100:05:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional