You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 08:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Fri Nov 28, 2025 00:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
246912193176,12sleep00-21swapper/019:06:540
224752191174,12sleep30-21swapper/319:05:013
248212189172,12sleep10-21swapper/119:08:321
227092172153,14sleep20-21swapper/219:05:052
62992440,0sleep00-21swapper/000:00:270
69052420,0sleep30-21swapper/322:23:463
258062420,0sleep10-21swapper/123:54:521
241352400,0sleep00-21swapper/000:10:120
158602120,0sleep30-21swapper/322:44:323
2513799111,1cyclictest4396-21sort22:55:002
2513799110,7cyclictest30445-21cron19:17:012
2513799100,1cyclictest15391-21ssh23:16:122
2513799100,1cyclictest1418-21chrt20:28:382
251379992,0cyclictest9137-21ssh00:02:172
251379991,5cyclictest5744-21taskset19:32:592
251379990,1cyclictest9234-21ssh22:08:362
251379990,1cyclictest30522-21rm23:56:332
251379990,1cyclictest27589-21df_inode21:45:142
251259992,3cyclictest11457-21munin-run20:50:010
27819280,0sleep20-21swapper/222:34:182
251379982,5cyclictest27417-21ssh23:39:202
251379981,1cyclictest9409-21apt-get22:25:122
251379980,7cyclictest14909-21wc19:50:192
251379980,7cyclictest12519-21sh20:50:162
251379980,1cyclictest9872-21latency19:40:162
251379980,1cyclictest8859-21cron22:25:002
251379980,1cyclictest4980-21ssh21:17:312
251379980,1cyclictest471-21ssh22:36:062
251379980,1cyclictest30543-21ssh00:29:512
251379980,1cyclictest26463-21munin-run21:45:002
251379980,1cyclictest2382-21cpuspeed23:10:142
251379980,1cyclictest23154-21ssh00:09:482
251379980,1cyclictest20229-21ssh22:14:172
251379980,1cyclictest16709-21ssh23:50:142
251379980,1cyclictest1591-21munin-run23:10:002
251379980,1cyclictest14557-21ssh23:49:002
251379980,1cyclictest12700-21ssh21:54:102
251379980,1cyclictest0-21swapper/223:20:122
251379980,0cyclictest8227-21ssh00:34:592
251379980,0cyclictest11663-21ssh00:35:502
251259982,3cyclictest11489-21basename23:15:000
251379971,1cyclictest8063-21cut20:40:182
251379971,1cyclictest714-21snmpd21:09:512
251379971,1cyclictest27255-21cat20:15:162
251379971,1cyclictest17384-21cut21:00:182
251379971,1cyclictest12940-21wc19:45:182
251379970,1cyclictest6645-21wc19:35:132
251379970,1cyclictest3453-21apt19:30:002
251379970,1cyclictest32683-21sed19:20:232
251379970,1cyclictest31827-21ssh23:25:122
251379970,1cyclictest31295-21munin-plugin-st20:25:002
251379970,1cyclictest30057-21diskstats21:30:152
251379970,1cyclictest29990-21perl23:40:152
251379970,1cyclictest28434-21awk22:50:172
251379970,1cyclictest2837-21missed_timers00:15:182
251379970,1cyclictest26907-21sed19:10:212
251379970,1cyclictest24764-21/usr/sbin/munin20:10:152
251379970,1cyclictest24692-21ssh21:10:512
251379970,1cyclictest24534-21ssh22:00:112
251379970,1cyclictest24040-21sh00:10:112
251379970,1cyclictest23910-21ssh21:59:572
251379970,1cyclictest22910-21/usr/sbin/munin22:15:172
251379970,1cyclictest21184-21idleruntime-cro20:05:002
251379970,1cyclictest17168-21apt-get22:45:142
251379970,1cyclictest16127-21ssh23:00:192
251379970,1cyclictest14871-21perl20:55:152
251379970,1cyclictest12912-21ntp_states00:20:182
251379970,1cyclictest10790-21ssh21:20:172
251379970,1cyclictest10672-21ssh21:36:202
251379970,1cyclictest10189-21basename23:30:142
251379970,1cyclictest0-21swapper/222:40:132
251379970,1cyclictest0-21swapper/221:25:172
251379970,1cyclictest0-21swapper/220:05:142
251379961,4cyclictest0-21swapper/219:55:112
251379960,1cyclictest0-21swapper/220:35:112
251379960,1cyclictest0-21swapper/220:30:132
251379960,0cyclictest9211-21apt-get20:45:122
251259960,6cyclictest0-21swapper/000:30:150
251259960,3cyclictest8891-21perf22:25:000
32705250,0sleep10-21swapper/123:09:121
251389955,0cyclictest27002-21kworker/3:122:13:163
251389951,2cyclictest0-21swapper/319:15:123
251389950,5cyclictest0-21swapper/323:56:223
251389950,5cyclictest0-21swapper/322:28:463
251389950,5cyclictest0-21swapper/300:10:543
251389950,4cyclictest29039-21apt-get20:20:023
251339950,5cyclictest0-21swapper/122:28:281
251259955,0cyclictest4248-21apt-get20:35:120
251259952,3cyclictest27691-21kworker/0:021:41:130
251259951,4cyclictest28817-21kworker/0:223:20:100
251259950,5cyclictest0-21swapper/019:40:030
251259950,5cyclictest0-21swapper/000:35:110
23909250,0sleep10-21swapper/121:10:221
251389944,0cyclictest7020-21kworker/3:219:58:563
251389944,0cyclictest0-21swapper/323:41:483
251389942,2cyclictest5816-21ssh23:11:243
251389942,1cyclictest11220-21ssh00:35:333
251389941,3cyclictest26247-21ssh00:27:073
251389941,1cyclictest0-21swapper/323:00:123
251389941,1cyclictest0-21swapper/319:50:143
251389940,4cyclictest55950irq/130-eth000:30:153
251389940,4cyclictest4671-21apt-get21:50:143
251389940,4cyclictest29388-21apt-get22:35:023
251389940,4cyclictest23592-21systemctl22:15:203
251389940,4cyclictest20349-21perl22:30:153
251389940,4cyclictest0-21swapper/323:50:143
251389940,4cyclictest0-21swapper/323:30:123
251389940,4cyclictest0-21swapper/322:55:123
251389940,4cyclictest0-21swapper/322:50:123
251389940,4cyclictest0-21swapper/322:45:133
251389940,4cyclictest0-21swapper/322:05:143
251389940,4cyclictest0-21swapper/322:04:403
251389940,4cyclictest0-21swapper/321:55:093
251389940,4cyclictest0-21swapper/321:45:133
251389940,4cyclictest0-21swapper/321:35:143
251389940,4cyclictest0-21swapper/321:30:263
251389940,4cyclictest0-21swapper/321:25:203
251389940,4cyclictest0-21swapper/321:20:133
251389940,4cyclictest0-21swapper/320:55:113
251389940,4cyclictest0-21swapper/320:40:113
251389940,4cyclictest0-21swapper/320:15:233
251389940,4cyclictest0-21swapper/319:35:123
251389940,4cyclictest0-21swapper/319:10:123
251389940,4cyclictest0-21swapper/300:15:403
251389940,4cyclictest0-21swapper/300:00:143
251389940,3cyclictest32473-21sensors_temp19:20:223
251389940,3cyclictest321-21ssh23:25:173
251389940,3cyclictest26287-21cat20:15:003
251389940,3cyclictest23179-21ssh23:20:173
251389940,3cyclictest22954-21irqstats21:10:173
251389940,3cyclictest19854-21fw_packets21:05:163
251389940,3cyclictest1931-21ssh21:15:373
251389940,3cyclictest16645-21/usr/sbin/munin21:00:163
251389940,3cyclictest14859-21perl00:05:133
251389940,3cyclictest14512-21ssh00:20:583
251389940,2cyclictest15892-21ssh23:16:313
251389940,1cyclictest4245-21/usr/sbin/munin20:35:143
251389940,1cyclictest19086-21/usr/sbin/munin23:35:123
251339944,0cyclictest0-21swapper/123:49:221
251339944,0cyclictest0-21swapper/121:32:531
251339942,2cyclictest0-21swapper/123:40:131
251339942,1cyclictest0-21swapper/119:25:121
251339941,2cyclictest0-21swapper/123:15:131
251339941,2cyclictest0-21swapper/122:05:151
251339941,2cyclictest0-21swapper/121:45:201
251339941,2cyclictest0-21swapper/121:35:151
251339941,2cyclictest0-21swapper/121:15:121
251339941,2cyclictest0-21swapper/121:05:151
251339941,2cyclictest0-21swapper/120:30:121
251339941,2cyclictest0-21swapper/120:25:121
251339941,2cyclictest0-21swapper/120:00:111
251339941,2cyclictest0-21swapper/119:50:141
251339941,2cyclictest0-21swapper/119:35:011
251339941,2cyclictest0-21swapper/119:10:121
251339941,2cyclictest0-21swapper/100:30:151
251339941,2cyclictest0-21swapper/100:20:121
251339940,4cyclictest0-21swapper/123:22:351
251339940,4cyclictest0-21swapper/122:44:281
251339940,4cyclictest0-21swapper/120:09:561
251339940,4cyclictest0-21swapper/100:19:211
251339940,2cyclictest0-21swapper/121:00:161
251339940,0cyclictest0-21swapper/121:43:211
251259944,0cyclictest0-21swapper/023:16:080
251259944,0cyclictest0-21swapper/022:15:270
251259944,0cyclictest0-21swapper/022:05:440
251259941,3cyclictest28817-21kworker/0:222:36:040
251259941,2cyclictest0-21swapper/022:10:130
251259940,4cyclictest5917-21dump-pmu-power19:35:010
251259940,4cyclictest27691-21kworker/0:019:20:180
251259940,4cyclictest25225-21apt-get19:10:120
251259940,4cyclictest16646-21apt-get21:00:160
251259940,4cyclictest1654-21ssh21:15:270
251259940,4cyclictest0-21swapper/023:54:060
251259940,4cyclictest0-21swapper/022:51:210
251259940,4cyclictest0-21swapper/022:45:130
251259940,4cyclictest0-21swapper/022:41:360
251259940,4cyclictest0-21swapper/022:25:130
251259940,4cyclictest0-21swapper/021:35:140
251259940,4cyclictest0-21swapper/021:25:300
251259940,4cyclictest0-21swapper/021:20:120
251259940,4cyclictest0-21swapper/021:10:130
251259940,4cyclictest0-21swapper/020:54:410
251259940,4cyclictest0-21swapper/019:35:120
251259940,4cyclictest0-21swapper/000:25:130
251259940,4cyclictest0-21swapper/000:15:130
251259940,4cyclictest0-21swapper/000:05:110
251259940,3cyclictest771-21apt-get19:25:020
251259940,3cyclictest6964-21sh23:28:530
251259940,3cyclictest6715-21awk21:50:230
251259940,3cyclictest31651-21dpkg20:25:000
251259940,3cyclictest30534-21ssh21:30:170
251259940,3cyclictest29585-21diskstats20:20:150
251259940,3cyclictest27777-21perl21:45:140
251259940,3cyclictest25672-21irqstats23:05:160
251259940,3cyclictest2479-21sort20:30:140
251259940,3cyclictest21049-21sed00:25:010
251259940,3cyclictest19108-21/usr/sbin/munin21:05:160
251259940,3cyclictest14983-21hddtemp_smartct20:55:140
251259940,3cyclictest0-21swapper/021:55:120
251259940,1cyclictest18702-21munin-run20:00:010
251259940,0cyclictest0-21swapper/023:32:520
21238240,0sleep10-21swapper/100:08:331
251389932,1cyclictest714-21snmpd21:40:033
251389931,1cyclictest0-21swapper/320:05:123
251389931,1cyclictest0-21swapper/319:30:123
251389930,3cyclictest31647-21dpkg20:25:013
251389930,3cyclictest24597-21ssh23:05:123
251389930,3cyclictest2294-21df20:30:133
251389930,3cyclictest0-21swapper/323:45:123
251389930,3cyclictest0-21swapper/320:50:143
251389930,3cyclictest0-21swapper/320:45:123
251389930,3cyclictest0-21swapper/320:00:123
251389930,3cyclictest0-21swapper/319:45:033
251389930,3cyclictest0-21swapper/319:40:053
251389930,3cyclictest0-21swapper/319:25:013
251339933,0cyclictest0-21swapper/121:26:111
251339933,0cyclictest0-21swapper/119:24:591
251339933,0cyclictest0-21swapper/100:13:581
251339931,1cyclictest0-21swapper/123:55:111
251339931,1cyclictest0-21swapper/123:35:111
251339931,1cyclictest0-21swapper/123:30:111
251339931,1cyclictest0-21swapper/123:25:121
251339931,1cyclictest0-21swapper/122:55:121
251339931,1cyclictest0-21swapper/122:50:131
251339931,1cyclictest0-21swapper/122:45:131
251339931,1cyclictest0-21swapper/122:35:011
251339931,1cyclictest0-21swapper/122:30:121
251339931,1cyclictest0-21swapper/122:15:121
251339931,1cyclictest0-21swapper/122:00:121
251339931,1cyclictest0-21swapper/120:55:111
251339931,1cyclictest0-21swapper/120:40:121
251339931,1cyclictest0-21swapper/120:20:131
251339931,1cyclictest0-21swapper/120:10:131
251339931,1cyclictest0-21swapper/119:55:111
251339931,1cyclictest0-21swapper/119:40:111
251339931,1cyclictest0-21swapper/119:30:111
251339931,1cyclictest0-21swapper/100:25:131
251339930,3cyclictest0-21swapper/119:15:041
251339930,2cyclictest11305-21chrt19:45:071
251339930,2cyclictest0-21swapper/100:00:141
251259932,1cyclictest714-21snmpd20:05:020
251259931,1cyclictest24527-21apt-get22:00:110
251259930,3cyclictest4079-21apt-get22:55:020
251259930,3cyclictest26556-21apt23:55:120
251259930,3cyclictest19001-21basename20:00:110
251259930,3cyclictest0-21swapper/022:30:100
251259930,3cyclictest0-21swapper/020:40:000
251259930,3cyclictest0-21swapper/020:15:100
251259930,3cyclictest0-21swapper/020:10:120
251259930,3cyclictest0-21swapper/019:50:140
251259930,3cyclictest0-21swapper/019:45:100
251259930,3cyclictest0-21swapper/019:15:050
251259930,2cyclictest21354-21diskmemload23:45:080
251259930,2cyclictest21354-21diskmemload23:40:070
251259930,2cyclictest18948-21sh23:35:040
251259930,2cyclictest14512-21ssh23:00:060
251339921,1cyclictest92022chrt20:45:071
251339921,1cyclictest0-21swapper/121:55:131
251339921,1cyclictest0-21swapper/121:20:121
251339921,1cyclictest0-21swapper/120:50:131
251339920,1cyclictest4446-21chrt21:50:051
251339920,1cyclictest31653-21chrt22:20:041
251339920,1cyclictest144502chrt23:00:031
251339920,1cyclictest0-21swapper/123:10:121
251339920,1cyclictest0-21swapper/122:10:121
251339920,1cyclictest0-21swapper/120:35:111
251339920,1cyclictest0-21swapper/120:15:121
251339920,1cyclictest0-21swapper/100:35:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional