You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-04 - 02:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Fri Jul 04, 2025 00:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
77232195179,11sleep10-21swapper/119:08:361
77772180163,12sleep00-21swapper/019:09:190
74852180163,12sleep30-21swapper/319:05:323
78212173156,12sleep20-21swapper/219:09:532
245262490,0sleep10-21swapper/122:09:351
323682470,0sleep10-21swapper/121:56:291
75302460,0sleep10-21swapper/121:27:591
189222450,0sleep10-21swapper/100:34:541
78322440,0sleep10-21swapper/122:00:211
81362430,0sleep10-21swapper/100:12:201
219802420,0sleep00-21swapper/000:03:040
25742410,0sleep00-21swapper/000:25:420
180832390,0sleep10-21swapper/121:33:231
802999113,4cyclictest20643-21munin-run21:35:000
17022110,0sleep10-21swapper/123:35:501
269482100,0sleep30-21swapper/300:05:193
199672100,0sleep10-21swapper/122:40:011
80359991,1cyclictest6912-21cat23:55:162
80359990,1cyclictest13427-21ssh22:03:472
24807290,0sleep10-21swapper/100:20:261
24568290,0sleep10-21swapper/120:45:231
80359980,1cyclictest9648-21sshd20:15:192
80359980,1cyclictest6770-21sed23:05:212
80359980,1cyclictest5158-21munin-run23:55:012
80359980,1cyclictest4298-21cat23:05:012
80359980,1cyclictest30937-21apt19:55:002
80359980,1cyclictest30715-21munin-run20:59:592
80359980,1cyclictest24986-21ls23:15:132
80359980,1cyclictest2398-21munin-run00:10:002
80359980,1cyclictest23966-21munin-run23:14:592
80359980,1cyclictest22878-21sed20:40:232
80359980,1cyclictest220-21systemd-journal19:40:012
80359980,1cyclictest220-21systemd-journal19:30:012
80359980,1cyclictest21994-21ssh23:46:162
80359980,1cyclictest20559-21/usr/sbin/munin00:35:162
80359980,1cyclictest20159-21tr19:30:212
80359980,1cyclictest18974-21/usr/sbin/munin21:50:142
80359980,1cyclictest15878-21/usr/sbin/munin22:05:172
80359980,1cyclictest15858-21idleruntime-cro20:30:002
80359980,1cyclictest14803-21python22:20:202
80359980,1cyclictest1480-21cut00:25:182
80359980,1cyclictest13393-21grep20:25:002
80359980,1cyclictest0-21swapper/221:43:472
80299980,4cyclictest31899-21idleruntime-cro00:25:000
80399970,7cyclictest61-21kswapd000:30:143
80359971,1cyclictest22020-21expr23:30:122
80359970,1cyclictest9780-21if_eth423:40:172
80359970,1cyclictest8825-21sort19:10:152
80359970,1cyclictest7640-21aten2_r5power_e22:50:152
80359970,1cyclictest6917-21cut20:10:162
80359970,1cyclictest525-21nscd20:05:132
80359970,1cyclictest4886-21grep21:10:182
80359970,1cyclictest3766-21df_inode22:15:142
80359970,1cyclictest364-21ssh22:30:142
80359970,1cyclictest364-21ssh22:30:142
80359970,1cyclictest3477-21grep20:05:002
80359970,1cyclictest31986-21sed19:55:152
80359970,1cyclictest31300-21ssh23:35:112
80359970,1cyclictest29434-21hddtemp_smartct21:55:152
80359970,1cyclictest2748-21rm00:10:062
80359970,1cyclictest26925-21cut19:45:162
80359970,1cyclictest26527-21df20:50:182
80359970,1cyclictest26144-21tr22:10:152
80359970,1cyclictest25449-21timerwakeupswit19:40:222
80359970,1cyclictest25367-21ntpq21:20:212
80359970,1cyclictest24853-21rm22:58:502
80359970,1cyclictest24301-21ntp_states22:25:182
80359970,1cyclictest23992-21expr20:45:202
80359970,1cyclictest21891-21cut22:40:232
80359970,1cyclictest21688-21ls21:35:172
80359970,1cyclictest2030-21cut23:20:142
80359970,1cyclictest19400-21sh20:35:152
80359970,1cyclictest17445-21awk20:30:182
80359970,1cyclictest16317-21df_inode00:00:142
80359970,1cyclictest15687-21python21:15:222
80359970,1cyclictest15668-21head19:20:242
80359970,1cyclictest1515-21df21:05:162
80359970,1cyclictest1346-21hostname21:25:122
80359970,1cyclictest13457-21grep00:15:182
80359970,1cyclictest13103-21ssh22:35:522
80359970,1cyclictest12815-21expr19:15:192
80359970,1cyclictest12815-21expr19:15:192
80359970,1cyclictest11696-21grep23:25:102
80359970,1cyclictest11147-21rm21:30:092
80359970,1cyclictest10607-21ssh00:30:152
80359970,1cyclictest10175-21readlink21:45:192
80359970,1cyclictest0-21swapper/200:20:132
80359960,1cyclictest30938-21taskset21:00:012
80359960,1cyclictest30389-21tr22:45:152
80299962,4cyclictest23982-21ssh23:15:000
80299960,3cyclictest21047-21munin-run19:35:000
80299960,3cyclictest19577-21munin-run22:40:010
80299960,3cyclictest13405-21munin-run21:15:000
80399955,0cyclictest25070-21kworker/3:222:30:253
80399955,0cyclictest25070-21kworker/3:222:30:253
80399955,0cyclictest25070-21kworker/3:221:45:383
80399955,0cyclictest25070-21kworker/3:221:12:033
80399955,0cyclictest0-21swapper/322:26:073
80399955,0cyclictest0-21swapper/319:40:223
80399954,1cyclictest25070-21kworker/3:222:41:383
80399951,4cyclictest0-21swapper/322:49:143
80399950,5cyclictest0-21swapper/321:22:243
80399950,4cyclictest8654-21/usr/sbin/munin20:15:163
80399950,3cyclictest13971-21/usr/sbin/munin21:15:203
80399950,0cyclictest0-21swapper/300:15:493
80339954,1cyclictest0-21swapper/123:55:371
80339951,2cyclictest0-21swapper/122:20:121
80299950,5cyclictest14014-21ssh23:42:270
80299950,5cyclictest0-21swapper/023:54:400
80299950,5cyclictest0-21swapper/022:51:430
80299950,5cyclictest0-21swapper/021:40:390
80299950,4cyclictest14175-21irqstats22:20:170
80299950,4cyclictest0-21swapper/021:35:150
80299950,1cyclictest11134-21aten2_r5power_p19:15:020
80299950,1cyclictest11134-21aten2_r5power_p19:15:020
80299950,0cyclictest0-21swapper/000:07:380
80399944,0cyclictest25070-21kworker/3:221:34:273
80399943,1cyclictest8119-21/usr/sbin/munin19:10:153
80399943,1cyclictest19161-21tr19:30:153
80399942,1cyclictest30956-21/usr/sbin/munin21:00:173
80399941,2cyclictest6176-21/usr/sbin/munin20:10:183
80399940,4cyclictest6224-21ssh00:11:083
80399940,4cyclictest21475-21apt-get23:30:113
80399940,4cyclictest1524-21df_inode21:05:163
80399940,4cyclictest0-21swapper/323:54:393
80399940,4cyclictest0-21swapper/323:15:053
80399940,4cyclictest0-21swapper/323:10:133
80399940,4cyclictest0-21swapper/322:58:293
80399940,4cyclictest0-21swapper/322:15:203
80399940,4cyclictest0-21swapper/322:10:343
80399940,4cyclictest0-21swapper/321:50:133
80399940,4cyclictest0-21swapper/321:40:143
80399940,4cyclictest0-21swapper/319:50:133
80399940,4cyclictest0-21swapper/319:38:163
80399940,4cyclictest0-21swapper/319:25:153
80399940,4cyclictest0-21swapper/319:19:523
80399940,4cyclictest0-21swapper/319:19:513
80399940,4cyclictest0-21swapper/300:36:033
80399940,4cyclictest0-21swapper/300:28:463
80399940,3cyclictest9932-21rm22:35:023
80399940,3cyclictest3707-21apt-get20:05:123
80399940,3cyclictest27809-21df_inode23:00:133
80399940,3cyclictest20985-21latency_hist23:30:003
80399940,3cyclictest19935-21/usr/sbin/munin20:35:213
80399940,3cyclictest16554-21perl00:00:163
80399940,3cyclictest16442-21df22:05:133
80399940,3cyclictest1593-21sh23:20:013
80399940,3cyclictest14147-21df_inode20:25:133
80399940,3cyclictest1173-21date20:00:003
80399940,1cyclictest8717-21/usr/sbin/munin23:40:143
80399940,1cyclictest7572-21/usr/sbin/munin22:50:233
80399940,1cyclictest4831-21/usr/sbin/munin23:05:153
80399940,1cyclictest31294-21/usr/sbin/munin23:35:143
80399940,1cyclictest25996-21/usr/sbin/munin20:50:173
80399940,1cyclictest24256-21perl20:45:203
80399940,1cyclictest21936-21grep21:35:173
80339944,0cyclictest0-21swapper/119:25:231
80339941,2cyclictest25655-21chrt19:40:501
80339941,2cyclictest22718-21chrt22:57:271
80339941,2cyclictest0-21swapper/121:10:151
80339941,2cyclictest0-21swapper/121:05:131
80339941,2cyclictest0-21swapper/120:40:121
80339941,1cyclictest0-21swapper/123:50:111
80339941,1cyclictest0-21swapper/123:45:101
80339941,1cyclictest0-21swapper/120:15:141
80339940,4cyclictest0-21swapper/123:20:321
80339940,2cyclictest0-21swapper/123:40:131
80299944,0cyclictest0-21swapper/019:25:020
80299944,0cyclictest0-21swapper/019:20:240
80299944,0cyclictest0-21swapper/000:31:210
80299943,1cyclictest3534-21find23:20:200
80299942,1cyclictest14312-21grep23:10:010
80299942,1cyclictest10900-21latency_hist20:20:000
80299941,3cyclictest11943-21ssh22:02:500
80299941,2cyclictest0-21swapper/020:20:120
80299941,2cyclictest0-21swapper/019:10:140
80299940,4cyclictest6179-21apt-get20:10:150
80299940,4cyclictest23764-21apt-get19:40:140
80299940,4cyclictest1497-21ssh23:35:420
80299940,4cyclictest0-21swapper/023:45:110
80299940,4cyclictest0-21swapper/023:18:130
80299940,4cyclictest0-21swapper/022:45:140
80299940,4cyclictest0-21swapper/022:40:170
80299940,4cyclictest0-21swapper/020:50:210
80299940,4cyclictest0-21swapper/020:45:170
80299940,4cyclictest0-21swapper/020:30:130
80299940,4cyclictest0-21swapper/000:35:130
80299940,3cyclictest9146-21fschecks_count21:45:150
80299940,3cyclictest2926-21ssh00:10:130
80299940,3cyclictest28650-21ls19:50:000
80299940,3cyclictest25815-21ssh23:31:470
80299940,3cyclictest17454-21ssh22:55:140
80299940,3cyclictest15884-21apt22:05:110
80299940,3cyclictest13632-21grep23:25:180
80299940,3cyclictest0-21swapper/021:55:140
80299940,1cyclictest31701-21diskstats19:55:130
80299940,0cyclictest0-21swapper/023:57:430
80299940,0cyclictest0-21swapper/019:53:250
1279240,0sleep00-21swapper/021:25:110
80399932,1cyclictest16202-21grep20:30:133
80399931,2cyclictest57250irq/125-eth000:20:013
80399930,3cyclictest7120-21irqstats22:00:183
80399930,3cyclictest18382-21rm23:45:073
80399930,3cyclictest13076-21apt-get22:20:133
80399930,3cyclictest1132-21sh21:25:053
80399930,3cyclictest0-21swapper/323:55:003
80399930,3cyclictest0-21swapper/321:55:023
80399930,3cyclictest0-21swapper/320:55:063
80399930,3cyclictest0-21swapper/320:40:053
80399930,3cyclictest0-21swapper/320:20:123
80399930,3cyclictest0-21swapper/320:00:153
80399930,3cyclictest0-21swapper/319:20:093
80399930,2cyclictest30997-21snmpd19:45:153
80339933,0cyclictest211ktimersoftd/121:40:461
80339933,0cyclictest211ktimersoftd/100:17:471
80339933,0cyclictest0-21swapper/123:12:331
80339933,0cyclictest0-21swapper/122:54:211
80339931,2cyclictest205902sleep120:37:291
80339931,1cyclictest8331-21chrt21:45:051
80339931,1cyclictest24290-21chrt23:15:041
80339931,1cyclictest20893-21chrt21:50:371
80339931,1cyclictest0-21swapper/123:30:121
80339931,1cyclictest0-21swapper/123:25:101
80339931,1cyclictest0-21swapper/123:00:131
80339931,1cyclictest0-21swapper/122:35:131
80339931,1cyclictest0-21swapper/121:15:151
80339931,1cyclictest0-21swapper/121:00:161
80339931,1cyclictest0-21swapper/120:30:131
80339931,1cyclictest0-21swapper/120:05:131
80339931,1cyclictest0-21swapper/120:00:161
80339931,1cyclictest0-21swapper/119:55:111
80339931,1cyclictest0-21swapper/119:50:121
80339931,1cyclictest0-21swapper/119:35:131
80339931,1cyclictest0-21swapper/119:30:141
80339931,1cyclictest0-21swapper/119:20:141
80339931,1cyclictest0-21swapper/100:35:121
80339931,1cyclictest0-21swapper/100:25:121
80339931,0cyclictest0-21swapper/123:05:131
80339930,3cyclictest0-21swapper/120:50:141
80339930,2cyclictest0-21swapper/122:25:101
80339930,2cyclictest0-21swapper/120:25:131
80339930,2cyclictest0-21swapper/119:45:141
80339930,2cyclictest0-21swapper/119:10:141
80339930,2cyclictest0-21swapper/100:05:121
80339930,1cyclictest0-21swapper/119:15:131
80339930,1cyclictest0-21swapper/119:15:121
80299932,1cyclictest30997-21snmpd20:40:060
80299932,1cyclictest12679-21ssh00:15:080
80299931,2cyclictest0-21swapper/022:10:120
80299931,1cyclictest0-21swapper/021:20:160
80299930,3cyclictest3231-21diskmemload22:25:050
80299930,3cyclictest31468-21expr21:00:180
80299930,3cyclictest29717-21awk20:55:220
80299930,3cyclictest1472-21grep20:00:160
80299930,3cyclictest0-21swapper/022:30:130
80299930,3cyclictest0-21swapper/022:30:130
80299930,3cyclictest0-21swapper/022:15:140
80299930,3cyclictest0-21swapper/021:50:130
80299930,3cyclictest0-21swapper/021:15:110
80299930,3cyclictest0-21swapper/021:05:010
80299930,3cyclictest0-21swapper/020:35:050
80299930,3cyclictest0-21swapper/020:25:010
80299930,3cyclictest0-21swapper/020:05:140
80299930,3cyclictest0-21swapper/019:35:120
80299930,0cyclictest27241-21apt-get23:00:120
80339922,0cyclictest201rcuc/100:00:011
80339921,1cyclictest284542sleep120:55:081
80339921,1cyclictest236242sleep121:20:031
80339921,1cyclictest0-21swapper/121:35:161
80339921,0cyclictest0-21swapper/122:45:131
80339920,1cyclictest0-21swapper/122:30:141
80339920,1cyclictest0-21swapper/122:30:141
80339920,1cyclictest0-21swapper/122:15:141
80339920,1cyclictest0-21swapper/122:10:151
80339920,1cyclictest0-21swapper/120:20:121
80339920,1cyclictest0-21swapper/120:10:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional