You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-13 - 06:36
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Tue May 13, 2025 00:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
114942210194,11sleep10-21swapper/119:09:401
113492195178,12sleep30-21swapper/319:07:493
113142182165,12sleep00-21swapper/019:07:220
11443210993,12sleep20-21swapper/219:09:012
40542440,1sleep31172399cyclictest23:50:213
86432430,0sleep00-21swapper/023:36:330
262032430,0sleep10-21swapper/122:24:031
98332410,0sleep10-21swapper/122:48:171
98332410,0sleep10-21swapper/122:48:171
127302410,1sleep11171199cyclictest00:28:101
244342400,0sleep20-21swapper/200:00:452
246782100,0sleep30-21swapper/300:17:443
117169991,1cyclictest0-21swapper/223:40:132
117169982,1cyclictest32125-21ssh23:15:242
117169980,3cyclictest11951-21sh21:11:412
28636270,0sleep20-21swapper/220:45:202
117169973,0cyclictest21215-21grep22:05:152
117169972,1cyclictest27628-21ssh22:57:012
117169972,1cyclictest20961-21ssh23:59:532
117169972,1cyclictest13662-21ssh23:06:092
117169971,1cyclictest29781-21ssh22:41:372
117169971,1cyclictest23654-21ssh23:11:132
117169971,1cyclictest18706-21ssh22:52:392
117169971,1cyclictest0-21swapper/223:25:132
117169971,1cyclictest0-21swapper/219:50:152
117079971,3cyclictest4185-21munin-run23:35:000
117079971,3cyclictest32328-21munin-run19:50:010
117079970,3cyclictest7253-21perf21:10:010
117239961,4cyclictest0-21swapper/319:45:133
117169962,1cyclictest11849-21ssh00:10:502
117169961,1cyclictest6692-21ssh22:46:182
117169961,1cyclictest6692-21ssh22:46:182
117169961,1cyclictest32055-21ssh22:26:292
117169960,6cyclictest0-21swapper/223:45:282
117169960,3cyclictest0-21swapper/221:15:182
117169960,3cyclictest0-21swapper/220:30:182
117079960,5cyclictest0-21swapper/020:05:020
28226250,0sleep30-21swapper/323:30:173
117239955,0cyclictest17083-21kworker/3:023:40:313
117239955,0cyclictest17083-21kworker/3:023:18:093
117239955,0cyclictest17083-21kworker/3:023:06:583
117239955,0cyclictest17083-21kworker/3:022:44:343
117239955,0cyclictest17083-21kworker/3:021:59:473
117239955,0cyclictest17083-21kworker/3:021:48:353
117239955,0cyclictest17083-21kworker/3:021:37:223
117239955,0cyclictest17083-21kworker/3:019:56:373
117239955,0cyclictest17083-21kworker/3:000:36:293
117239955,0cyclictest17083-21kworker/3:000:14:063
117239955,0cyclictest0-21swapper/323:11:013
117239950,5cyclictest7233-21diskmemload00:33:053
117239950,5cyclictest0-21swapper/323:29:123
117239950,5cyclictest0-21swapper/323:24:233
117239950,5cyclictest0-21swapper/322:14:433
117239950,4cyclictest0-21swapper/320:50:133
117169955,0cyclictest0-21swapper/222:23:422
117169951,1cyclictest2228-21ssh00:39:272
117169950,5cyclictest0-21swapper/219:10:132
117119951,2cyclictest0-21swapper/123:20:141
117119951,2cyclictest0-21swapper/123:10:151
117119951,2cyclictest0-21swapper/100:05:141
117119950,5cyclictest0-21swapper/123:47:421
117119950,5cyclictest0-21swapper/123:41:181
117079950,5cyclictest0-21swapper/023:58:140
117079950,5cyclictest0-21swapper/021:27:300
117079950,4cyclictest1215-21memory19:50:190
117239944,0cyclictest9299-21kworker/3:119:11:493
117239944,0cyclictest17083-21kworker/3:020:30:133
117239943,1cyclictest12449-21sh20:15:003
117239941,3cyclictest9451-21ssh22:31:253
117239941,3cyclictest27306-21latency_hist20:45:003
117239941,3cyclictest25200-21ssh21:50:193
117239941,3cyclictest0-21swapper/321:40:413
117239940,4cyclictest2530-21apt-get21:00:123
117239940,4cyclictest15444-21ssh23:56:193
117239940,4cyclictest0-21swapper/323:47:003
117239940,4cyclictest0-21swapper/322:50:523
117239940,4cyclictest0-21swapper/322:45:133
117239940,4cyclictest0-21swapper/322:45:133
117239940,4cyclictest0-21swapper/322:35:143
117239940,4cyclictest0-21swapper/322:15:143
117239940,4cyclictest0-21swapper/322:00:113
117239940,4cyclictest0-21swapper/321:15:143
117239940,4cyclictest0-21swapper/320:35:243
117239940,4cyclictest0-21swapper/320:25:073
117239940,4cyclictest0-21swapper/300:21:063
117239940,4cyclictest0-21swapper/300:05:143
117239940,4cyclictest0-21swapper/300:00:133
117239940,3cyclictest31089-21ssh22:59:133
117239940,3cyclictest30253-21ssh21:20:313
117239940,3cyclictest2619-21/usr/sbin/munin23:00:263
117239940,3cyclictest23275-21ssh22:22:113
117239940,3cyclictest22627-21/usr/sbin/munin19:30:143
117239940,3cyclictest22627-21/usr/sbin/munin19:30:133
117239940,3cyclictest21080-21ssh22:05:133
117239940,3cyclictest15362-21ssh21:30:043
117239940,3cyclictest15130-21wc20:20:003
117239940,3cyclictest1483-21ssh22:27:403
117239940,3cyclictest13500-21ssh00:28:403
117239940,3cyclictest0-21swapper/323:35:153
117239940,3cyclictest0-21swapper/321:10:123
117169944,0cyclictest0-21swapper/221:20:152
117169944,0cyclictest0-21swapper/200:18:032
117169940,4cyclictest7233-21diskmemload23:50:522
117169940,4cyclictest3917-21ssh21:40:162
117169940,4cyclictest17540-21apt-get19:20:152
117169940,4cyclictest15626-21apt-get22:35:132
117169940,4cyclictest0-21swapper/223:37:012
117169940,4cyclictest0-21swapper/222:13:182
117169940,4cyclictest0-21swapper/200:23:312
117169940,4cyclictest0-21swapper/200:05:022
117169940,3cyclictest3123-21uniq21:00:132
117169940,3cyclictest28136-21df19:40:132
117169940,3cyclictest26625-21ssh21:35:182
117169940,3cyclictest26400-21ntp_states19:35:182
117169940,3cyclictest25523-21cat00:35:002
117169940,3cyclictest23104-21df20:35:152
117169940,3cyclictest23024-21apt-config21:50:012
117169940,3cyclictest19998-21/usr/sbin/munin19:25:132
117169940,3cyclictest1653-21irqstats23:00:192
117169940,3cyclictest15585-21/usr/sbin/munin21:30:152
117169940,3cyclictest14922-21tail19:15:022
117169940,3cyclictest14090-21ssh21:29:252
117169940,3cyclictest1128-21memory20:55:152
117169940,3cyclictest10033-21sh23:20:372
117169940,1cyclictest27306-21ssh23:30:122
117119942,1cyclictest0-21swapper/119:55:111
117119941,2cyclictest0-21swapper/123:55:111
117119941,2cyclictest0-21swapper/123:35:141
117119941,2cyclictest0-21swapper/123:05:141
117119941,2cyclictest0-21swapper/123:00:161
117119941,2cyclictest0-21swapper/122:35:141
117119941,2cyclictest0-21swapper/121:50:101
117119941,2cyclictest0-21swapper/121:30:121
117119941,2cyclictest0-21swapper/120:55:111
117119941,2cyclictest0-21swapper/120:50:141
117119941,2cyclictest0-21swapper/120:35:151
117119941,2cyclictest0-21swapper/120:30:141
117119941,2cyclictest0-21swapper/120:25:151
117119941,2cyclictest0-21swapper/120:10:141
117119941,2cyclictest0-21swapper/120:05:131
117119941,2cyclictest0-21swapper/119:50:141
117119941,2cyclictest0-21swapper/119:45:131
117119941,2cyclictest0-21swapper/119:30:131
117119941,2cyclictest0-21swapper/119:30:121
117119941,2cyclictest0-21swapper/119:25:111
117119941,2cyclictest0-21swapper/119:15:151
117119941,2cyclictest0-21swapper/100:10:151
117119941,1cyclictest0-21swapper/122:00:121
117119941,1cyclictest0-21swapper/119:10:131
117119940,4cyclictest0-21swapper/122:53:521
117119940,4cyclictest0-21swapper/100:33:231
117119940,4cyclictest0-21swapper/100:01:131
117119940,2cyclictest0-21swapper/122:40:131
117119940,0cyclictest0-21swapper/122:30:171
117079944,0cyclictest0-21swapper/023:01:120
117079944,0cyclictest0-21swapper/000:16:340
117079943,1cyclictest9075-21/usr/sbin/munin00:10:180
117079942,1cyclictest30997-21snmpd00:30:120
117079941,3cyclictest30997-21snmpd22:18:010
117079940,4cyclictest29670-21fw_packets00:20:160
117079940,4cyclictest21063-21perl23:10:170
117079940,4cyclictest19603-21ssh23:42:070
117079940,4cyclictest11028-21ssh22:00:110
117079940,4cyclictest0-21swapper/022:50:320
117079940,4cyclictest0-21swapper/022:25:060
117079940,4cyclictest0-21swapper/022:12:180
117079940,4cyclictest0-21swapper/021:45:330
117079940,4cyclictest0-21swapper/020:40:130
117079940,4cyclictest0-21swapper/020:12:160
117079940,4cyclictest0-21swapper/000:35:540
117079940,3cyclictest32231-21latency_hist20:54:590
117079940,3cyclictest27621-21apt-get19:40:130
117079940,3cyclictest26542-21hddtemp_smartct21:35:170
117079940,3cyclictest25220-21apt-config22:40:020
117079940,3cyclictest25007-21sh22:06:340
117079940,3cyclictest23070-21apt-config21:50:010
117079940,3cyclictest22932-21ssh21:33:410
117079940,3cyclictest19746-21basename23:10:010
117079940,3cyclictest18063-21df_inode19:20:170
117079940,3cyclictest17800-21sh22:35:270
117079940,2cyclictest0-21swapper/021:17:460
117079940,1cyclictest7118-21ntp_states22:30:190
117079940,1cyclictest6276-21/usr/sbin/munin00:25:170
117079940,1cyclictest3183-21/usr/sbin/munin22:45:220
117079940,1cyclictest3183-21/usr/sbin/munin22:45:220
117079940,1cyclictest31599-21/usr/sbin/munin00:05:150
117079940,1cyclictest26784-21munin-run23:30:000
117079940,1cyclictest25128-21/usr/sbin/munin19:35:150
117079940,1cyclictest24821-21munin-run20:40:000
117079940,1cyclictest22973-21/usr/sbin/munin22:55:150
117079940,1cyclictest1061-21/usr/sbin/munin21:55:170
117079940,0cyclictest0-21swapper/021:10:100
117239932,1cyclictest30997-21snmpd19:40:033
117239931,1cyclictest0-21swapper/319:35:013
117239930,3cyclictest5326-21rm21:25:053
117239930,3cyclictest14352-21ntp_states19:15:003
117239930,3cyclictest0-21swapper/321:05:143
117239930,3cyclictest0-21swapper/320:55:113
117239930,3cyclictest0-21swapper/320:45:053
117239930,3cyclictest0-21swapper/320:20:123
117239930,3cyclictest0-21swapper/320:05:023
117239930,3cyclictest0-21swapper/320:00:053
117239930,3cyclictest0-21swapper/319:50:143
117239930,3cyclictest0-21swapper/319:25:023
117239930,3cyclictest0-21swapper/319:20:163
117169932,1cyclictest30150-21cut19:45:122
117169932,1cyclictest12973-21cat20:15:112
117169931,1cyclictest7747-21apt-get20:05:112
117169931,1cyclictest0-21swapper/220:40:142
117169930,3cyclictest6367-21ssh00:25:142
117169930,3cyclictest4110-21python19:55:202
117169930,3cyclictest30322-21expr20:50:162
117169930,3cyclictest22667-21tr19:30:142
117169930,3cyclictest22667-21tr19:30:142
117169930,3cyclictest15924-21perl20:20:162
117169930,3cyclictest13346-21cut21:45:142
117169930,3cyclictest0-21swapper/222:30:132
117169930,3cyclictest0-21swapper/222:15:132
117169930,3cyclictest0-21swapper/222:00:052
117169930,3cyclictest0-21swapper/221:55:142
117169930,3cyclictest0-21swapper/221:05:142
117169930,3cyclictest0-21swapper/220:25:152
117169930,3cyclictest0-21swapper/220:10:132
117169930,3cyclictest0-21swapper/220:01:232
117119933,0cyclictest0-21swapper/121:46:381
117119931,2cyclictest0-21swapper/119:40:131
117119931,1cyclictest0-21swapper/123:30:131
117119931,1cyclictest0-21swapper/123:25:141
117119931,1cyclictest0-21swapper/123:15:161
117119931,1cyclictest0-21swapper/122:15:131
117119931,1cyclictest0-21swapper/122:10:151
117119931,1cyclictest0-21swapper/121:20:141
117119931,1cyclictest0-21swapper/121:10:131
117119931,1cyclictest0-21swapper/121:00:101
117119931,1cyclictest0-21swapper/120:45:151
117119931,1cyclictest0-21swapper/120:40:121
117119931,1cyclictest0-21swapper/120:15:091
117119931,1cyclictest0-21swapper/100:35:011
117119931,1cyclictest0-21swapper/100:20:131
117119930,2cyclictest0-21swapper/122:55:131
117119930,2cyclictest0-21swapper/122:30:011
117119930,2cyclictest0-21swapper/121:55:141
117119930,2cyclictest0-21swapper/121:40:131
117119930,1cyclictest0-21swapper/121:25:121
117119930,1cyclictest0-21swapper/121:15:151
117079932,1cyclictest30997-21snmpd21:20:070
117079932,1cyclictest30997-21snmpd20:20:020
117079931,1cyclictest0-21swapper/021:40:110
117079930,3cyclictest61-21kswapd023:15:150
117079930,3cyclictest3366-21hddtemp_smartct21:00:140
117079930,3cyclictest32513-21cut20:55:110
117079930,3cyclictest22467-21apt-get19:30:130
117079930,3cyclictest22467-21apt-get19:30:120
117079930,3cyclictest20519-21df19:25:130
117079930,3cyclictest18908-21df22:20:150
117079930,3cyclictest18487-21hddtemp_smartct20:25:170
117079930,3cyclictest14352-21ntp_states19:15:010
117079930,3cyclictest0-21swapper/023:45:130
117079930,3cyclictest0-21swapper/020:45:140
117079930,3cyclictest0-21swapper/020:30:140
117079930,3cyclictest0-21swapper/020:15:100
117079930,3cyclictest0-21swapper/020:00:130
117079930,3cyclictest0-21swapper/019:55:100
117079930,3cyclictest0-21swapper/019:10:130
117079930,3cyclictest0-21swapper/000:00:010
117079930,2cyclictest1722-21ssh23:50:000
117079930,1cyclictest7502-21ssh23:20:100
117119921,1cyclictest0-21swapper/122:05:141
117119921,1cyclictest0-21swapper/121:05:131
117119921,1cyclictest0-21swapper/100:15:191
117119920,2cyclictest0-21swapper/119:20:161
117119920,1cyclictest0-21swapper/123:50:121
117119920,1cyclictest0-21swapper/121:35:151
117119920,1cyclictest0-21swapper/120:00:131
117119920,1cyclictest0-21swapper/119:35:021
117119920,0cyclictest0-21swapper/120:20:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional