You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-15 - 10:46
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Sat Feb 15, 2025 00:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
44422200165,13sleep30-21swapper/319:05:093
65942198182,12sleep00-21swapper/019:09:070
64982187171,11sleep10-21swapper/119:07:521
66482182164,13sleep20-21swapper/219:09:492
198772440,0sleep00-21swapper/022:14:510
171042440,0sleep10-21swapper/120:40:151
140442430,0sleep00-21swapper/021:17:060
56442420,0sleep10-21swapper/121:13:101
199192380,0sleep10-21swapper/121:38:121
683999103,4cyclictest21103-21munin-run20:50:000
225722100,0sleep00-21swapper/023:14:240
13403290,0sleep30-21swapper/323:26:243
68399960,6cyclictest0-21swapper/023:08:050
68529955,0cyclictest0-21swapper/322:32:503
68529955,0cyclictest0-21swapper/322:21:113
68529950,5cyclictest0-21swapper/323:53:263
68529950,5cyclictest0-21swapper/323:47:023
68529950,5cyclictest0-21swapper/322:44:373
68529950,5cyclictest0-21swapper/321:30:163
68529950,4cyclictest21369-21ntp_states19:35:203
68529950,0cyclictest0-21swapper/321:54:413
68529950,0cyclictest0-21swapper/321:46:253
68529950,0cyclictest0-21swapper/300:10:453
68479950,5cyclictest0-21swapper/219:19:572
68439950,5cyclictest0-21swapper/121:17:251
68439950,5cyclictest0-21swapper/100:16:071
68399952,3cyclictest31003-21diskmemload22:36:100
68399952,3cyclictest0-21swapper/023:56:260
68399952,3cyclictest0-21swapper/023:52:420
68399952,3cyclictest0-21swapper/023:47:220
68399952,3cyclictest0-21swapper/021:31:420
68399952,3cyclictest0-21swapper/000:17:100
68399951,4cyclictest0-21swapper/022:59:460
68399950,5cyclictest0-21swapper/023:31:500
68399950,5cyclictest0-21swapper/023:21:100
68399950,5cyclictest0-21swapper/023:17:260
68399950,5cyclictest0-21swapper/022:53:020
68399950,5cyclictest0-21swapper/022:30:060
68399950,5cyclictest0-21swapper/022:27:380
68399950,5cyclictest0-21swapper/022:20:140
68399950,5cyclictest0-21swapper/022:15:210
68399950,5cyclictest0-21swapper/022:07:420
68399950,5cyclictest0-21swapper/022:01:420
68399950,5cyclictest0-21swapper/021:55:300
68399950,5cyclictest0-21swapper/021:51:380
68399950,5cyclictest0-21swapper/021:45:060
68399950,5cyclictest0-21swapper/021:36:300
68399950,5cyclictest0-21swapper/021:28:050
68399950,5cyclictest0-21swapper/021:22:570
68399950,5cyclictest0-21swapper/021:14:140
68399950,5cyclictest0-21swapper/000:38:420
68399950,5cyclictest0-21swapper/000:32:450
68399950,5cyclictest0-21swapper/000:25:220
68399950,5cyclictest0-21swapper/000:21:220
68399950,5cyclictest0-21swapper/000:06:330
68399950,5cyclictest0-21swapper/000:01:340
68399950,1cyclictest0-21swapper/023:25:580
68399950,1cyclictest0-21swapper/023:02:020
68399950,1cyclictest0-21swapper/021:43:500
28346250,0sleep30-21swapper/322:18:533
68529944,0cyclictest0-21swapper/322:57:253
68529943,1cyclictest1105-21ssh00:30:063
68529940,4cyclictest571-21ntp_states21:10:223
68529940,4cyclictest31003-21diskmemload23:36:453
68529940,4cyclictest16452-21perl00:20:143
68529940,4cyclictest11220-21apt-get20:30:073
68529940,4cyclictest0-21swapper/323:55:143
68529940,4cyclictest0-21swapper/323:30:253
68529940,4cyclictest0-21swapper/323:20:083
68529940,4cyclictest0-21swapper/323:06:113
68529940,4cyclictest0-21swapper/323:00:393
68529940,4cyclictest0-21swapper/322:45:393
68529940,4cyclictest0-21swapper/322:28:023
68529940,4cyclictest0-21swapper/322:13:013
68529940,4cyclictest0-21swapper/322:08:233
68529940,4cyclictest0-21swapper/322:01:583
68529940,4cyclictest0-21swapper/321:55:063
68529940,4cyclictest0-21swapper/321:23:063
68529940,4cyclictest0-21swapper/321:18:533
68529940,4cyclictest0-21swapper/320:35:213
68529940,4cyclictest0-21swapper/319:25:203
68529940,4cyclictest0-21swapper/300:39:373
68529940,4cyclictest0-21swapper/300:29:373
68529940,4cyclictest0-21swapper/300:17:103
68529940,4cyclictest0-21swapper/300:05:243
68529940,3cyclictest8804-21perl20:20:173
68529940,3cyclictest5807-21grep20:15:133
68529940,3cyclictest4489-21ssh23:40:053
68529940,3cyclictest31003-21diskmemload23:15:013
68529940,3cyclictest31003-21diskmemload22:51:533
68529940,3cyclictest30612-21date20:00:003
68529940,3cyclictest23173-21users20:50:233
68529940,3cyclictest1964-21ntp_states20:05:183
68529940,3cyclictest10592-21df00:00:143
68529940,3cyclictest0-21swapper/321:45:023
68529940,3cyclictest0-21swapper/319:50:073
68529940,0cyclictest0-21swapper/319:20:193
68479944,0cyclictest0-21swapper/223:05:052
68479944,0cyclictest0-21swapper/222:29:162
68479944,0cyclictest0-21swapper/222:11:412
68479944,0cyclictest0-21swapper/221:15:482
68479944,0cyclictest0-21swapper/221:02:332
68479943,1cyclictest57250irq/125-eth022:20:002
68479943,1cyclictest17632-21/usr/sbin/munin19:30:182
68479943,1cyclictest16888-21sed19:25:212
68479941,3cyclictest15785-21ssh21:35:362
68479940,4cyclictest25467-21awk19:49:592
68479940,4cyclictest23796-21/usr/sbin/munin20:55:192
68479940,4cyclictest0-21swapper/223:51:302
68479940,4cyclictest0-21swapper/223:47:332
68479940,4cyclictest0-21swapper/223:35:382
68479940,4cyclictest0-21swapper/223:23:272
68479940,4cyclictest0-21swapper/222:03:092
68479940,4cyclictest0-21swapper/221:57:162
68479940,4cyclictest0-21swapper/221:46:422
68479940,4cyclictest0-21swapper/221:40:112
68479940,4cyclictest0-21swapper/221:13:572
68479940,4cyclictest0-21swapper/219:35:152
68479940,4cyclictest0-21swapper/219:14:172
68479940,4cyclictest0-21swapper/200:35:112
68479940,4cyclictest0-21swapper/200:30:142
68479940,4cyclictest0-21swapper/200:26:062
68479940,4cyclictest0-21swapper/200:15:112
68479940,3cyclictest9601-21ssh21:31:522
68479940,3cyclictest5776-21/usr/sbin/munin23:40:092
68479940,3cyclictest3905-21sh21:29:372
68479940,3cyclictest2953-21rm23:55:522
68479940,3cyclictest28317-21/usr/sbin/munin19:55:182
68479940,3cyclictest23728-21latency_hist20:55:002
68479940,3cyclictest22588-21/usr/sbin/munin19:40:152
68479940,3cyclictest20245-21perl23:30:102
68479940,3cyclictest16386-21sh22:31:252
68479940,3cyclictest15657-21aten2_r5power_c00:20:122
68479940,1cyclictest3169-21/usr/sbin/munin22:07:082
68479940,1cyclictest3162-21/usr/sbin/munin20:10:232
68479940,1cyclictest22577-21/usr/sbin/munin22:35:132
68479940,1cyclictest0-21swapper/221:20:122
68479940,0cyclictest0-21swapper/220:35:292
68439944,0cyclictest0-21swapper/123:35:231
68439944,0cyclictest0-21swapper/123:11:131
68439944,0cyclictest0-21swapper/122:55:041
68439944,0cyclictest0-21swapper/121:56:001
68439944,0cyclictest0-21swapper/121:43:111
68439942,1cyclictest0-21swapper/123:30:081
68439942,1cyclictest0-21swapper/120:20:141
68439941,2cyclictest0-21swapper/123:50:121
68439941,2cyclictest0-21swapper/120:55:111
68439941,2cyclictest0-21swapper/120:50:141
68439941,2cyclictest0-21swapper/120:30:081
68439941,2cyclictest0-21swapper/120:15:121
68439941,2cyclictest0-21swapper/120:10:151
68439941,2cyclictest0-21swapper/120:05:131
68439941,2cyclictest0-21swapper/119:50:071
68439941,2cyclictest0-21swapper/100:30:071
68439941,1cyclictest0-21swapper/119:25:111
68439940,4cyclictest0-21swapper/122:12:441
68439940,0cyclictest0-21swapper/123:40:111
68439940,0cyclictest0-21swapper/123:03:561
68439940,0cyclictest0-21swapper/122:53:281
68439940,0cyclictest0-21swapper/122:05:531
68439940,0cyclictest0-21swapper/121:32:081
68399943,1cyclictest25460-21perf19:50:000
68399941,3cyclictest0-21swapper/023:40:540
68399941,3cyclictest0-21swapper/022:40:530
68399941,2cyclictest30997-21snmpd20:05:260
68399941,2cyclictest0-21swapper/020:30:070
68399941,2cyclictest0-21swapper/019:20:130
68399940,4cyclictest31003-21diskmemload22:45:340
68399940,4cyclictest0-21swapper/023:35:210
68399940,4cyclictest0-21swapper/020:50:100
68399940,4cyclictest0-21swapper/020:26:420
68399940,4cyclictest0-21swapper/020:20:340
68399940,4cyclictest0-21swapper/020:10:100
68399940,4cyclictest0-21swapper/020:02:370
68399940,4cyclictest0-21swapper/019:30:140
68399940,4cyclictest0-21swapper/019:15:150
68399940,4cyclictest0-21swapper/000:10:070
68399940,3cyclictest7675-21perl19:10:150
68399940,3cyclictest23339-21grep19:40:160
68399940,0cyclictest0-21swapper/019:36:220
68529932,1cyclictest30997-21snmpd20:25:173
68529932,1cyclictest30997-21snmpd19:15:023
68529931,1cyclictest0-21swapper/320:45:123
68529931,0cyclictest0-21swapper/319:40:133
68529930,3cyclictest7473-21df19:10:143
68529930,3cyclictest31003-21diskmemload23:10:343
68529930,3cyclictest31003-21diskmemload22:35:023
68529930,3cyclictest29290-21diskstats21:05:133
68529930,3cyclictest0-21swapper/321:35:063
68529930,3cyclictest0-21swapper/321:25:143
68529930,3cyclictest0-21swapper/321:00:103
68529930,3cyclictest0-21swapper/320:55:113
68529930,3cyclictest0-21swapper/320:40:123
68529930,3cyclictest0-21swapper/320:10:153
68529930,3cyclictest0-21swapper/320:00:153
68529930,3cyclictest0-21swapper/319:30:153
68529930,2cyclictest252392sleep319:47:293
68479932,1cyclictest8319-21ssh22:45:352
68479932,1cyclictest24656-21ssh22:55:322
68479931,1cyclictest30997-21snmpd22:50:422
68479930,3cyclictest32311-21ssh23:00:152
68479930,3cyclictest30645-21/usr/sbin/munin20:00:212
68479930,3cyclictest20239-21ssh00:05:232
68479930,3cyclictest12692-21apt-get19:20:132
68479930,3cyclictest12063-21smartctl20:30:112
68479930,3cyclictest11237-21ssh23:25:012
68479930,3cyclictest0-21swapper/223:10:072
68479930,3cyclictest0-21swapper/222:40:042
68479930,3cyclictest0-21swapper/221:05:112
68479930,3cyclictest0-21swapper/220:45:122
68479930,3cyclictest0-21swapper/220:40:002
68479930,3cyclictest0-21swapper/220:25:512
68479930,3cyclictest0-21swapper/220:20:152
68479930,3cyclictest0-21swapper/220:15:112
68479930,3cyclictest0-21swapper/220:05:132
68479930,3cyclictest0-21swapper/219:50:072
68479930,3cyclictest0-21swapper/200:10:012
68479930,3cyclictest0-21swapper/200:00:132
68479930,1cyclictest8285-21ssh21:50:032
68479930,1cyclictest30725-21ssh22:20:142
68479930,1cyclictest23913-21ssh23:15:062
68439933,0cyclictest0-21swapper/122:23:211
68439933,0cyclictest0-21swapper/122:02:501
68439933,0cyclictest0-21swapper/121:00:241
68439931,2cyclictest0-21swapper/121:50:071
68439931,1cyclictest0-21swapper/123:55:131
68439931,1cyclictest0-21swapper/123:20:081
68439931,1cyclictest0-21swapper/122:20:001
68439931,1cyclictest0-21swapper/121:45:101
68439931,1cyclictest0-21swapper/120:00:151
68439931,1cyclictest0-21swapper/119:55:161
68439931,1cyclictest0-21swapper/119:15:131
68439931,1cyclictest0-21swapper/119:10:121
68439931,1cyclictest0-21swapper/100:35:111
68439930,3cyclictest0-21swapper/120:36:501
68439930,2cyclictest4504-21chrt22:43:181
68439930,2cyclictest19559-21chrt20:45:141
68439930,2cyclictest146962sleep119:22:121
68439930,2cyclictest0-21swapper/123:15:201
68439930,2cyclictest0-21swapper/122:46:051
68439930,2cyclictest0-21swapper/100:20:111
68439930,2cyclictest0-21swapper/100:00:131
68399930,3cyclictest29482-21perl21:05:140
68399930,3cyclictest0-21swapper/021:00:110
68399930,3cyclictest0-21swapper/020:55:100
68399930,3cyclictest0-21swapper/020:40:010
68399930,3cyclictest0-21swapper/020:35:120
68399930,3cyclictest0-21swapper/020:15:120
68399930,3cyclictest0-21swapper/019:55:170
68399930,3cyclictest0-21swapper/019:50:090
68399930,3cyclictest0-21swapper/019:25:120
13320230,0sleep10-21swapper/122:29:381
68439921,1cyclictest107462chrt20:27:021
68439921,1cyclictest0-21swapper/121:20:121
68439921,1cyclictest0-21swapper/121:05:121
68439921,1cyclictest0-21swapper/100:10:011
68439920,2cyclictest8766-21taskset23:05:441
68439920,2cyclictest24510-21taskset19:41:061
68439920,1cyclictest26872-21chrt00:09:371
68439920,1cyclictest25446-21chrt00:25:071
68439920,1cyclictest15450-21chrt22:30:491
68439920,1cyclictest0-21swapper/123:45:051
68439920,1cyclictest0-21swapper/123:25:041
68439920,1cyclictest0-21swapper/122:35:101
68439920,1cyclictest0-21swapper/121:25:091
68439920,1cyclictest0-21swapper/119:35:151
68439920,1cyclictest0-21swapper/119:30:151
68439910,1cyclictest0-21swapper/119:45:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional