You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-05 - 05:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Mon May 05, 2025 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
296272200185,10sleep10-21swapper/119:06:471
298152187169,13sleep00-21swapper/019:09:150
276622180163,12sleep30-21swapper/319:05:093
297502171154,13sleep20-21swapper/219:08:262
95412440,0sleep00-21swapper/021:34:160
201172440,0sleep30-21swapper/321:55:203
56312400,0sleep10-21swapper/122:53:481
97182100,0sleep20-21swapper/221:18:012
18577290,0sleep10-21swapper/123:16:221
300609982,3cyclictest17420-21awk23:00:010
300649971,5cyclictest0-21swapper/223:30:132
300649971,5cyclictest0-21swapper/223:30:132
300649971,5cyclictest0-21swapper/220:55:162
300609972,5cyclictest12746-21cron00:20:000
300609971,3cyclictest5718-21perf19:25:000
300609970,6cyclictest0-21swapper/022:15:140
300649960,1cyclictest18773-21sh23:16:282
300609960,1cyclictest30264-21rm23:39:210
6516250,0sleep30-21swapper/323:26:493
300699955,0cyclictest25382-21diskmemload23:04:093
300699955,0cyclictest0-21swapper/323:32:043
300699955,0cyclictest0-21swapper/323:32:043
300699955,0cyclictest0-21swapper/322:44:033
300699950,5cyclictest0-21swapper/323:22:463
300699950,5cyclictest0-21swapper/321:22:433
300649955,0cyclictest0-21swapper/223:10:132
300649954,1cyclictest3755-21ssh21:47:092
300649950,5cyclictest0-21swapper/222:31:392
300649950,5cyclictest0-21swapper/219:45:142
300649950,5cyclictest0-21swapper/200:30:002
300649950,2cyclictest0-21swapper/222:20:152
300619951,2cyclictest0-21swapper/121:40:161
300619950,5cyclictest0-21swapper/122:49:591
300619950,5cyclictest0-21swapper/122:08:301
300619950,5cyclictest0-21swapper/100:36:041
300609953,1cyclictest2066-21head20:20:230
300609950,5cyclictest0-21swapper/022:05:140
300609950,4cyclictest840-21apt-get22:35:010
300609950,0cyclictest0-21swapper/000:26:210
18529250,0sleep30-21swapper/319:50:143
9901240,0sleep00-21swapper/000:01:230
300699944,0cyclictest0-21swapper/323:44:263
300699944,0cyclictest0-21swapper/300:11:133
300699943,1cyclictest9262-21ssh23:45:143
300699943,1cyclictest7168-21ntp_kernel_err19:25:193
300699942,1cyclictest18859-21ssh00:22:373
300699941,2cyclictest0-21swapper/320:00:153
300699941,2cyclictest0-21swapper/319:45:143
300699940,4cyclictest3999-21diskstats19:20:173
300699940,4cyclictest2139-21perl21:30:183
300699940,4cyclictest17639-21cut22:10:213
300699940,4cyclictest15915-21apt-get00:05:003
300699940,4cyclictest0-21swapper/323:50:143
300699940,4cyclictest0-21swapper/323:17:513
300699940,4cyclictest0-21swapper/322:50:263
300699940,4cyclictest0-21swapper/322:32:013
300699940,4cyclictest0-21swapper/322:15:143
300699940,4cyclictest0-21swapper/321:46:583
300699940,4cyclictest0-21swapper/321:40:023
300699940,4cyclictest0-21swapper/320:40:143
300699940,4cyclictest0-21swapper/300:30:143
300699940,4cyclictest0-21swapper/300:15:133
300699940,3cyclictest9812-21ssh22:06:283
300699940,3cyclictest9572-21ssh22:55:223
300699940,3cyclictest8929-21sh23:11:273
300699940,3cyclictest7486-21users20:30:243
300699940,3cyclictest7037-21expr00:00:153
300699940,3cyclictest699-21dpkg22:35:013
300699940,3cyclictest3322-21ssh22:20:023
300699940,3cyclictest26670-21perl21:10:163
300699940,3cyclictest25382-21diskmemload23:38:383
300699940,3cyclictest25382-21diskmemload22:25:443
300699940,3cyclictest17999-21/usr/sbin/munin20:55:173
300699940,3cyclictest12422-21grep21:35:183
300699940,1cyclictest3213-21sh21:15:043
300699940,1cyclictest30037-21memory23:55:173
300699940,0cyclictest0-21swapper/322:00:433
300649944,0cyclictest0-21swapper/221:11:032
300649943,1cyclictest9452-21tr20:35:202
300649943,1cyclictest13654-21/usr/sbin/munin22:25:172
300649942,1cyclictest1607-21ssh22:02:312
300649942,1cyclictest0-21swapper/221:50:162
300649941,3cyclictest0-21swapper/200:12:422
300649941,2cyclictest0-21swapper/222:55:162
300649941,2cyclictest0-21swapper/222:15:152
300649941,2cyclictest0-21swapper/220:50:132
300649941,2cyclictest0-21swapper/219:35:142
300649941,1cyclictest0-21swapper/219:50:122
300649940,4cyclictest21752-21apt-get23:35:022
300649940,4cyclictest0-21swapper/223:50:132
300649940,4cyclictest0-21swapper/223:05:132
300649940,4cyclictest0-21swapper/222:10:332
300649940,4cyclictest0-21swapper/200:35:102
300649940,4cyclictest0-21swapper/200:25:262
300649940,3cyclictest8544-21taskset00:17:182
300649940,3cyclictest7415-21sh22:38:042
300649940,3cyclictest7079-21sh21:32:422
300649940,3cyclictest5211-21sh23:25:582
300649940,3cyclictest419-21latency_hist20:20:002
300649940,3cyclictest28740-21ssh23:55:102
300649940,3cyclictest21745-21fw_packets22:45:192
300649940,3cyclictest19710-21ssh21:55:182
300649940,3cyclictest19492-21sh21:22:522
300649940,3cyclictest14298-21ssh22:09:202
300649940,3cyclictest12871-21sh00:03:202
300649940,3cyclictest1094-21sh22:50:512
300649940,2cyclictest28860-21ssh21:27:222
300649940,2cyclictest10435-21ssh23:45:182
300649940,2cyclictest0-21swapper/200:20:572
300649940,1cyclictest31798-21/usr/sbin/munin23:40:122
300649940,1cyclictest23150-21/usr/sbin/munin21:05:212
300649940,1cyclictest21473-21/usr/sbin/munin21:40:232
300619944,0cyclictest0-21swapper/123:50:391
300619944,0cyclictest0-21swapper/123:05:331
300619944,0cyclictest0-21swapper/122:44:351
300619944,0cyclictest0-21swapper/119:20:271
300619944,0cyclictest0-21swapper/100:26:441
300619944,0cyclictest0-21swapper/100:01:251
300619942,1cyclictest0-21swapper/100:05:181
300619941,2cyclictest0-21swapper/123:55:131
300619941,2cyclictest0-21swapper/123:35:021
300619941,2cyclictest0-21swapper/123:00:121
300619941,2cyclictest0-21swapper/122:35:011
300619941,2cyclictest0-21swapper/122:00:141
300619941,2cyclictest0-21swapper/121:55:161
300619941,2cyclictest0-21swapper/121:30:171
300619941,2cyclictest0-21swapper/121:15:151
300619941,2cyclictest0-21swapper/120:50:131
300619941,2cyclictest0-21swapper/120:45:131
300619941,2cyclictest0-21swapper/120:00:141
300619941,2cyclictest0-21swapper/119:35:131
300619941,2cyclictest0-21swapper/100:15:111
300619941,1cyclictest0-21swapper/123:45:141
300619941,1cyclictest0-21swapper/100:30:151
300619940,4cyclictest0-21swapper/123:10:121
300619940,4cyclictest0-21swapper/122:27:081
300619940,4cyclictest0-21swapper/121:13:391
300619940,4cyclictest0-21swapper/121:01:351
300619940,4cyclictest0-21swapper/120:39:281
300619940,0cyclictest0-21swapper/121:46:581
300609944,0cyclictest0-21swapper/023:59:260
300609944,0cyclictest0-21swapper/023:33:060
300609944,0cyclictest0-21swapper/023:33:060
300609944,0cyclictest0-21swapper/023:22:120
300609944,0cyclictest0-21swapper/022:10:100
300609944,0cyclictest0-21swapper/021:59:410
300609944,0cyclictest0-21swapper/021:21:340
300609943,1cyclictest3-21ksoftirqd/022:50:200
300609943,1cyclictest3-21ksoftirqd/020:45:140
300609942,1cyclictest18027-21smartctl00:05:240
300609942,1cyclictest0-21swapper/021:29:550
300609940,4cyclictest30165-21apt-get19:10:130
300609940,4cyclictest0-21swapper/023:45:150
300609940,4cyclictest0-21swapper/023:15:120
300609940,4cyclictest0-21swapper/022:03:170
300609940,4cyclictest0-21swapper/021:40:150
300609940,4cyclictest0-21swapper/020:55:170
300609940,4cyclictest0-21swapper/019:40:110
300609940,4cyclictest0-21swapper/019:35:130
300609940,4cyclictest0-21swapper/000:10:130
300609940,3cyclictest6855-21ssh00:32:550
300609940,3cyclictest6674-21perl19:25:170
300609940,3cyclictest6085-21ssh21:15:420
300609940,3cyclictest606-21users19:15:010
300609940,3cyclictest26867-21ssh22:47:540
300609940,3cyclictest22246-21ssh00:24:470
300609940,3cyclictest15087-21ssh22:25:190
300609940,3cyclictest15072-21rm21:53:020
300609940,3cyclictest12673-21/usr/sbin/munin21:35:190
300609940,3cyclictest11868-21ntp_states20:40:200
300609940,3cyclictest1173-21ssh21:45:300
300609940,3cyclictest0-21swapper/023:52:270
300609940,1cyclictest25472-21latency_hist20:05:000
300609940,0cyclictest0-21swapper/023:02:420
300609940,0cyclictest0-21swapper/000:35:400
16346240,0sleep10-21swapper/122:10:151
300699933,0cyclictest0-21swapper/321:00:013
300699932,1cyclictest30997-21snmpd19:55:063
300699931,1cyclictest0-21swapper/320:20:163
300699930,3cyclictest8775-21perl20:35:153
300699930,3cyclictest30643-21seq20:15:163
300699930,3cyclictest25382-21diskmemload00:35:023
300699930,3cyclictest23089-21apt-config21:05:003
300699930,3cyclictest0-21swapper/323:05:033
300699930,3cyclictest0-21swapper/322:45:053
300699930,3cyclictest0-21swapper/321:50:083
300699930,3cyclictest0-21swapper/321:25:013
300699930,3cyclictest0-21swapper/320:50:133
300699930,3cyclictest0-21swapper/320:45:023
300699930,3cyclictest0-21swapper/320:10:153
300699930,3cyclictest0-21swapper/320:05:173
300699930,3cyclictest0-21swapper/319:40:113
300699930,3cyclictest0-21swapper/319:35:053
300699930,3cyclictest0-21swapper/319:30:143
300699930,3cyclictest0-21swapper/319:15:113
300699930,3cyclictest0-21swapper/319:10:033
300699930,3cyclictest0-21swapper/300:25:033
300699930,2cyclictest3176-21/usr/sbin/munin20:25:213
300649932,0cyclictest5934-21apt-get19:25:142
300649931,2cyclictest57250irq/125-eth000:05:002
300649931,2cyclictest2927-21kworker/2:219:40:022
300649931,1cyclictest0-21swapper/221:00:182
300649931,1cyclictest0-21swapper/220:40:142
300649931,1cyclictest0-21swapper/220:25:152
300649931,1cyclictest0-21swapper/220:10:152
300649930,3cyclictest9104-21grep19:30:162
300649930,3cyclictest4986-21python19:20:222
300649930,3cyclictest31255-21memory19:10:182
300649930,3cyclictest25709-21/usr/sbin/munin20:05:192
300649930,3cyclictest21275-21df_inode19:55:142
300649930,3cyclictest1489-21df19:15:142
300649930,3cyclictest13594-21df_inode20:45:152
300649930,3cyclictest0-21swapper/223:20:012
300649930,3cyclictest0-21swapper/222:40:142
300649930,3cyclictest0-21swapper/220:30:162
300649930,3cyclictest0-21swapper/220:20:152
300649930,3cyclictest0-21swapper/220:00:152
300649930,1cyclictest17829-21ssh23:00:092
300649930,1cyclictest10977-21ssh21:35:012
300619933,0cyclictest0-21swapper/120:15:531
300619933,0cyclictest0-21swapper/119:27:171
300619932,1cyclictest321162sleep122:34:331
300619931,2cyclictest0-21swapper/100:20:121
300619931,1cyclictest0-21swapper/123:40:121
300619931,1cyclictest0-21swapper/123:30:121
300619931,1cyclictest0-21swapper/123:30:121
300619931,1cyclictest0-21swapper/123:25:121
300619931,1cyclictest0-21swapper/122:55:161
300619931,1cyclictest0-21swapper/122:15:021
300619931,1cyclictest0-21swapper/120:55:141
300619931,1cyclictest0-21swapper/120:30:151
300619931,1cyclictest0-21swapper/119:55:121
300619931,1cyclictest0-21swapper/119:50:121
300619931,1cyclictest0-21swapper/119:40:121
300619931,1cyclictest0-21swapper/119:30:141
300619931,1cyclictest0-21swapper/119:10:141
300619931,1cyclictest0-21swapper/100:10:141
300619930,3cyclictest0-21swapper/120:07:001
300619930,2cyclictest20116-21chrt21:23:151
300609932,1cyclictest5033-21ssh23:10:070
300609932,1cyclictest30997-21snmpd20:15:110
300609931,1cyclictest0-21swapper/020:35:120
300609930,3cyclictest3425-21ssh22:20:060
300609930,3cyclictest0-21swapper/023:40:050
300609930,3cyclictest0-21swapper/023:25:070
300609930,3cyclictest0-21swapper/023:05:120
300609930,3cyclictest0-21swapper/022:40:130
300609930,3cyclictest0-21swapper/022:30:130
300609930,3cyclictest0-21swapper/021:10:130
300609930,3cyclictest0-21swapper/021:05:010
300609930,3cyclictest0-21swapper/021:00:170
300609930,3cyclictest0-21swapper/020:50:130
300609930,3cyclictest0-21swapper/020:30:150
300609930,3cyclictest0-21swapper/020:25:150
300609930,3cyclictest0-21swapper/020:10:150
300609930,3cyclictest0-21swapper/020:05:050
300609930,3cyclictest0-21swapper/019:55:140
300609930,3cyclictest0-21swapper/019:50:130
300609930,3cyclictest0-21swapper/019:45:020
300609930,3cyclictest0-21swapper/019:30:030
300619921,1cyclictest0-21swapper/122:20:131
300619921,1cyclictest0-21swapper/120:10:161
300619921,1cyclictest0-21swapper/119:15:141
300619921,0cyclictest0-21swapper/120:25:151
300619920,2cyclictest15811-21chrt19:45:031
300619920,1cyclictest0-21swapper/123:20:141
300619920,1cyclictest0-21swapper/121:50:161
300619920,1cyclictest0-21swapper/121:35:131
300619920,1cyclictest0-21swapper/121:25:021
300619920,1cyclictest0-21swapper/121:05:021
300619920,1cyclictest0-21swapper/120:40:151
300619920,1cyclictest0-21swapper/120:20:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional