You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 10:35
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Sun Nov 16, 2025 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
209902204188,11sleep10-21swapper/119:09:521
207592189171,13sleep00-21swapper/019:06:540
207352187170,12sleep30-21swapper/319:06:363
187752169151,13sleep20-21swapper/219:05:042
262652480,0sleep10-21swapper/122:51:251
291962470,0sleep00-21swapper/022:53:180
150312440,0sleep20-21swapper/222:13:402
12652440,0sleep30-21swapper/300:17:083
74402130,0sleep10-21swapper/122:58:461
282662120,0sleep328268-21ntp_kernel_pll_19:20:183
137572120,0sleep30-21swapper/323:18:073
2117099102,4cyclictest13146-21ssh00:40:000
211709992,3cyclictest9403-21apt-get23:00:010
211709991,6cyclictest0-21swapper/022:15:120
4112280,0sleep00-21swapper/022:08:010
22967280,0sleep20-21swapper/223:22:382
211779970,7cyclictest0-21swapper/222:33:452
211709972,5cyclictest20729-21kworker/0:122:49:150
211709972,5cyclictest20729-21kworker/0:121:51:160
211709970,7cyclictest20729-21kworker/0:122:34:230
211819960,5cyclictest22748-21diskstats20:15:163
211779965,1cyclictest0-21swapper/223:57:242
211759961,3cyclictest0-21swapper/122:40:131
211709962,4cyclictest61-21kswapd023:20:000
211709962,4cyclictest20729-21kworker/0:123:36:270
211709962,4cyclictest20729-21kworker/0:123:32:460
211709962,4cyclictest20729-21kworker/0:123:12:270
211709962,4cyclictest20729-21kworker/0:122:11:000
211709962,4cyclictest20729-21kworker/0:121:46:040
211709962,4cyclictest20729-21kworker/0:121:20:160
211709962,4cyclictest20729-21kworker/0:100:31:470
211709962,4cyclictest20729-21kworker/0:100:03:390
211709961,5cyclictest20729-21kworker/0:123:57:380
211709961,5cyclictest20729-21kworker/0:123:50:160
211709961,5cyclictest20729-21kworker/0:123:46:470
211709961,5cyclictest20729-21kworker/0:123:27:260
211709961,5cyclictest20729-21kworker/0:123:02:500
211709961,5cyclictest20729-21kworker/0:122:35:140
211709961,5cyclictest20729-21kworker/0:122:20:200
211709961,5cyclictest20729-21kworker/0:122:02:480
211709961,5cyclictest20729-21kworker/0:121:40:070
211709961,5cyclictest20729-21kworker/0:121:30:220
211709961,5cyclictest20729-21kworker/0:121:29:340
211709961,5cyclictest20729-21kworker/0:121:15:190
211709961,5cyclictest20729-21kworker/0:121:11:360
211709961,5cyclictest20729-21kworker/0:120:15:150
211709961,5cyclictest20729-21kworker/0:100:25:230
211709961,5cyclictest20729-21kworker/0:100:05:590
211819955,0cyclictest0-21swapper/322:10:223
211819955,0cyclictest0-21swapper/321:21:373
211819954,1cyclictest31631-21/usr/sbin/munin00:00:213
211819951,4cyclictest0-21swapper/320:05:013
211819950,5cyclictest7772-21ntp_states00:20:183
211819950,5cyclictest0-21swapper/321:34:263
211819950,4cyclictest0-21swapper/321:25:123
211819950,3cyclictest0-21swapper/322:43:293
211819950,3cyclictest0-21swapper/300:27:563
211779954,1cyclictest31-21ksoftirqd/223:10:122
211779954,1cyclictest11737-21ssh23:16:492
211779950,5cyclictest0-21swapper/223:43:252
211779950,5cyclictest0-21swapper/222:21:042
211779950,5cyclictest0-21swapper/221:35:122
211779950,0cyclictest0-21swapper/223:25:412
211779950,0cyclictest0-21swapper/221:27:142
211759951,2cyclictest0-21swapper/119:15:151
211759951,2cyclictest0-21swapper/119:15:151
211759950,5cyclictest0-21swapper/123:36:351
211759950,5cyclictest0-21swapper/123:14:191
211759950,5cyclictest0-21swapper/122:39:151
211759950,5cyclictest0-21swapper/100:33:291
211709952,3cyclictest20729-21kworker/0:120:50:140
211709952,3cyclictest20729-21kworker/0:119:20:230
211709952,3cyclictest20729-21kworker/0:100:21:020
211709952,3cyclictest20729-21kworker/0:100:15:220
211709952,3cyclictest20729-21kworker/0:100:10:190
211709951,4cyclictest20729-21kworker/0:123:40:310
211709951,4cyclictest20729-21kworker/0:123:21:510
211709951,4cyclictest20729-21kworker/0:123:05:350
211709951,4cyclictest20729-21kworker/0:122:40:150
211709951,4cyclictest20729-21kworker/0:122:28:160
211709951,4cyclictest20729-21kworker/0:121:36:120
211709950,5cyclictest20729-21kworker/0:121:55:240
211709950,5cyclictest20729-21kworker/0:120:05:190
211709950,5cyclictest20729-21kworker/0:119:55:220
211709950,0cyclictest20729-21kworker/0:119:15:150
211709950,0cyclictest20729-21kworker/0:119:15:150
25049240,3sleep20-21swapper/219:15:172
25049240,3sleep20-21swapper/219:15:172
211819944,0cyclictest0-21swapper/323:12:133
211819944,0cyclictest0-21swapper/322:59:333
211819944,0cyclictest0-21swapper/322:46:543
211819944,0cyclictest0-21swapper/319:35:143
211819943,1cyclictest10521-21ntp_states00:05:193
211819941,2cyclictest15606-21cat23:35:173
211819940,4cyclictest7640-21perl19:45:173
211819940,4cyclictest3608-21perl21:35:143
211819940,4cyclictest21250-21sh22:16:193
211819940,4cyclictest15240-21/usr/sbin/munin22:30:213
211819940,4cyclictest14805-21/usr/sbin/munin21:05:213
211819940,4cyclictest0-21swapper/323:21:053
211819940,4cyclictest0-21swapper/322:52:093
211819940,4cyclictest0-21swapper/322:25:143
211819940,4cyclictest0-21swapper/322:20:143
211819940,4cyclictest0-21swapper/322:05:123
211819940,4cyclictest0-21swapper/321:55:223
211819940,4cyclictest0-21swapper/321:51:423
211819940,4cyclictest0-21swapper/321:15:073
211819940,4cyclictest0-21swapper/320:55:223
211819940,4cyclictest0-21swapper/320:40:163
211819940,4cyclictest0-21swapper/300:34:313
211819940,3cyclictest744-21sh23:44:363
211819940,3cyclictest4725-21sh23:45:493
211819940,3cyclictest4537-21aten2_r5power_e19:40:153
211819940,3cyclictest31962-21apt-get19:30:113
211819940,3cyclictest23883-21munin-node19:15:013
211819940,3cyclictest23883-21munin-node19:15:013
211819940,3cyclictest23036-21/usr/sbin/munin21:45:143
211819940,3cyclictest12966-21ssh21:40:153
211819940,3cyclictest12409-21df_inode23:50:143
211819940,2cyclictest0-21swapper/320:48:463
211819940,1cyclictest7176-21/usr/sbin/munin20:50:153
211819940,1cyclictest4483-21fschecks_time00:35:163
211819940,1cyclictest21482-21irqstats22:00:203
211819940,1cyclictest21400-21ntp_states23:05:163
211819940,1cyclictest19075-21/usr/sbin/munin00:10:233
211819940,1cyclictest14392-21/usr/sbin/munin20:00:143
211779944,0cyclictest0-21swapper/222:54:332
211779944,0cyclictest0-21swapper/222:15:062
211779944,0cyclictest0-21swapper/200:10:392
211779943,1cyclictest8330-21needreboot20:50:172
211779943,1cyclictest5206-21cut19:40:162
211779943,1cyclictest31-21ksoftirqd/221:10:122
211779942,1cyclictest14702-21ssh22:45:232
211779941,3cyclictest12601-21ssh00:06:162
211779940,4cyclictest3479-21perl22:40:152
211779940,4cyclictest0-21swapper/222:55:562
211779940,4cyclictest0-21swapper/222:03:572
211779940,4cyclictest0-21swapper/221:55:122
211779940,4cyclictest0-21swapper/221:50:132
211779940,4cyclictest0-21swapper/221:00:112
211779940,4cyclictest0-21swapper/200:30:132
211779940,4cyclictest0-21swapper/200:21:482
211779940,4cyclictest0-21swapper/200:00:112
211779940,3cyclictest6030-21ls00:20:002
211779940,3cyclictest5398-21sh21:20:122
211779940,3cyclictest21778-21rm21:44:392
211779940,3cyclictest16034-21ntp_states23:35:192
211779940,3cyclictest0-21swapper/221:15:022
211779940,1cyclictest31151-21hddtemp_smartct22:05:152
211779940,1cyclictest22065-21users23:05:202
211779940,1cyclictest11005-21ssh23:33:032
211779940,1cyclictest0-21swapper/222:25:512
211779940,0cyclictest0-21swapper/223:00:312
211759944,0cyclictest0-21swapper/123:53:371
211759944,0cyclictest0-21swapper/122:28:021
211759944,0cyclictest0-21swapper/121:52:071
211759944,0cyclictest0-21swapper/121:24:201
211759944,0cyclictest0-21swapper/121:13:411
211759944,0cyclictest0-21swapper/100:21:411
211759941,2cyclictest0-21swapper/123:55:131
211759941,2cyclictest0-21swapper/123:15:121
211759941,2cyclictest0-21swapper/122:15:121
211759941,2cyclictest0-21swapper/121:35:111
211759941,2cyclictest0-21swapper/120:45:151
211759941,2cyclictest0-21swapper/120:20:141
211759941,2cyclictest0-21swapper/119:50:121
211759941,2cyclictest0-21swapper/119:30:131
211759941,2cyclictest0-21swapper/119:25:161
211759941,2cyclictest0-21swapper/119:10:151
211759940,4cyclictest0-21swapper/121:07:351
211759940,4cyclictest0-21swapper/100:12:041
211759940,2cyclictest0-21swapper/100:00:121
211759940,0cyclictest0-21swapper/122:22:051
211759940,0cyclictest0-21swapper/122:11:371
211709941,3cyclictest20729-21kworker/0:120:35:150
211709941,3cyclictest20729-21kworker/0:120:34:550
211709941,3cyclictest20729-21kworker/0:120:25:140
211709941,3cyclictest20729-21kworker/0:120:20:140
211709941,3cyclictest20729-21kworker/0:120:00:140
211709941,3cyclictest20729-21kworker/0:119:45:230
211709941,3cyclictest20729-21kworker/0:119:25:230
211709940,4cyclictest21295-21apt-get19:10:150
211709940,4cyclictest20729-21kworker/0:119:50:150
211709940,4cyclictest0-21swapper/021:05:140
211709940,4cyclictest0-21swapper/019:40:150
211709940,3cyclictest9818-21cat20:55:000
211709940,3cyclictest21016-21ntp_states20:10:210
25815230,0sleep10-21swapper/123:07:441
211819931,1cyclictest0-21swapper/319:25:173
211819930,3cyclictest9964-21df_inode19:50:143
211819930,3cyclictest27891-21grep20:25:153
211819930,3cyclictest24914-21sh22:35:033
211819930,3cyclictest21589-21ssh23:55:063
211819930,3cyclictest13170-21hddtemp_smartct21:00:153
211819930,3cyclictest0-21swapper/323:30:063
211819930,3cyclictest0-21swapper/321:10:063
211819930,3cyclictest0-21swapper/320:35:143
211819930,3cyclictest0-21swapper/320:30:173
211819930,3cyclictest0-21swapper/320:20:143
211819930,3cyclictest0-21swapper/320:10:023
211819930,3cyclictest0-21swapper/319:55:093
211819930,3cyclictest0-21swapper/319:10:073
211819930,2cyclictest26964-21sh23:25:023
211819930,1cyclictest9995-21ssh23:00:063
211779933,0cyclictest31-21ksoftirqd/220:04:592
211779933,0cyclictest19303-21apt-get20:10:022
211779933,0cyclictest17908-21kworker/2:219:10:132
211779932,1cyclictest714-21snmpd19:55:012
211779932,1cyclictest17908-21kworker/2:220:40:052
211779931,1cyclictest0-21swapper/220:25:142
211779930,3cyclictest32125-21apt-get20:35:142
211779930,3cyclictest25236-21df20:20:152
211779930,3cyclictest25190-21ssh21:30:072
211779930,3cyclictest25166-21ssh22:35:132
211779930,3cyclictest15639-21hddtemp_smartct21:05:162
211779930,3cyclictest11626-21rm23:50:042
211779930,3cyclictest0-21swapper/223:45:132
211779930,3cyclictest0-21swapper/220:55:002
211779930,3cyclictest0-21swapper/220:45:142
211779930,3cyclictest0-21swapper/220:30:162
211779930,3cyclictest0-21swapper/220:15:142
211779930,3cyclictest0-21swapper/220:00:112
211779930,3cyclictest0-21swapper/219:50:132
211779930,3cyclictest0-21swapper/219:45:142
211779930,3cyclictest0-21swapper/219:30:122
211779930,3cyclictest0-21swapper/219:25:082
211779930,3cyclictest0-21swapper/219:20:132
211779930,3cyclictest0-21swapper/200:35:022
211779930,3cyclictest0-21swapper/200:25:102
211779930,2cyclictest2384-21/usr/sbin/munin19:35:162
211779930,2cyclictest22583-21ssh21:45:022
211759933,0cyclictest0-21swapper/122:45:131
211759933,0cyclictest0-21swapper/120:05:091
211759931,2cyclictest0-21swapper/122:00:151
211759931,1cyclictest0-21swapper/123:40:121
211759931,1cyclictest0-21swapper/123:30:121
211759931,1cyclictest0-21swapper/123:00:021
211759931,1cyclictest0-21swapper/122:30:011
211759931,1cyclictest0-21swapper/122:05:131
211759931,1cyclictest0-21swapper/121:45:151
211759931,1cyclictest0-21swapper/121:40:151
211759931,1cyclictest0-21swapper/121:30:131
211759931,1cyclictest0-21swapper/121:25:121
211759931,1cyclictest0-21swapper/121:15:141
211759931,1cyclictest0-21swapper/120:50:121
211759931,1cyclictest0-21swapper/120:40:131
211759931,1cyclictest0-21swapper/120:15:131
211759931,1cyclictest0-21swapper/120:00:111
211759931,1cyclictest0-21swapper/119:40:151
211759931,1cyclictest0-21swapper/100:35:121
211759931,1cyclictest0-21swapper/100:25:141
211759930,2cyclictest8887-21chrt00:05:051
211759930,2cyclictest29525-21chrt23:25:291
211759930,2cyclictest0-21swapper/120:10:141
211759930,2cyclictest0-21swapper/119:55:151
211759930,2cyclictest0-21swapper/119:45:141
211759930,2cyclictest0-21swapper/119:20:121
211759930,0cyclictest0-21swapper/121:01:281
211709930,3cyclictest2197-21apt-get20:40:140
211709930,3cyclictest0-21swapper/021:00:120
211709930,3cyclictest0-21swapper/020:45:080
211709930,3cyclictest0-21swapper/019:35:150
211709930,3cyclictest0-21swapper/019:30:110
211759921,1cyclictest0-21swapper/120:35:151
211759921,1cyclictest0-21swapper/120:30:171
211759921,1cyclictest0-21swapper/119:35:151
211759920,1cyclictest28693-21chrt00:15:051
211759920,1cyclictest0-21swapper/123:45:121
211759920,1cyclictest0-21swapper/123:20:121
211759920,1cyclictest0-21swapper/121:55:131
211759920,1cyclictest0-21swapper/120:55:011
211759920,1cyclictest0-21swapper/120:25:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional