You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 13:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Thu Nov 20, 2025 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
194442199166,28sleep00-21swapper/019:08:070
195552195179,11sleep10-21swapper/119:09:361
195622189172,12sleep30-21swapper/319:09:423
173842169149,15sleep20-21swapper/219:05:122
258682490,0sleep10-21swapper/123:15:201
287882430,0sleep20-21swapper/222:45:072
91472120,0sleep00-21swapper/000:12:480
289622110,0sleep00-21swapper/023:00:380
206982100,0sleep1211ktimersoftd/100:18:501
197639991,4cyclictest1032-21munin-run23:20:010
197729960,6cyclictest0-21swapper/223:50:152
197729960,5cyclictest61-21kswapd023:40:082
197639962,4cyclictest28151-21munin-run23:50:000
197639960,5cyclictest20471-21kworker/0:020:05:010
197639960,3cyclictest32125-21munin-run21:10:010
32415250,0sleep10-21swapper/122:46:141
197779955,0cyclictest2079-21kworker/3:200:16:003
197779955,0cyclictest1967-21kworker/3:023:20:013
197779955,0cyclictest1967-21kworker/3:022:46:273
197779955,0cyclictest1967-21kworker/3:021:50:283
197779950,5cyclictest8210-21ssh22:18:273
197779950,5cyclictest0-21swapper/322:30:203
197779950,5cyclictest0-21swapper/321:21:313
197779950,4cyclictest20781-21diskstats19:15:003
197779950,4cyclictest20694-21apt-get22:25:023
197729955,0cyclictest0-21swapper/223:12:282
197729955,0cyclictest0-21swapper/221:39:202
197729953,2cyclictest3466-21kworker/2:022:58:292
197729950,5cyclictest0-21swapper/221:49:412
197729950,4cyclictest25907-21apt-get00:05:122
197679951,2cyclictest0-21swapper/100:30:161
197679950,5cyclictest0-21swapper/123:51:081
197679950,4cyclictest0-21swapper/121:12:411
197639955,0cyclictest32117-21diskmemload22:07:140
197639955,0cyclictest0-21swapper/022:36:200
197639952,3cyclictest20471-21kworker/0:020:00:010
197639951,4cyclictest20471-21kworker/0:022:42:530
197639951,4cyclictest11141-21kworker/0:123:34:130
197639950,5cyclictest12771-21grep00:30:230
197639950,5cyclictest0-21swapper/023:53:090
197639950,5cyclictest0-21swapper/000:23:340
197639950,5cyclictest0-21swapper/000:16:440
197639950,1cyclictest11132-21munin-run23:25:000
197639950,0cyclictest0-21swapper/023:56:560
197639950,0cyclictest0-21swapper/021:38:250
197779944,0cyclictest1967-21kworker/3:022:24:033
197779944,0cyclictest1967-21kworker/3:022:01:403
197779944,0cyclictest1967-21kworker/3:020:32:063
197779944,0cyclictest1967-21kworker/3:019:24:563
197779944,0cyclictest0-21swapper/322:59:353
197779942,2cyclictest714-21snmpd19:50:433
197779942,2cyclictest29072-21ssh22:12:283
197779941,3cyclictest55950irq/130-eth021:56:533
197779941,3cyclictest0-21swapper/300:10:123
197779940,4cyclictest8967-21apt-get22:35:123
197779940,4cyclictest16671-21ssh23:10:483
197779940,4cyclictest0-21swapper/323:45:163
197779940,4cyclictest0-21swapper/323:40:093
197779940,4cyclictest0-21swapper/323:15:123
197779940,4cyclictest0-21swapper/322:50:113
197779940,4cyclictest0-21swapper/321:42:533
197779940,4cyclictest0-21swapper/321:11:273
197779940,4cyclictest0-21swapper/320:58:393
197779940,4cyclictest0-21swapper/320:16:273
197779940,4cyclictest0-21swapper/300:30:163
197779940,3cyclictest8000-21ssh21:30:113
197779940,3cyclictest6336-21sh23:05:343
197779940,3cyclictest3351-21/usr/sbin/munin20:10:173
197779940,3cyclictest32157-21apt20:05:013
197779940,3cyclictest31364-21hddtemp_smartct21:25:173
197779940,3cyclictest29895-21sed23:50:193
197779940,3cyclictest27061-21ssh21:39:423
197779940,3cyclictest24107-21ssh00:36:133
197779940,3cyclictest23072-21/usr/sbin/munin00:20:113
197779940,3cyclictest22072-21sh22:40:593
197779940,3cyclictest21332-21sh23:30:033
197779940,3cyclictest15984-21apt-get00:00:123
197779940,3cyclictest10910-21hddtemp_smartct20:25:153
197779940,1cyclictest6752-21irqstats21:45:143
197779940,1cyclictest6700-21diskstats23:55:143
197779940,1cyclictest1123-21df00:25:133
197779940,1cyclictest10903-21cut21:15:143
197779940,0cyclictest0-21swapper/323:35:053
197729943,1cyclictest31-21ksoftirqd/220:40:002
197729942,2cyclictest7884-21kworker/2:220:07:572
197729942,2cyclictest0-21swapper/200:34:062
197729942,1cyclictest3525-21ssh23:36:522
197729941,3cyclictest3400-21cat20:10:172
197729941,3cyclictest25679-21ssh21:55:092
197729940,4cyclictest5064-21ssh23:21:082
197729940,4cyclictest32117-21diskmemload23:25:142
197729940,4cyclictest0-21swapper/223:49:352
197729940,4cyclictest0-21swapper/223:30:052
197729940,4cyclictest0-21swapper/223:15:122
197729940,4cyclictest0-21swapper/223:05:112
197729940,4cyclictest0-21swapper/222:50:112
197729940,4cyclictest0-21swapper/222:40:152
197729940,4cyclictest0-21swapper/222:35:122
197729940,4cyclictest0-21swapper/222:29:112
197729940,4cyclictest0-21swapper/222:05:122
197729940,4cyclictest0-21swapper/221:40:112
197729940,4cyclictest0-21swapper/221:27:272
197729940,4cyclictest0-21swapper/221:20:122
197729940,4cyclictest0-21swapper/221:15:112
197729940,4cyclictest0-21swapper/221:12:022
197729940,4cyclictest0-21swapper/220:25:122
197729940,4cyclictest0-21swapper/219:50:092
197729940,4cyclictest0-21swapper/219:29:592
197729940,4cyclictest0-21swapper/200:23:052
197729940,4cyclictest0-21swapper/200:15:122
197729940,3cyclictest9063-21cat20:20:222
197729940,3cyclictest677-21sensors_temp22:30:202
197729940,3cyclictest6120-21/usr/sbin/munin23:55:122
197729940,3cyclictest3268-21/usr/sbin/munin00:10:172
197729940,3cyclictest25503-21cat19:50:002
197729940,3cyclictest20350-21ssh00:35:002
197729940,3cyclictest18874-21ssh00:00:522
197729940,3cyclictest16540-21perl21:50:152
197729940,3cyclictest11182-21/usr/sbin/munin22:20:142
197729940,3cyclictest0-21swapper/223:00:122
197729940,3cyclictest0-21swapper/221:10:002
197729940,1cyclictest4041-21df_inode22:00:152
197729940,1cyclictest25073-21tr22:10:162
197729940,0cyclictest0-21swapper/200:26:322
197679944,0cyclictest0-21swapper/123:35:171
197679944,0cyclictest0-21swapper/123:08:481
197679944,0cyclictest0-21swapper/100:39:531
197679944,0cyclictest0-21swapper/100:13:591
197679943,1cyclictest0-21swapper/122:25:101
197679941,2cyclictest0-21swapper/122:50:111
197679941,2cyclictest0-21swapper/121:05:131
197679941,1cyclictest0-21swapper/121:50:131
197679941,1cyclictest0-21swapper/121:35:121
197679941,1cyclictest0-21swapper/121:25:151
197679940,4cyclictest0-21swapper/123:20:091
197679940,4cyclictest0-21swapper/123:01:131
197679940,4cyclictest0-21swapper/122:06:311
197679940,4cyclictest0-21swapper/121:49:301
197679940,4cyclictest0-21swapper/121:31:451
197679940,0cyclictest0-21swapper/122:39:231
197639943,1cyclictest2096-21grep22:15:140
197639942,2cyclictest20471-21kworker/0:023:09:410
197639941,3cyclictest20471-21kworker/0:022:30:130
197639941,3cyclictest11141-21kworker/0:100:25:020
197639941,2cyclictest0-21swapper/023:40:090
197639941,2cyclictest0-21swapper/022:50:110
197639940,4cyclictest28079-21apt-get21:40:110
197639940,4cyclictest25358-21munin-run19:50:010
197639940,4cyclictest20471-21kworker/0:022:00:490
197639940,4cyclictest20471-21kworker/0:020:15:170
197639940,4cyclictest0-21swapper/023:25:140
197639940,4cyclictest0-21swapper/023:10:100
197639940,4cyclictest0-21swapper/022:49:030
197639940,4cyclictest0-21swapper/022:10:090
197639940,4cyclictest0-21swapper/021:55:120
197639940,4cyclictest0-21swapper/021:45:100
197639940,4cyclictest0-21swapper/021:30:110
197639940,4cyclictest0-21swapper/021:25:140
197639940,4cyclictest0-21swapper/021:22:160
197639940,4cyclictest0-21swapper/021:10:140
197639940,4cyclictest0-21swapper/020:50:140
197639940,4cyclictest0-21swapper/020:45:110
197639940,4cyclictest0-21swapper/000:38:470
197639940,4cyclictest0-21swapper/000:00:120
197639940,3cyclictest31447-21/usr/sbin/munin23:35:120
197639940,3cyclictest29786-21sh00:06:330
197639940,3cyclictest25805-21ssh22:59:570
197639940,3cyclictest23642-21ls19:35:010
197639940,3cyclictest22117-21perl22:25:130
197639940,1cyclictest24741-21munin-run19:45:010
197639940,1cyclictest24741-21munin-run19:45:000
29492230,0sleep10-21swapper/100:23:061
22451230,1sleep30-21swapper/320:50:143
197779931,1cyclictest27415-21aten2_r5power_c21:00:113
197779930,3cyclictest32391-21apt-get20:05:113
197779930,3cyclictest26181-21rm23:00:033
197779930,3cyclictest20194-21tr20:45:123
197779930,3cyclictest17841-21tr20:40:143
197779930,3cyclictest15012-21/usr/sbin/munin20:35:123
197779930,3cyclictest11469-21sh23:25:053
197779930,3cyclictest0-21swapper/321:05:133
197779930,3cyclictest0-21swapper/320:20:203
197779930,3cyclictest0-21swapper/319:55:063
197779930,3cyclictest0-21swapper/319:45:303
197779930,3cyclictest0-21swapper/319:41:013
197779930,3cyclictest0-21swapper/319:41:013
197779930,3cyclictest0-21swapper/319:37:253
197779930,3cyclictest0-21swapper/319:30:313
197779930,3cyclictest0-21swapper/319:25:283
197779930,3cyclictest0-21swapper/319:11:253
197779930,3cyclictest0-21swapper/300:05:023
197779930,1cyclictest13200-21ssh22:05:053
197729933,0cyclictest7884-21kworker/2:220:00:132
197729933,0cyclictest7884-21kworker/2:219:40:452
197729933,0cyclictest7884-21kworker/2:219:40:452
197729933,0cyclictest7884-21kworker/2:219:35:052
197729932,1cyclictest714-21snmpd19:20:042
197729931,2cyclictest0-21swapper/222:15:092
197729931,2cyclictest0-21swapper/221:30:062
197729930,3cyclictest0-21swapper/221:00:122
197729930,3cyclictest0-21swapper/220:55:062
197729930,3cyclictest0-21swapper/220:50:112
197729930,3cyclictest0-21swapper/220:45:112
197729930,3cyclictest0-21swapper/220:40:132
197729930,3cyclictest0-21swapper/220:30:152
197729930,3cyclictest0-21swapper/220:15:132
197729930,3cyclictest0-21swapper/219:55:072
197729930,3cyclictest0-21swapper/219:30:042
197729930,3cyclictest0-21swapper/219:15:252
197729930,3cyclictest0-21swapper/219:10:212
197679933,0cyclictest0-21swapper/123:31:301
197679933,0cyclictest0-21swapper/123:27:431
197679931,1cyclictest0-21swapper/123:55:111
197679931,1cyclictest0-21swapper/123:45:161
197679931,1cyclictest0-21swapper/123:40:091
197679931,1cyclictest0-21swapper/123:10:111
197679931,1cyclictest0-21swapper/122:55:141
197679931,1cyclictest0-21swapper/121:40:101
197679931,1cyclictest0-21swapper/121:00:121
197679931,1cyclictest0-21swapper/120:55:091
197679931,1cyclictest0-21swapper/120:50:141
197679931,1cyclictest0-21swapper/120:25:011
197679931,1cyclictest0-21swapper/120:20:141
197679931,1cyclictest0-21swapper/120:10:141
197679931,1cyclictest0-21swapper/120:00:141
197679931,1cyclictest0-21swapper/119:55:071
197679931,1cyclictest0-21swapper/100:25:121
197679931,1cyclictest0-21swapper/100:05:121
197679931,1cyclictest0-21swapper/100:00:121
197679930,3cyclictest0-21swapper/119:11:371
197679930,2cyclictest3245-21chrt22:31:441
197679930,2cyclictest303012sleep121:56:541
197679930,2cyclictest0-21swapper/122:10:091
197679930,2cyclictest0-21swapper/120:15:131
197679930,1cyclictest0-21swapper/122:00:141
197679930,1cyclictest0-21swapper/121:20:111
197639932,1cyclictest714-21snmpd19:25:580
197639932,1cyclictest714-21snmpd19:20:130
197639932,1cyclictest12540-21sshd20:30:100
197639930,3cyclictest32389-21apt20:05:110
197639930,3cyclictest25750-21hddtemp_smartct20:55:110
197639930,3cyclictest21195-21tune2fs19:15:010
197639930,3cyclictest15006-21sshd20:35:090
197639930,3cyclictest0-21swapper/022:20:110
197639930,3cyclictest0-21swapper/021:50:120
197639930,3cyclictest0-21swapper/021:15:020
197639930,3cyclictest0-21swapper/021:00:120
197639930,3cyclictest0-21swapper/020:40:130
197639930,3cyclictest0-21swapper/020:25:000
197639930,3cyclictest0-21swapper/020:20:140
197639930,3cyclictest0-21swapper/020:10:140
197639930,3cyclictest0-21swapper/019:50:090
197639930,3cyclictest0-21swapper/019:38:140
197639930,3cyclictest0-21swapper/019:10:550
197679921,1cyclictest0-21swapper/122:40:151
197679921,1cyclictest0-21swapper/120:40:131
197679921,1cyclictest0-21swapper/120:05:121
197679921,0cyclictest0-21swapper/120:45:101
197679920,2cyclictest25074-21taskset19:46:341
197679920,2cyclictest0-21swapper/119:50:091
197679920,2cyclictest0-21swapper/119:25:171
197679920,1cyclictest959-21chrt22:15:031
197679920,1cyclictest24606-21chrt19:43:301
197679920,1cyclictest24606-21chrt19:43:301
197679920,1cyclictest22166-21chrt19:23:521
197679920,1cyclictest21538-21chrt19:18:561
197679920,1cyclictest0-21swapper/122:20:111
197679920,1cyclictest0-21swapper/121:15:001
197679920,1cyclictest0-21swapper/120:35:121
197679920,1cyclictest0-21swapper/120:30:151
197679920,1cyclictest0-21swapper/119:35:001
197679910,1cyclictest0-21swapper/119:35:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional