You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-07 - 03:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Sun Dec 07, 2025 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
284152205173,28sleep30-21swapper/319:08:333
283312199183,12sleep10-21swapper/119:07:271
281742190174,11sleep20-21swapper/219:05:272
284172184168,11sleep00-21swapper/019:08:340
121672470,0sleep10-21swapper/122:24:441
246482460,0sleep20-21swapper/222:46:342
84592430,0sleep30-21swapper/323:44:053
226082430,0sleep30-21swapper/300:23:563
170552410,0sleep00-21swapper/000:37:060
133032370,0sleep10-21swapper/123:45:501
290282140,0sleep10-21swapper/123:37:501
173652130,0sleep20-21swapper/200:05:012
76752110,0sleep10-21swapper/123:43:341
2871899110,1cyclictest8153-21ssh00:16:042
199502110,0sleep20-21swapper/221:23:432
79242100,0sleep10-21swapper/122:05:391
287189982,5cyclictest3331-21grep23:25:112
287189981,1cyclictest6034-21forks22:05:152
287189981,1cyclictest2625-21awk20:25:142
287189981,0cyclictest0-21swapper/223:46:122
287189980,8cyclictest0-21swapper/221:18:332
287189980,2cyclictest3214-21diskmemload22:20:132
287189980,2cyclictest0-21swapper/219:55:122
287189980,1cyclictest9142-21ntp_states00:00:182
287189980,1cyclictest7566-21munin-run22:55:012
287189980,1cyclictest32123-21ssh23:23:142
287189980,1cyclictest31798-21munin-run20:20:002
287189980,1cyclictest24371-21munin-run20:05:002
287189980,1cyclictest20212-21ssh21:40:142
287189980,1cyclictest19373-21awk21:00:002
287189980,1cyclictest17822-21ssh21:55:112
287189980,1cyclictest15721-21expr22:10:162
287189980,1cyclictest14421-21munin-run20:50:012
287129981,3cyclictest24324-21awk00:25:000
287189971,1cyclictest808-21snmpd20:21:372
287189971,1cyclictest0-21swapper/219:45:132
287189971,0cyclictest24314-21diskmemload00:30:132
287189970,7cyclictest9301-21latency_hist19:35:002
287189970,7cyclictest0-21swapper/221:46:192
287189970,1cyclictest9494-21ssh23:11:322
287189970,1cyclictest9257-21sed22:55:182
287189970,1cyclictest7745-21ssh21:50:102
287189970,1cyclictest7585-21sshd21:33:432
287189970,1cyclictest4475-21taskset23:09:402
287189970,1cyclictest4351-21cron19:25:012
287189970,1cyclictest31650-21ntp_states23:55:172
287189970,1cyclictest28899-21wget19:10:122
287189970,1cyclictest28456-21ssh00:10:162
287189970,1cyclictest27526-21ssh22:15:242
287189970,1cyclictest27247-21grep20:10:162
287189970,1cyclictest26848-21awk20:10:002
287189970,1cyclictest25311-21ls00:25:152
287189970,1cyclictest24326-21ssh21:10:012
287189970,1cyclictest24306-21sed22:30:162
287189970,1cyclictest23739-21ssh23:35:152
287189970,1cyclictest22556-21ssh21:25:132
287189970,1cyclictest21836-21turbostat.cron21:05:002
287189970,1cyclictest20394-21ssh23:50:122
287189970,1cyclictest19507-21ssh23:00:182
287189970,1cyclictest1870-21sh19:20:002
287189970,1cyclictest16282-21grep23:15:182
287189970,1cyclictest15527-21cut00:20:142
287189970,1cyclictest15227-21cut20:50:152
287189970,1cyclictest13780-21df22:25:172
287189970,1cyclictest13537-21needreboot00:35:162
287189970,1cyclictest13114-21ssh23:30:112
287189970,1cyclictest12086-21ntpq22:40:182
287189970,1cyclictest1109-21diskstats23:40:132
287189970,1cyclictest10944-21cut21:35:142
287189970,1cyclictest1083-21wc22:35:122
287189970,1cyclictest10807-21ntp_states19:35:182
287129973,4cyclictest14380-21munin-run00:20:000
287189960,1cyclictest9524-21sort20:40:012
287189960,1cyclictest7015-21/usr/sbin/munin20:35:112
287189960,1cyclictest5125-21diskstats19:25:152
287189960,1cyclictest4724-21grep20:30:142
287189960,1cyclictest27872-21/usr/sbin/munin22:00:152
287189960,1cyclictest27872-21/usr/sbin/munin22:00:142
287189960,1cyclictest25461-21rm21:10:162
287189960,1cyclictest17149-21chrt19:50:042
287189960,1cyclictest12026-21apt19:40:132
287129960,3cyclictest29743-21munin-run23:55:000
287219951,2cyclictest0-21swapper/321:20:123
287219951,2cyclictest0-21swapper/300:30:133
287219950,4cyclictest7593-21df_inode19:30:153
287169951,2cyclictest0-21swapper/123:20:131
287169951,2cyclictest0-21swapper/100:00:121
287169950,5cyclictest0-21swapper/123:17:261
287129955,0cyclictest21432-21kworker/0:223:46:580
287129955,0cyclictest21432-21kworker/0:223:24:340
287129955,0cyclictest21432-21kworker/0:222:17:250
287129955,0cyclictest21432-21kworker/0:221:43:510
287129955,0cyclictest21432-21kworker/0:220:47:500
287129955,0cyclictest21432-21kworker/0:220:14:160
287129955,0cyclictest21432-21kworker/0:219:18:160
287129950,5cyclictest22167-21ntp_states22:45:220
287129950,5cyclictest0-21swapper/023:18:170
287129950,5cyclictest0-21swapper/021:33:240
287129950,4cyclictest0-21swapper/019:50:120
287129950,0cyclictest0-21swapper/021:51:260
287219944,0cyclictest0-21swapper/323:48:513
287219943,1cyclictest1858-21cron19:20:013
287219942,2cyclictest56750irq/130-eth022:51:053
287219942,1cyclictest0-21swapper/300:00:123
287219941,3cyclictest26172-21sh22:47:343
287219941,2cyclictest0-21swapper/320:25:123
287219940,4cyclictest7798-21ssh22:05:333
287219940,4cyclictest56750irq/130-eth000:10:133
287219940,4cyclictest0-21swapper/322:42:203
287219940,4cyclictest0-21swapper/322:15:143
287219940,4cyclictest0-21swapper/321:40:133
287219940,4cyclictest0-21swapper/321:16:133
287219940,4cyclictest0-21swapper/321:12:253
287219940,4cyclictest0-21swapper/300:38:473
287219940,3cyclictest7492-21date21:50:013
287219940,3cyclictest6967-21ntp_states23:10:193
287219940,3cyclictest5479-21cat19:25:163
287219940,3cyclictest4548-21/usr/sbin/munin20:30:183
287219940,3cyclictest3968-21users19:20:213
287219940,3cyclictest3916-21hddtemp_smartct23:25:133
287219940,3cyclictest3779-21df_inode22:20:153
287219940,3cyclictest322922sleep321:30:023
287219940,3cyclictest29805-21irqstats19:10:173
287219940,3cyclictest26745-21sh23:04:333
287219940,3cyclictest26226-21perl23:20:153
287219940,3cyclictest24562-21ssh00:25:013
287219940,3cyclictest23133-21ssh21:25:153
287219940,3cyclictest21273-21hddtemp_smartct23:50:153
287219940,3cyclictest2063-21ntp_states22:35:173
287219940,3cyclictest18981-21memory21:55:163
287219940,3cyclictest18295-21ssh23:15:483
287219940,3cyclictest17587-21ssh22:26:513
287219940,3cyclictest173962sleep300:05:033
287219940,3cyclictest15617-21if_eth420:50:173
287219940,3cyclictest13869-21ssh23:30:143
287219940,3cyclictest13128-21ssh21:35:553
287219940,3cyclictest10567-21sh21:50:473
287219940,3cyclictest10178-21users22:55:213
287219940,2cyclictest31982-21ssh22:01:403
287219940,2cyclictest31982-21ssh22:01:403
287219940,2cyclictest0-21swapper/320:45:143
287219940,1cyclictest872-21ntp_states20:20:183
287219940,1cyclictest27065-21/usr/sbin/munin20:10:213
287219940,1cyclictest24592-21/usr/sbin/munin20:05:193
287219940,1cyclictest24403-21cut21:10:013
287219940,1cyclictest23062-21/usr/sbin/munin23:35:223
287219940,1cyclictest19630-21/usr/sbin/munin19:55:163
287219940,1cyclictest17247-21tr19:50:133
287219940,1cyclictest14552-21tr19:45:143
287169944,0cyclictest0-21swapper/122:30:001
287169944,0cyclictest0-21swapper/121:42:521
287169944,0cyclictest0-21swapper/119:10:221
287169944,0cyclictest0-21swapper/100:24:411
287169944,0cyclictest0-21swapper/100:09:401
287169941,2cyclictest0-21swapper/122:10:151
287169941,2cyclictest0-21swapper/121:20:131
287169941,2cyclictest0-21swapper/119:30:141
287169940,4cyclictest0-21swapper/123:11:201
287169940,4cyclictest0-21swapper/122:53:431
287169940,4cyclictest0-21swapper/121:30:041
287169940,4cyclictest0-21swapper/100:19:251
287169940,2cyclictest0-21swapper/122:35:121
287169940,0cyclictest0-21swapper/123:08:161
287169940,0cyclictest0-21swapper/121:38:211
287129944,0cyclictest21432-21kworker/0:222:50:590
287129944,0cyclictest21432-21kworker/0:221:21:280
287129944,0cyclictest21432-21kworker/0:220:25:260
287129944,0cyclictest21432-21kworker/0:220:03:030
287129944,0cyclictest21432-21kworker/0:219:29:280
287129944,0cyclictest0-21swapper/023:35:020
287129944,0cyclictest0-21swapper/021:16:040
287129943,1cyclictest6721-21grep22:05:180
287129942,2cyclictest21432-21kworker/0:221:28:340
287129942,1cyclictest19113-21ssh22:27:490
287129940,4cyclictest7546-21diskstats20:35:140
287129940,4cyclictest24314-21diskmemload00:29:240
287129940,4cyclictest0-21swapper/023:59:340
287129940,4cyclictest0-21swapper/023:28:110
287129940,4cyclictest0-21swapper/022:20:130
287129940,4cyclictest0-21swapper/022:00:080
287129940,4cyclictest0-21swapper/022:00:080
287129940,4cyclictest0-21swapper/021:58:070
287129940,4cyclictest0-21swapper/021:35:120
287129940,4cyclictest0-21swapper/020:55:130
287129940,4cyclictest0-21swapper/020:30:140
287129940,4cyclictest0-21swapper/019:35:120
287129940,4cyclictest0-21swapper/019:20:120
287129940,4cyclictest0-21swapper/000:30:140
287129940,4cyclictest0-21swapper/000:10:280
287129940,3cyclictest704-21cat20:20:170
287129940,3cyclictest322-21ssh21:45:410
287129940,3cyclictest30526-21irqstats20:15:170
287129940,3cyclictest24021-21ssh22:30:150
287129940,3cyclictest22727-21grep21:05:150
287129940,3cyclictest22500-21munin-run23:35:000
287129940,3cyclictest20164-21diskstats19:55:140
287129940,3cyclictest17680-21ssh00:05:140
287129940,3cyclictest13538-21sensors_temp19:40:220
287129940,3cyclictest1232-21df_inode22:35:130
287129940,3cyclictest0-21swapper/023:10:010
287129940,0cyclictest0-21swapper/023:02:170
15684240,1sleep3381rcuc/322:10:163
32128230,0sleep10-21swapper/122:01:461
32128230,0sleep10-21swapper/122:01:461
287219932,1cyclictest4753-21ssh00:15:033
287219932,1cyclictest22877-21ssh22:30:033
287219931,1cyclictest0-21swapper/320:00:133
287219930,3cyclictest195962sleep321:00:103
287219930,3cyclictest0-21swapper/323:55:053
287219930,3cyclictest0-21swapper/323:05:113
287219930,3cyclictest0-21swapper/320:55:043
287219930,3cyclictest0-21swapper/320:40:023
287219930,3cyclictest0-21swapper/320:35:113
287219930,3cyclictest0-21swapper/320:15:043
287219930,3cyclictest0-21swapper/319:40:113
287219930,3cyclictest0-21swapper/319:35:143
287169933,0cyclictest0-21swapper/123:33:271
287169932,1cyclictest0-21swapper/123:57:371
287169931,1cyclictest22092chrt21:15:021
287169931,1cyclictest0-21swapper/121:50:131
287169931,1cyclictest0-21swapper/121:45:131
287169931,1cyclictest0-21swapper/121:10:151
287169931,1cyclictest0-21swapper/121:05:121
287169931,1cyclictest0-21swapper/120:55:141
287169931,1cyclictest0-21swapper/120:45:141
287169931,1cyclictest0-21swapper/120:25:111
287169931,1cyclictest0-21swapper/120:20:121
287169931,1cyclictest0-21swapper/120:05:121
287169931,1cyclictest0-21swapper/119:55:121
287169931,1cyclictest0-21swapper/119:45:131
287169931,1cyclictest0-21swapper/100:35:111
287169931,1cyclictest0-21swapper/100:30:131
287169930,2cyclictest28836-21chrt22:49:141
287169930,2cyclictest27501-21chrt23:53:351
287169930,2cyclictest1772-21chrt00:13:171
287169930,2cyclictest167182sleep120:53:071
287169930,2cyclictest0-21swapper/122:30:131
287169930,2cyclictest0-21swapper/122:15:121
287169930,2cyclictest0-21swapper/121:25:141
287169930,2cyclictest0-21swapper/120:40:151
287169930,2cyclictest0-21swapper/120:35:121
287169930,2cyclictest0-21swapper/120:15:121
287169930,2cyclictest0-21swapper/120:10:161
287169930,2cyclictest0-21swapper/119:35:121
287169930,2cyclictest0-21swapper/119:25:131
287169930,0cyclictest0-21swapper/119:20:071
287129931,2cyclictest0-21swapper/022:10:080
287129930,3cyclictest7923-21sh22:55:050
287129930,3cyclictest1094-21df23:40:130
287129930,3cyclictest10487-21ssh22:40:060
287129930,3cyclictest0-21swapper/023:05:120
287129930,3cyclictest0-21swapper/021:10:150
287129930,3cyclictest0-21swapper/021:00:140
287129930,3cyclictest0-21swapper/020:50:010
287129930,3cyclictest0-21swapper/020:05:050
287129930,3cyclictest0-21swapper/019:45:130
287129930,3cyclictest0-21swapper/019:10:120
287129930,3cyclictest0-21swapper/000:00:110
287129930,2cyclictest7050-21taskset19:30:010
287129930,0cyclictest9257-21apt-get20:40:020
21982230,0sleep10-21swapper/121:05:001
287169922,0cyclictest0-21swapper/120:30:071
287169921,1cyclictest0-21swapper/123:00:131
287169921,1cyclictest0-21swapper/122:55:121
287169921,1cyclictest0-21swapper/119:50:011
287169921,1cyclictest0-21swapper/119:15:111
287169921,1cyclictest0-21swapper/100:25:141
287169920,1cyclictest22116-21taskset20:00:081
287169920,1cyclictest0-21swapper/123:25:111
287169920,1cyclictest0-21swapper/122:40:131
287169920,1cyclictest0-21swapper/121:55:111
287169920,1cyclictest0-21swapper/119:40:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional