You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 21:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Sat Nov 22, 2025 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
293142209191,13sleep10-21swapper/107:08:401
292112204189,11sleep00-21swapper/007:07:190
291942191178,9sleep30-21swapper/307:07:053
290792171152,14sleep20-21swapper/207:05:352
89482460,0sleep10-21swapper/112:15:171
304322420,0sleep30-21swapper/312:25:523
274722330,0sleep00-21swapper/011:35:200
215202110,0sleep20-21swapper/211:16:132
2958799102,4cyclictest12642-21sh08:44:590
27752100,0sleep30-21swapper/311:40:003
225482100,0sleep10-21swapper/109:40:161
7685290,0sleep10-21swapper/110:20:401
4932290,0sleep00-21swapper/011:40:210
295879992,1cyclictest225-21systemd-journal07:55:000
295879991,4cyclictest27593-21sh11:20:010
295879990,1cyclictest5227-21munin-run08:29:590
20230290,0sleep10-21swapper/110:27:161
295879982,1cyclictest8059-21ssh09:17:030
25533280,0sleep00-21swapper/009:25:210
295879972,1cyclictest23790-21ssh10:29:320
295879970,2cyclictest7417-21munin-run11:10:010
295879970,2cyclictest19824-21latency_hist12:05:000
295929960,6cyclictest0-21swapper/212:20:122
295879961,1cyclictest7141-21ssh09:32:300
16195260,0sleep10-21swapper/108:50:151
5777250,0sleep00-21swapper/010:20:140
295979955,0cyclictest32144-21kworker/3:111:25:033
295979955,0cyclictest32144-21kworker/3:110:51:283
295979955,0cyclictest32144-21kworker/3:109:21:563
295979955,0cyclictest32144-21kworker/3:109:10:443
295979955,0cyclictest32144-21kworker/3:108:59:303
295979955,0cyclictest32144-21kworker/3:108:48:193
295979955,0cyclictest32144-21kworker/3:108:37:083
295979955,0cyclictest32144-21kworker/3:108:25:553
295979955,0cyclictest32144-21kworker/3:108:03:323
295979955,0cyclictest32144-21kworker/3:107:29:573
295979955,0cyclictest32144-21kworker/3:107:18:473
295979955,0cyclictest11435-21kworker/3:212:32:133
295979955,0cyclictest11435-21kworker/3:212:09:513
295979950,5cyclictest0-21swapper/308:10:143
295929955,0cyclictest0-21swapper/209:44:532
295929953,2cyclictest11741-21kworker/2:010:20:332
295929950,5cyclictest28130-21ssh09:10:412
295929950,5cyclictest0-21swapper/211:11:452
295929950,5cyclictest0-21swapper/210:44:592
295929950,5cyclictest0-21swapper/207:30:132
295889951,2cyclictest0-21swapper/111:20:121
295879953,1cyclictest2760-21ssh10:35:130
295879950,5cyclictest0-21swapper/011:55:120
295879950,5cyclictest0-21swapper/010:52:170
295879950,0cyclictest0-21swapper/012:05:020
295979944,0cyclictest0-21swapper/311:45:033
295979944,0cyclictest0-21swapper/311:01:243
295979944,0cyclictest0-21swapper/311:01:233
295979943,1cyclictest15689-21tr08:50:133
295979942,1cyclictest3342-21diskstats10:35:163
295979942,1cyclictest18253-21tune2fs10:10:163
295979941,1cyclictest0-21swapper/307:55:133
295979941,1cyclictest0-21swapper/307:35:123
295979940,4cyclictest25941-21ssh12:24:193
295979940,4cyclictest23551-21df_inode11:50:153
295979940,4cyclictest20308-21apt-get09:00:133
295979940,4cyclictest16321-21memory10:25:153
295979940,4cyclictest0-21swapper/312:36:113
295979940,4cyclictest0-21swapper/312:18:183
295979940,4cyclictest0-21swapper/311:40:073
295979940,4cyclictest0-21swapper/310:45:163
295979940,4cyclictest0-21swapper/310:20:123
295979940,4cyclictest0-21swapper/310:06:253
295979940,4cyclictest0-21swapper/309:55:143
295979940,4cyclictest0-21swapper/309:52:083
295979940,4cyclictest0-21swapper/309:45:023
295979940,4cyclictest0-21swapper/309:05:133
295979940,4cyclictest0-21swapper/308:17:423
295979940,4cyclictest0-21swapper/307:10:133
295979940,3cyclictest6962-21sh09:16:233
295979940,3cyclictest4053-21/usr/sbin/munin07:20:213
295979940,3cyclictest32721-21sh09:29:383
295979940,3cyclictest32057-21sh10:33:253
295979940,3cyclictest31588-21sh10:16:363
295979940,3cyclictest30869-21sh11:20:493
295979940,3cyclictest30707-21ssh09:44:353
295979940,3cyclictest21534-21cut09:40:023
295979940,3cyclictest19231-21irqstats11:15:183
295979940,3cyclictest18463-21fschecks_count07:50:183
295979940,3cyclictest17720-21sh11:30:233
295979940,3cyclictest1608-21irqstats11:55:163
295979940,3cyclictest13567-21sh11:12:283
295979940,3cyclictest11522-21irqstats12:00:153
295979940,1cyclictest30334-21/usr/sbin/munin12:10:173
295979940,1cyclictest15400-21/usr/sbin/munin07:45:193
295979940,1cyclictest15400-21/usr/sbin/munin07:45:183
295979940,1cyclictest12928-21/usr/sbin/munin07:40:223
295979940,1cyclictest10192-21/usr/sbin/munin10:55:203
295979940,0cyclictest0-21swapper/309:32:123
295929944,0cyclictest31-21ksoftirqd/211:50:132
295929944,0cyclictest31-21ksoftirqd/211:00:132
295929944,0cyclictest31-21ksoftirqd/211:00:122
295929944,0cyclictest0-21swapper/210:06:512
295929944,0cyclictest0-21swapper/209:25:052
295929943,1cyclictest31-21ksoftirqd/210:35:132
295929943,1cyclictest28002-21tr08:10:132
295929943,1cyclictest11741-21kworker/2:007:56:062
295929942,2cyclictest11741-21kworker/2:012:37:442
295929942,2cyclictest11741-21kworker/2:007:51:582
295929941,3cyclictest714-21snmpd09:23:522
295929941,3cyclictest11741-21kworker/2:008:40:122
295929940,4cyclictest6967-21python08:30:202
295929940,4cyclictest27897-21ssh12:08:492
295929940,4cyclictest10967-21diskstats07:35:152
295929940,4cyclictest0-21swapper/212:30:432
295929940,4cyclictest0-21swapper/212:25:552
295929940,4cyclictest0-21swapper/211:44:122
295929940,4cyclictest0-21swapper/211:37:502
295929940,4cyclictest0-21swapper/211:05:142
295929940,4cyclictest0-21swapper/210:55:122
295929940,4cyclictest0-21swapper/210:50:122
295929940,4cyclictest0-21swapper/210:48:022
295929940,4cyclictest0-21swapper/210:15:132
295929940,4cyclictest0-21swapper/210:12:502
295929940,4cyclictest0-21swapper/209:55:142
295929940,4cyclictest0-21swapper/209:31:452
295929940,4cyclictest0-21swapper/209:15:162
295929940,4cyclictest0-21swapper/209:05:122
295929940,4cyclictest0-21swapper/208:25:122
295929940,4cyclictest0-21swapper/207:20:152
295929940,3cyclictest8149-21ssh09:49:262
295929940,3cyclictest7882-21ssh12:15:132
295929940,3cyclictest5397-21ssh11:57:112
295929940,3cyclictest28088-21/usr/sbin/munin11:20:212
295929940,3cyclictest27700-21ssh10:30:402
295929940,3cyclictest2530-21ssh12:11:592
295929940,3cyclictest22475-21rm10:28:412
295929940,3cyclictest21231-21if_eth009:00:172
295929940,3cyclictest1627-21ssh10:01:472
295929940,3cyclictest13905-21users09:35:242
295929940,3cyclictest13742-21df11:45:182
295929940,3cyclictest12898-21ssh09:51:042
295929940,3cyclictest10139-21ssh12:00:002
295889944,0cyclictest0-21swapper/112:30:171
295889944,0cyclictest0-21swapper/111:53:281
295889944,0cyclictest0-21swapper/111:45:011
295889944,0cyclictest0-21swapper/111:36:361
295889944,0cyclictest0-21swapper/110:55:511
295889944,0cyclictest0-21swapper/110:50:291
295889942,1cyclictest0-21swapper/111:55:121
295889942,1cyclictest0-21swapper/107:35:121
295889941,2cyclictest0-21swapper/112:20:111
295889941,2cyclictest0-21swapper/112:05:131
295889941,2cyclictest0-21swapper/111:25:121
295889941,2cyclictest0-21swapper/110:45:151
295889941,2cyclictest0-21swapper/109:55:151
295889941,2cyclictest0-21swapper/109:35:151
295889941,2cyclictest0-21swapper/109:25:141
295889941,2cyclictest0-21swapper/109:10:111
295889941,2cyclictest0-21swapper/107:15:131
295889941,2cyclictest0-21swapper/107:10:141
295889941,1cyclictest0-21swapper/112:10:131
295889941,1cyclictest0-21swapper/111:05:131
295889941,1cyclictest0-21swapper/110:35:141
295889940,4cyclictest0-21swapper/111:17:231
295889940,4cyclictest0-21swapper/111:12:011
295879943,1cyclictest6723-21ssh11:25:160
295879943,1cyclictest19145-21munin-run09:55:010
295879942,2cyclictest714-21snmpd07:34:150
295879942,1cyclictest0-21swapper/010:40:120
295879941,2cyclictest0-21swapper/008:10:140
295879940,4cyclictest3030-21apt-get07:20:160
295879940,4cyclictest25017-21diskmemload12:37:490
295879940,4cyclictest25017-21diskmemload10:47:080
295879940,4cyclictest25017-21diskmemload10:10:130
295879940,4cyclictest21726-21ssh09:40:080
295879940,4cyclictest0-21swapper/012:28:190
295879940,4cyclictest0-21swapper/012:15:130
295879940,4cyclictest0-21swapper/011:50:130
295879940,4cyclictest0-21swapper/011:20:210
295879940,4cyclictest0-21swapper/011:00:120
295879940,4cyclictest0-21swapper/011:00:120
295879940,4cyclictest0-21swapper/010:30:150
295879940,4cyclictest0-21swapper/010:16:220
295879940,4cyclictest0-21swapper/009:11:060
295879940,4cyclictest0-21swapper/008:15:120
295879940,3cyclictest9922-21/usr/sbin/munin11:10:210
295879940,3cyclictest8091-21diskstats10:05:150
295879940,3cyclictest31706-21nfs4_client12:10:190
295879940,3cyclictest29712-21apt07:10:130
295879940,3cyclictest20269-21df_inode09:55:150
295879940,3cyclictest14447-21python07:40:210
295879940,3cyclictest13182-21/usr/sbin/munin11:45:250
295879940,3cyclictest10966-21diskstats07:35:140
29631230,0sleep10-21swapper/110:15:241
295979930,3cyclictest0-21swapper/311:05:023
295979930,3cyclictest0-21swapper/310:00:093
295979930,3cyclictest0-21swapper/308:30:123
295979930,3cyclictest0-21swapper/308:20:053
295979930,3cyclictest0-21swapper/308:05:143
295979930,3cyclictest0-21swapper/307:30:133
295979930,1cyclictest12485-21ssh10:40:043
295979930,0cyclictest0-21swapper/308:40:073
295929933,0cyclictest31-21ksoftirqd/207:15:012
295929933,0cyclictest11741-21kworker/2:008:55:042
295929931,1cyclictest12931-21apt-get07:40:132
295929931,1cyclictest0-21swapper/208:45:122
295929930,3cyclictest5622-21apt-get11:25:122
295929930,3cyclictest15403-21apt-get07:45:162
295929930,3cyclictest15403-21apt-get07:45:162
295929930,3cyclictest0-21swapper/208:50:122
295929930,3cyclictest0-21swapper/208:35:062
295929930,3cyclictest0-21swapper/208:20:132
295929930,3cyclictest0-21swapper/208:15:122
295929930,3cyclictest0-21swapper/208:05:132
295929930,3cyclictest0-21swapper/208:00:122
295929930,3cyclictest0-21swapper/207:25:122
295929930,3cyclictest0-21swapper/207:10:132
295929930,1cyclictest16322-21grep11:30:152
295889933,0cyclictest0-21swapper/109:06:261
295889933,0cyclictest0-21swapper/108:47:301
295889933,0cyclictest0-21swapper/108:01:151
295889931,2cyclictest0-21swapper/111:40:121
295889931,2cyclictest0-21swapper/107:25:121
295889931,1cyclictest0-21swapper/111:30:121
295889931,1cyclictest0-21swapper/111:00:121
295889931,1cyclictest0-21swapper/111:00:121
295889931,1cyclictest0-21swapper/110:30:141
295889931,1cyclictest0-21swapper/110:05:021
295889931,1cyclictest0-21swapper/109:50:151
295889931,1cyclictest0-21swapper/109:45:171
295889931,1cyclictest0-21swapper/109:30:121
295889931,1cyclictest0-21swapper/109:15:151
295889931,1cyclictest0-21swapper/109:00:131
295889931,1cyclictest0-21swapper/108:20:131
295889931,1cyclictest0-21swapper/107:50:001
295889931,1cyclictest0-21swapper/107:50:001
295889931,1cyclictest0-21swapper/107:30:121
295889930,3cyclictest0-21swapper/107:42:371
295889930,2cyclictest0-21swapper/112:25:121
295889930,2cyclictest0-21swapper/112:00:121
295889930,2cyclictest0-21swapper/110:40:121
295889930,2cyclictest0-21swapper/110:00:131
295889930,2cyclictest0-21swapper/109:20:131
295889930,2cyclictest0-21swapper/107:55:131
295889930,2cyclictest0-21swapper/107:20:151
295889930,1cyclictest0-21swapper/108:25:131
295879933,0cyclictest0-21swapper/009:45:130
295879932,1cyclictest714-21snmpd12:30:020
295879932,1cyclictest17625-21apt-get12:20:110
295879930,3cyclictest29661-21ssh10:00:120
295879930,3cyclictest23297-21df_inode09:05:140
295879930,3cyclictest20709-21seq07:55:140
295879930,3cyclictest15348-21apt-get08:50:120
295879930,3cyclictest1043-21df07:15:150
295879930,3cyclictest0-21swapper/011:30:120
295879930,3cyclictest0-21swapper/010:55:130
295879930,3cyclictest0-21swapper/009:35:160
295879930,3cyclictest0-21swapper/009:00:140
295879930,3cyclictest0-21swapper/008:55:020
295879930,3cyclictest0-21swapper/008:45:110
295879930,3cyclictest0-21swapper/008:30:120
295879930,3cyclictest0-21swapper/008:20:060
295879930,3cyclictest0-21swapper/008:05:130
295879930,3cyclictest0-21swapper/008:00:120
295879930,3cyclictest0-21swapper/007:45:170
295879930,3cyclictest0-21swapper/007:45:160
295879930,3cyclictest0-21swapper/007:25:010
295879930,2cyclictest79042sleep008:35:010
295879930,2cyclictest0-21swapper/009:20:050
295889921,1cyclictest0-21swapper/110:10:131
295889921,1cyclictest0-21swapper/108:05:131
295889921,1cyclictest0-21swapper/107:50:161
295889921,0cyclictest148542sleep112:35:111
295889920,2cyclictest0-21swapper/108:40:121
295889920,1cyclictest0-21swapper/108:55:121
295889920,1cyclictest0-21swapper/108:35:141
295889920,1cyclictest0-21swapper/108:30:131
295889920,1cyclictest0-21swapper/108:15:121
295889920,0cyclictest0-21swapper/108:10:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional