You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-16 - 14:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Sat May 16, 2026 12:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
325249913158,8cyclictest0-21swapper/009:16:190
32544991263,89cyclictest20018-21apt-get09:21:372
325379912171,10cyclictest32515-21cyclictest07:06:441
325249911547,16cyclictest7617-21latency_hist09:01:070
325249911547,16cyclictest7617-21latency_hist09:01:070
325249910384,8cyclictest0-21swapper/011:56:490
197421020,9sleep30-21swapper/308:41:393
197421020,9sleep30-21swapper/308:41:393
325379910179,16cyclictest14834-21fschecks_time10:06:251
325379910179,16cyclictest14834-21fschecks_time10:06:251
316302998,63sleep131039-21/usr/sbin/munin06:56:461
316302998,63sleep131039-21/usr/sbin/munin06:56:461
3237529714,51sleep232391-21apt07:01:032
3237529714,51sleep232391-21apt07:01:032
3254499962,37cyclictest21196-21grep11:11:262
3254499962,37cyclictest21196-21grep11:11:262
32537999480,7cyclictest0-21swapper/112:16:411
3254499933,38cyclictest19063-21fschecks_count12:01:242
32537999373,9cyclictest0-21swapper/108:56:211
32537999373,9cyclictest0-21swapper/108:56:211
3254499922,28cyclictest12957-21grep07:36:412
32524999069,15cyclictest11085-21ntpd09:22:380
3254499893,34cyclictest15111-21ssh11:01:182
3254499893,29cyclictest352-21runrttasks08:55:462
32537998948,7cyclictest181rcu_preempt09:39:231
32537998948,7cyclictest181rcu_preempt09:39:231
3254499883,28cyclictest25892-21latency_hist09:31:062
3254499883,28cyclictest25892-21latency_hist09:31:062
3254499873,38cyclictest7533-21switchtime11:41:382
3254499873,37cyclictest30454-21cstates08:31:552
3254499873,37cyclictest18549-21sh12:01:032
3254499873,31cyclictest16077-21cat07:46:402
3254499873,29cyclictest27845-21ssh09:33:392
32537998771,11cyclictest32515-21cyclictest10:06:061
32537998767,14cyclictest18946-21ssh11:06:401
32537998758,7cyclictest181rcu_preempt07:46:261
32524998760,16cyclictest29815-21apt-key11:26:040
3254499869,43cyclictest27084-21apt-get11:21:202
3254499863,38cyclictest2423-21ssh10:40:202
3254499863,37cyclictest20503-21/usr/sbin/munin10:16:192
3254499863,34cyclictest31979-21/usr/sbin/munin09:41:392
3254499863,30cyclictest30090-21sh09:36:422
3254499863,30cyclictest30090-21sh09:36:422
3254499862,54cyclictest12291-21ssh10:56:262
3254499862,54cyclictest12291-21ssh10:56:262
3254499862,33cyclictest352-21runrttasks09:49:462
3254499862,33cyclictest352-21runrttasks09:49:462
32524998651,27cyclictest0-21swapper/009:11:210
32524998651,27cyclictest0-21swapper/009:11:210
3254499856,66cyclictest7917-21taskset09:01:272
3254499856,11cyclictest0-21swapper/211:53:592
3254499853,38cyclictest31748-21ntp_kernel_pll_12:21:322
3254499853,36cyclictest352-21runrttasks11:08:262
3254499853,33cyclictest352-21runrttasks08:14:052
3254499853,28cyclictest11111-21fschecks_time09:06:262
3254499852,24cyclictest7096-21latency_hist07:21:062
32537998566,14cyclictest32515-21cyclictest10:56:061
32537998566,14cyclictest32515-21cyclictest10:56:061
32524998556,22cyclictest24332-21kworker/u17:2+nfsiod10:10:390
32524998556,22cyclictest24332-21kworker/u17:2+nfsiod10:10:390
3254499843,36cyclictest20469-21ls08:01:302
3254499843,34cyclictest10643-21cat11:46:382
3254499843,29cyclictest352-21runrttasks10:06:362
3254499843,29cyclictest352-21runrttasks10:06:362
3254499842,35cyclictest938-21latency_hist08:41:062
3254499842,26cyclictest9752-21tr10:51:372
3254499842,26cyclictest9752-21tr10:51:372
3254499842,24cyclictest11730-21modprobe07:36:032
32524998465,9cyclictest0-21swapper/008:12:060
32524998463,15cyclictest30726-21fschecks_time11:26:500
3160928428,29sleep00-21swapper/006:56:450
3160928428,29sleep00-21swapper/006:56:450
3254499833,39cyclictest23492-21/usr/sbin/munin10:21:242
3254499833,39cyclictest23492-21/usr/sbin/munin10:21:242
3254499833,30cyclictest352-21runrttasks08:49:342
3254499833,30cyclictest24749-21latency_hist12:11:052
3254499833,30cyclictest24749-21latency_hist12:11:052
3254499833,28cyclictest28538-21cstates08:26:212
3254499833,28cyclictest28538-21cstates08:26:212
3254499833,27cyclictest8979-21fschecks_count07:26:262
3254499833,27cyclictest447-21taskset11:31:032
3254499833,26cyclictest16939-21sh09:16:192
3254499833,21cyclictest2666-21unixbench_multi12:26:402
3254499833,21cyclictest2666-21unixbench_multi12:26:402
32544998329,40cyclictest0-21swapper/210:46:352
32544998327,48cyclictest0-21swapper/209:16:032
32544998327,48cyclictest0-21swapper/209:16:032
3254499832,30cyclictest12488-21timerwakeupswit10:01:392
32537998368,7cyclictest0-21swapper/108:26:021
32537998368,7cyclictest0-21swapper/108:26:021
32537998361,10cyclictest0-21swapper/110:41:161
32524998367,10cyclictest300-21rs:main0
32524998360,8cyclictest181rcu_preempt10:41:060
3254499824,35cyclictest22192-21ntp_kernel_pll_08:06:332
3254499824,35cyclictest22192-21ntp_kernel_pll_08:06:332
3254499823,34cyclictest3527-21ssh10:41:312
3254499823,27cyclictest2597-21latency_hist08:46:062
3254499823,27cyclictest2597-21latency_hist08:46:062
3254499823,25cyclictest19102-21mailstats07:56:382
3254499822,34cyclictest25681-21grep12:11:362
3254499822,29cyclictest352-21runrttasks07:10:072
32537998263,7cyclictest181rcu_preempt09:55:051
32524998262,14cyclictest19724-21sh12:01:390
32524998219,40cyclictest24829-21latency_hist12:11:060
32524998219,40cyclictest24829-21latency_hist12:11:060
3254499814,38cyclictest938-21ls11:31:202
3254499814,38cyclictest938-21ls11:31:202
3254499814,34cyclictest4481-21grep11:36:362
3254499814,27cyclictest16840-21find07:51:142
3254499813,39cyclictest20421-21cat10:16:062
3254499813,36cyclictest11345-21ssh10:01:062
3254499813,35cyclictest7567-21latency_hist09:01:072
3254499813,35cyclictest7567-21latency_hist09:01:072
32544998133,40cyclictest0-21swapper/210:26:342
3254499813,24cyclictest4090-21/usr/sbin/munin07:11:222
3254499812,25cyclictest32380-21grep10:36:062
32537998164,12cyclictest23758-21sh09:26:391
32537998164,12cyclictest23758-21sh09:26:391
32537998163,12cyclictest21739-21apt-key12:06:071
32524998166,10cyclictest3413-21switchtime08:46:380
32524998161,10cyclictest0-21swapper/008:46:050
32524998161,10cyclictest0-21swapper/008:46:050
32524998159,16cyclictest30081-21latency_hist08:31:060
32524998159,16cyclictest30081-21latency_hist08:31:060
32524998151,21cyclictest0-21swapper/010:51:360
32524998151,21cyclictest0-21swapper/010:51:360
3254499804,23cyclictest23882-21/usr/sbin/munin11:16:402
3254499804,23cyclictest23882-21/usr/sbin/munin11:16:402
3254499803,35cyclictest6157-21ntp_offset09:51:332
3254499803,34cyclictest13842-21fschecks_time07:41:262
3254499802,23cyclictest28201-21apt-get12:16:262
3254499802,21cyclictest27709-21/usr/sbin/munin08:21:392
3254499802,21cyclictest27709-21/usr/sbin/munin08:21:392
32537998062,12cyclictest352-21runrttasks09:06:421
32524998060,7cyclictest0-21swapper/007:31:240
32524998056,8cyclictest181rcu_preempt11:01:160
32524998046,25cyclictest0-21swapper/009:26:390
32524998046,25cyclictest0-21swapper/009:26:390
3254499794,29cyclictest352-21runrttasks08:17:372
32537997964,8cyclictest0-21swapper/111:57:381
32537997964,8cyclictest0-21swapper/109:02:041
32537997927,24cyclictest0-21swapper/108:31:061
32537997927,24cyclictest0-21swapper/108:31:061
32524997959,8cyclictest181rcu_preempt10:31:220
88072780,4sleep08810-21grep10:51:050
3254499783,26cyclictest1174-21tail07:01:402
32537997858,10cyclictest0-21swapper/109:11:281
32537997858,10cyclictest0-21swapper/109:11:281
32524997859,12cyclictest7288-21diskmemload11:15:160
32524997859,12cyclictest7288-21diskmemload11:15:160
32524997858,9cyclictest0-21swapper/007:46:270
32524997851,20cyclictest10106-21sendmail07:31:030
32537997762,8cyclictest0-21swapper/110:46:411
32537997760,9cyclictest0-21swapper/111:26:511
32537997759,11cyclictest13458-21ssh11:51:361
32537997758,9cyclictest0-21swapper/108:11:131
32537997758,11cyclictest0-21swapper/111:11:211
32537997758,11cyclictest0-21swapper/111:11:211
32537997754,15cyclictest544-21cron09:46:021
32537997752,18cyclictest30068-21ssh11:26:061
32537997733,8cyclictest181rcu_preempt09:56:181
32524997758,9cyclictest0-21swapper/008:06:270
32524997758,9cyclictest0-21swapper/008:06:270
32524997758,8cyclictest0-21swapper/010:41:260
32524997741,7cyclictest181rcu_preempt12:26:380
32524997741,7cyclictest181rcu_preempt12:26:380
3254499763,23cyclictest7210-21taskset07:21:212
3254499763,23cyclictest7210-21taskset07:21:212
3253799762,36cyclictest20212-21idleruntime-cro10:16:021
32524997662,7cyclictest0-21swapper/008:36:360
32524997658,12cyclictest14958-21idleruntime-cro07:46:030
32524997654,8cyclictest181rcu_preempt11:31:320
32524997654,8cyclictest181rcu_preempt11:31:320
32524997653,17cyclictest298-21in:imuxsock09:02:000
32524997620,13cyclictest181rcu_preempt10:21:400
32524997620,13cyclictest181rcu_preempt10:21:400
32537997557,9cyclictest0-21swapper/111:01:241
32537997533,8cyclictest0-21swapper/109:25:011
3253799751,38cyclictest5755-21fschecks_count07:16:251
32524997562,6cyclictest0-21swapper/007:25:050
32524997562,6cyclictest0-21swapper/007:25:050
32524997556,12cyclictest6610-21latency_hist11:41:060
32537997461,9cyclictest14723-21kworker/u17:0-writeback12:26:301
32537997461,9cyclictest14723-21kworker/u17:0-writeback12:26:301
32537997459,8cyclictest0-21swapper/110:21:361
32537997459,8cyclictest0-21swapper/110:21:361
32537997455,10cyclictest0-21swapper/109:46:171
32537997455,10cyclictest0-21swapper/109:46:171
32537997454,14cyclictest4040-21sendmail08:51:031
32537997453,15cyclictest7833-21vmstat11:41:421
32537997444,22cyclictest21415-21modprobe08:06:031
32524997458,9cyclictest11560-21kworker/u18:0-nfsiod12:11:210
32524997453,15cyclictest8372-21latency_hist09:56:050
32537997358,8cyclictest0-21swapper/111:36:391
32537997358,8cyclictest0-21swapper/107:16:041
32537997354,13cyclictest31232-21runrttasks10:33:271
32537997349,10cyclictest181rcu_preempt07:26:051
32537997349,10cyclictest181rcu_preempt07:26:051
32524997359,8cyclictest0-21swapper/008:01:400
32524997359,10cyclictest32515-21cyclictest10:02:560
32524997342,24cyclictest0-21swapper/009:46:390
32524997342,24cyclictest0-21swapper/009:46:390
32524997330,11cyclictest32515-21cyclictest08:16:210
32537997257,8cyclictest0-21swapper/108:21:031
32537997255,8cyclictest0-21swapper/112:07:551
32537997255,8cyclictest0-21swapper/112:07:551
32537997250,14cyclictest19676-21snmpd10:58:001
32537997250,14cyclictest19676-21snmpd10:58:001
32537997237,11cyclictest0-21swapper/108:31:211
32537997235,13cyclictest0-21swapper/107:35:361
32524997257,8cyclictest0-21swapper/011:06:240
32524997255,11cyclictest7288-21diskmemload09:39:380
32524997255,11cyclictest7288-21diskmemload09:39:380
32524997255,11cyclictest21094-21/usr/sbin/munin10:16:310
3252499725,38cyclictest32632-21ntp_states09:41:350
32524997250,16cyclictest544-21cron11:51:030
32537997154,11cyclictest1404-21latency_hist12:26:051
32537997153,9cyclictest0-21swapper/110:16:251
32537997152,7cyclictest181rcu_preempt09:34:141
32537997145,20cyclictest24282-21kworker/u18:1+nfsiod11:34:111
32537997145,20cyclictest24282-21kworker/u18:1+nfsiod11:34:111
32537997139,13cyclictest0-21swapper/111:19:551
32537997139,13cyclictest0-21swapper/111:19:551
32524997157,10cyclictest32515-21cyclictest09:34:100
32524997154,12cyclictest12016-21cpu07:36:210
32524997144,19cyclictest1336-21modprobe12:26:020
32524997137,8cyclictest181rcu_preempt08:31:560
32524997116,10cyclictest181rcu_preempt07:56:400
32537997055,7cyclictest0-21swapper/110:36:201
32524997056,10cyclictest32515-21cyclictest10:26:540
32524997055,8cyclictest0-21swapper/010:16:050
32524997048,16cyclictest28239-21ssh12:16:200
32537996953,9cyclictest0-21swapper/110:26:371
32537996951,8cyclictest0-21swapper/107:51:371
32537996950,9cyclictest0-21swapper/111:46:121
32524996954,8cyclictest0-21swapper/007:51:400
32524996951,8cyclictest0-21swapper/011:16:360
32524996951,8cyclictest0-21swapper/011:16:360
32524996927,9cyclictest0-21swapper/010:57:480
32524996927,9cyclictest0-21swapper/010:57:480
32537996824,30cyclictest1526-21memory08:41:301
32537996824,30cyclictest1526-21memory08:41:301
32524996848,9cyclictest0-21swapper/007:11:290
32524996844,7cyclictest181rcu_preempt09:06:180
32537996750,12cyclictest0-21swapper/107:01:251
32537996748,13cyclictest32717-21timerandwakeup08:36:381
32537996733,9cyclictest0-21swapper/112:11:201
32524996753,10cyclictest58-21kcompactd009:58:010
32524996642,18cyclictest30393-21kworker/u17:1+nfsiod11:41:450
204002660,10sleep30-21swapper/309:22:163
32537996541,12cyclictest181rcu_preempt09:20:511
32524996549,8cyclictest0-21swapper/007:06:240
32524996545,7cyclictest181rcu_preempt07:01:280
32537996447,8cyclictest181rcu_preempt07:56:191
32524996445,8cyclictest0-21swapper/007:16:210
32524996431,10cyclictest0-21swapper/008:52:230
310692648,32sleep30-21swapper/306:56:233
310692648,32sleep30-21swapper/306:56:233
32537996349,10cyclictest32515-21cyclictest07:26:221
32524996347,7cyclictest0-21swapper/008:21:200
32524996347,7cyclictest0-21swapper/008:21:200
3255099624,40cyclictest26575-21taskset10:26:203
3253799623,43cyclictest13702-21cstates07:41:211
3253799623,37cyclictest23094-21apt-key08:11:041
3253799623,37cyclictest23094-21apt-key08:11:041
3255099613,42cyclictest3240-21chrt07:06:423
3255099612,32cyclictest0-21swapper/308:11:213
32537996127,10cyclictest181rcu_preempt08:51:261
3255099603,43cyclictest5545-21chrt07:16:143
32537996029,22cyclictest12800-21grep07:36:401
3255099592,41cyclictest21504-21taskset12:06:033
3255099582,40cyclictest17226-21taskset07:51:303
3255099573,37cyclictest1589-21chrt12:26:073
3255099573,37cyclictest15389-21taskset11:01:263
3255099572,39cyclictest4468-21chrt10:42:423
3255099572,34cyclictest20465-21chrt10:16:073
3255099563,40cyclictest28939-21taskset09:36:073
32550995623,22cyclictest32724-21chrt10:36:253
3255099554,34cyclictest32056-21taskset08:36:213
32550995540,8cyclictest0-21swapper/311:56:203
3255099553,38cyclictest7626-21chrt09:01:073
3255099553,38cyclictest7626-21chrt09:01:073
3255099553,35cyclictest22009-21chrt08:06:283
3255099553,35cyclictest22009-21chrt08:06:283
3255099552,39cyclictest32174-21chrt09:41:223
3255099543,38cyclictest17074-21taskset09:16:243
3255099543,38cyclictest14183-21chrt11:52:503
3255099543,37cyclictest6924-21chrt11:41:233
3255099543,36cyclictest31636-21chrt11:27:253
3255099543,33cyclictest1206-21taskset07:01:403
3255099534,35cyclictest10314-21chrt11:46:343
3255099534,33cyclictest13726-21chrt07:41:213
3255099533,38cyclictest4385-21chrt09:48:483
3255099533,38cyclictest4385-21chrt09:48:483
3255099533,35cyclictest24388-21chrt12:10:173
3255099533,35cyclictest24388-21chrt12:10:173
3255099524,34cyclictest23795-21chrt10:21:263
3255099524,34cyclictest23795-21chrt10:21:263
3255099523,36cyclictest9007-21taskset07:26:273
3255099523,36cyclictest17790-21taskset10:11:253
3255099523,35cyclictest9564-21chrt09:56:423
32550995229,15cyclictest0-21swapper/310:36:063
3255099522,37cyclictest12792-21chrt10:57:533
3255099522,37cyclictest12792-21chrt10:57:533
3255099514,35cyclictest14292-21chrt09:11:303
3255099514,35cyclictest14292-21chrt09:11:303
3255099513,37cyclictest12836-21chrt09:09:143
3255099513,36cyclictest20509-21chrt08:01:303
3255099513,35cyclictest30089-21chrt08:31:063
3255099513,35cyclictest30089-21chrt08:31:063
3255099513,34cyclictest25188-21chrt11:17:113
3255099513,34cyclictest25188-21chrt11:17:113
3255099513,33cyclictest7993-21chrt09:01:403
3255099512,36cyclictest12752-21chrt07:36:393
3255099512,32cyclictest9775-21chrt10:51:373
3255099512,32cyclictest9775-21chrt10:51:373
3255099504,33cyclictest15998-21chrt07:46:403
3255099503,36cyclictest29767-21chrt11:26:033
3255099503,36cyclictest27811-21chrt12:16:053
3255099503,36cyclictest26573-21chrt08:21:053
3255099503,34cyclictest25217-21chrt09:29:463
3255099503,34cyclictest25217-21chrt09:29:463
3255099503,33cyclictest6693-21chrt09:51:483
3255099503,32cyclictest29348-21chrt09:36:273
3255099503,32cyclictest29348-21chrt09:36:273
3255099502,35cyclictest19927-21chrt11:09:093
3255099494,31cyclictest31217-21taskset08:32:163
32550994914,22cyclictest0-21swapper/312:16:183
3255099483,36cyclictest8543-21chrt10:50:483
3255099483,36cyclictest12832-21taskset10:02:133
3255099483,35cyclictest16230-21chrt10:08:213
3255099483,35cyclictest16230-21chrt10:08:213
3255099483,32cyclictest11966-21taskset07:36:073
3255099482,35cyclictest3184-21chrt12:27:533
3255099482,35cyclictest3184-21chrt12:27:533
3255099482,34cyclictest5972-21taskset11:39:493
3255099482,31cyclictest19917-21taskset08:01:063
3255099463,31cyclictest806-21taskset11:31:143
3255099463,31cyclictest806-21taskset11:31:143
3255099463,30cyclictest23386-21chrt11:15:293
3255099463,30cyclictest23386-21chrt11:15:293
3255099452,30cyclictest2693-21taskset08:46:173
3255099433,29cyclictest26789-21taskset08:21:183
3255099433,29cyclictest26789-21taskset08:21:183
3255099424,28cyclictest5714-21taskset08:56:053
3255099424,26cyclictest7159-21chrt07:21:163
3255099424,26cyclictest7159-21chrt07:21:163
3255099402,27cyclictest4289-21taskset07:11:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional