You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 06:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Tue May 26, 2026 00:45:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
50639912125,69cyclictest30434-21sh23:16:340
5080991183,46cyclictest24287-21ssh21:20:352
50809911348,11cyclictest0-21swapper/223:50:272
50639911325,50cyclictest0-21swapper/023:13:130
50639911240,56cyclictest11966-21diskmemload21:18:470
50639911116,72cyclictest29646-21kworker/u17:2+xprtiod00:02:310
50639910940,64cyclictest0-21swapper/000:10:240
50639910940,64cyclictest0-21swapper/000:10:240
3884210910,68sleep13930-21/usr/sbin/munin18:55:531
50639910747,51cyclictest6595-21ssh23:30:500
50809910634,13cyclictest0-21swapper/222:50:572
50639910630,61cyclictest0-21swapper/020:35:280
50809910531,13cyclictest0-21swapper/222:40:272
50809910531,13cyclictest0-21swapper/222:40:272
50639910525,42cyclictest0-21swapper/019:30:250
50639910525,42cyclictest0-21swapper/019:30:250
5063991045,59cyclictest0-21swapper/021:46:020
50639910419,52cyclictest245562chrt22:14:480
50809910353,42cyclictest0-21swapper/222:56:422
5063991033,58cyclictest21329-21sh22:09:150
5063991033,56cyclictest3405-21fschecks_time23:25:470
5063991033,56cyclictest3405-21fschecks_time23:25:470
50639910317,60cyclictest0-21swapper/021:15:260
50639910317,60cyclictest0-21swapper/021:15:260
50639910317,57cyclictest29549-21sh21:29:190
50639910310,58cyclictest18709-21rm22:05:190
50809910234,12cyclictest0-21swapper/221:50:282
5063991022,59cyclictest15214-21cut21:05:280
50639910218,53cyclictest5721-21memory22:36:470
50639910218,53cyclictest5721-21memory22:36:470
50639910144,50cyclictest21388-21sh23:55:410
50639910144,50cyclictest21388-21sh23:55:410
5080991002,41cyclictest30213-21idleruntime20:15:502
50639910049,46cyclictest5053-21cyclictest22:25:490
5063991003,58cyclictest0-21swapper/021:41:010
5063991003,58cyclictest0-21swapper/021:41:010
5063991002,60cyclictest31602-21cpuspeed20:20:410
5063991002,60cyclictest31602-21cpuspeed20:20:410
5063999939,53cyclictest298-21in:imuxsock23:40:250
506399993,85cyclictest0-21swapper/021:55:400
5063999937,52cyclictest0-21swapper/023:40:580
506399993,55cyclictest905-21apt-get20:25:400
506399992,59cyclictest30875-21cat22:25:260
5063999922,44cyclictest0-21swapper/022:33:240
5063999922,44cyclictest0-21swapper/022:33:240
5063999918,49cyclictest230952chrt19:54:260
36492997,54sleep03691-21cstates18:55:440
5080999835,12cyclictest0-21swapper/220:05:472
506399988,51cyclictest0-21swapper/000:25:530
5063999840,52cyclictest352-21runrttasks22:58:380
5063999836,55cyclictest21412-21unixbench_singl23:01:010
506399983,51cyclictest25713-21ntp_states22:15:550
506399983,51cyclictest25713-21ntp_states22:15:550
506399981,60cyclictest0-21swapper/021:36:090
5063999810,53cyclictest26387-21rm21:23:490
508099974,40cyclictest16058-21/usr/sbin/munin22:00:462
508099974,39cyclictest21621-21fschecks_count23:55:472
508099974,39cyclictest21621-21fschecks_count23:55:472
508099974,37cyclictest1583-21latency_hist22:30:282
508099974,34cyclictest352-21runrttasks21:31:592
5080999741,48cyclictest0-21swapper/221:15:242
5080999741,48cyclictest0-21swapper/221:15:242
506399977,55cyclictest0-21swapper/019:07:250
506399973,58cyclictest16318-21grep19:31:000
506399973,58cyclictest16318-21grep19:31:000
5063999734,53cyclictest0-21swapper/023:08:400
5063999729,58cyclictest0-21swapper/023:45:560
506399972,80cyclictest0-21swapper/020:55:380
5080999633,14cyclictest0-21swapper/223:32:172
506399963,54cyclictest17560-21ssh21:09:160
506399962,60cyclictest10880-21seq19:15:510
508099955,40cyclictest18433-21latency_hist19:40:272
508099954,66cyclictest20305-21kworker/u18:3+ext4-rsv-conversion00:05:392
508099954,66cyclictest20305-21kworker/u18:3+ext4-rsv-conversion00:05:392
508099954,37cyclictest24895-21latency_hist20:00:282
508099953,43cyclictest8905-21grep20:50:282
506399953,79cyclictest18403-21ssh23:50:430
506399953,59cyclictest11319-21modprobe21:51:240
506399953,58cyclictest9716-21sh22:42:280
5063999514,47cyclictest0-21swapper/020:45:170
506399948,50cyclictest1534-21uname00:15:510
506399943,57cyclictest20503-21cat19:45:480
508099935,32cyclictest806-21/usr/sbin/munin21:35:442
508099933,42cyclictest352-21runrttasks23:40:572
506399933,54cyclictest322-21chrt00:14:080
508099923,37cyclictest13338-21fschecks_count21:55:472
508099922,43cyclictest5255-21cpu19:00:402
5074999275,11cyclictest5053-21cyclictest23:21:011
507499922,58cyclictest8236-21ssh22:40:591
506399923,55cyclictest16824-21idleruntime-cro22:55:240
5063999234,48cyclictest18801-21fschecks_count19:40:460
506399922,55cyclictest11966-21diskmemload21:34:400
5080999169,13cyclictest5053-21cyclictest00:25:472
508099913,34cyclictest1858-21ssh22:30:462
508099913,34cyclictest1858-21ssh22:30:462
5080999113,47cyclictest8070-21apt-get22:41:132
506399914,73cyclictest28466-21diskstats20:10:450
506399913,52cyclictest0-21swapper/019:35:410
506399912,60cyclictest0-21swapper/020:00:390
506399912,60cyclictest0-21swapper/020:00:390
506399912,56cyclictest2006-21taskset23:23:160
446629114,47sleep24550-21unixbench_singl18:56:112
508099904,39cyclictest7839-21kernelversion21:46:492
507499902,53cyclictest31-21ksoftirqd/122:17:321
507499902,53cyclictest31-21ksoftirqd/122:17:321
506399904,50cyclictest0-21swapper/019:00:580
506399903,72cyclictest26664-21apt-get20:05:370
506399903,63cyclictest24409-21sed19:56:030
5063999023,60cyclictest0-21swapper/019:20:520
508099883,40cyclictest12492-21seq21:00:442
506399884,72cyclictest8925-21apt-get19:10:400
506399883,78cyclictest0-21swapper/022:45:370
508099874,41cyclictest29354-21apt-config23:15:282
5074998767,14cyclictest17837-40mandb23:50:281
507499872,52cyclictest31-21ksoftirqd/120:45:511
5074998617,40cyclictest31-21ksoftirqd/119:43:121
506399863,69cyclictest0-21swapper/020:45:390
5063998613,42cyclictest19676-21snmpd00:22:310
5074998558,10cyclictest181rcu_preempt20:36:011
5063998518,51cyclictest30455-1kworker/0:0H+mmc_complete20:20:280
508799845,56cyclictest0-21swapper/322:56:303
508099844,36cyclictest13470-21cron19:25:252
507499842,50cyclictest31-21ksoftirqd/120:04:101
507499842,50cyclictest31-21ksoftirqd/120:04:101
5080998367,10cyclictest16931-21latency_hist19:35:282
5080998367,10cyclictest16931-21latency_hist19:35:282
5080998348,9cyclictest181rcu_preempt23:05:132
5080998343,13cyclictest181rcu_preempt20:25:282
5080998343,13cyclictest181rcu_preempt20:25:282
5080998340,9cyclictest181rcu_preempt19:50:402
507499833,46cyclictest5780-21apt23:30:241
507499833,46cyclictest5780-21apt23:30:241
506399834,42cyclictest9458-21memory20:50:520
506399834,42cyclictest9458-21memory20:50:520
5074998261,15cyclictest12364-21fschecks_time19:20:461
5074998260,16cyclictest30672-21threads23:16:391
5074998256,17cyclictest12866-21sh22:47:551
507499824,71cyclictest18355-21apt-get23:50:411
5074998162,15cyclictest5053-21cyclictest21:27:421
5080998061,13cyclictest11966-21diskmemload00:16:022
507499803,27cyclictest31-21ksoftirqd/119:06:051
5080997947,22cyclictest13926-21fschecks_count19:25:452
5080997947,22cyclictest13926-21fschecks_count19:25:452
5074997957,14cyclictest545-21sh00:14:451
507499792,43cyclictest31-21ksoftirqd/120:32:131
507499792,40cyclictest31-21ksoftirqd/121:16:211
5074997922,36cyclictest31-21ksoftirqd/122:59:431
5080997856,16cyclictest6353-21processes20:40:582
5080997853,18cyclictest21777-21ssh22:10:252
5080997852,17cyclictest17971-21kworker/u19:321:10:242
5074997861,10cyclictest30708-21diskmemload21:30:501
5074997853,18cyclictest1036-21entropy21:35:461
507499783,69cyclictest18495-21kworker/u19:2+ext4-rsv-conversion22:35:461
507499783,69cyclictest18495-21kworker/u19:2+ext4-rsv-conversion22:35:461
507499782,42cyclictest31-21ksoftirqd/120:25:161
507499782,42cyclictest31-21ksoftirqd/120:25:161
507499782,29cyclictest31-21ksoftirqd/119:59:321
5080997755,16cyclictest11966-21diskmemload21:26:332
507499772,45cyclictest31-21ksoftirqd/122:05:181
5074997714,30cyclictest31-21ksoftirqd/100:02:371
5074997714,29cyclictest31-21ksoftirqd/119:48:511
5074997713,27cyclictest31-21ksoftirqd/120:41:521
508799765,51cyclictest18438-21chrt23:50:483
5080997660,10cyclictest11503-21kworker/u17:1-rpciod23:25:272
5080997656,14cyclictest300-21rs:main2
5074997658,12cyclictest7005-21kworker/u17:2-rpciod22:21:031
507499761,42cyclictest31-21ksoftirqd/120:14:101
506399763,67cyclictest0-21swapper/020:35:360
5080997542,10cyclictest28396-21fschecks_time22:20:472
5074997543,10cyclictest181rcu_preempt21:58:591
507499752,43cyclictest31-21ksoftirqd/100:19:021
5080997443,13cyclictest181rcu_preempt19:15:272
5074997452,15cyclictest13003-21processes21:00:571
5074997450,15cyclictest9213-21fschecks_count20:50:471
5074997450,15cyclictest9213-21fschecks_count20:50:471
507499743,26cyclictest12381-21fschecks_time23:40:461
507499742,34cyclictest31-21ksoftirqd/123:31:181
508099734,15cyclictest10623-21apt-get20:55:382
507499739,24cyclictest31-21ksoftirqd/119:04:241
5074997351,8cyclictest181rcu_preempt19:36:131
5074997344,8cyclictest181rcu_preempt22:13:461
507499734,41cyclictest31-21ksoftirqd/121:10:441
507499734,41cyclictest31-21ksoftirqd/121:10:441
507499732,48cyclictest31-21ksoftirqd/120:10:091
507499732,36cyclictest31-21ksoftirqd/121:23:101
507499732,35cyclictest31-21ksoftirqd/121:41:051
507499732,35cyclictest31-21ksoftirqd/121:41:051
5074997315,21cyclictest31-21ksoftirqd/121:08:181
5074997313,36cyclictest31-21ksoftirqd/123:11:031
5080997252,11cyclictest29646-21kworker/u17:2-rpciod00:10:482
5074997250,15cyclictest0-21swapper/119:35:241
5074997250,15cyclictest0-21swapper/119:35:241
5074997245,8cyclictest181rcu_preempt23:39:071
507499722,34cyclictest31-21ksoftirqd/122:33:371
507499722,34cyclictest31-21ksoftirqd/122:33:371
5074997223,24cyclictest31-21ksoftirqd/119:51:561
5074997211,33cyclictest31-21ksoftirqd/119:11:081
5074997211,29cyclictest31-21ksoftirqd/122:28:231
508099717,52cyclictest29701-21ssh23:15:552
5080997147,15cyclictest26427-21diskmemload00:03:482
5074997155,10cyclictest14560-21kernelversion22:50:501
5074997134,8cyclictest181rcu_preempt23:03:301
5080997050,14cyclictest10892-21kernelversion19:15:512
5080997039,10cyclictest181rcu_preempt20:30:472
5074997035,13cyclictest181rcu_preempt23:10:201
5074997033,8cyclictest181rcu_preempt21:46:211
5074997013,11cyclictest301ktimers/122:05:381
5080996947,16cyclictest4250-21kernelversion21:40:502
5080996947,16cyclictest4250-21kernelversion21:40:502
508099693,57cyclictest18588-21ssh23:51:152
5074996955,10cyclictest5053-21cyclictest20:26:381
5074996934,9cyclictest30464-21taskset20:16:061
5074996931,11cyclictest181rcu_preempt00:27:591
5074996926,8cyclictest181rcu_preempt00:00:281
5074996926,8cyclictest181rcu_preempt00:00:281
507499692,38cyclictest31-21ksoftirqd/100:24:051
507499692,35cyclictest31-21ksoftirqd/121:54:261
508099683,28cyclictest28359-21runrttasks20:10:412
507499683,27cyclictest31-21ksoftirqd/100:08:111
507499683,27cyclictest31-21ksoftirqd/100:08:111
507499682,32cyclictest31-21ksoftirqd/120:58:271
508099676,12cyclictest9114-21apt-get23:35:372
5080996746,7cyclictest181rcu_preempt00:20:582
508099674,45cyclictest9422-21latency20:50:512
508099674,45cyclictest9422-21latency20:50:512
507499672,24cyclictest31-21ksoftirqd/119:27:571
507499672,24cyclictest31-21ksoftirqd/119:27:571
5080996646,13cyclictest11966-21diskmemload23:25:482
5080996646,13cyclictest11966-21diskmemload23:25:482
5080996643,14cyclictest0-21swapper/219:46:012
5080996642,18cyclictest352-21runrttasks22:16:022
5080996642,18cyclictest352-21runrttasks22:16:022
508099663,44cyclictest4404-21fschecks_time20:35:472
5080996628,14cyclictest181rcu_preempt19:41:012
5074996645,7cyclictest181rcu_preempt19:16:091
5080996548,12cyclictest5053-21cyclictest21:15:522
508099653,43cyclictest24839-21cat22:15:262
5080996443,11cyclictest0-21swapper/223:10:232
508099643,46cyclictest26474-21latency_hist20:05:262
508099643,46cyclictest26474-21latency_hist20:05:262
508099632,25cyclictest11966-21diskmemload22:45:412
49942630,7sleep30-21swapper/321:41:123
49942630,7sleep30-21swapper/321:41:123
508799614,40cyclictest0-21swapper/323:15:423
5080996115,36cyclictest891-21sort20:25:362
508799604,26cyclictest0-21swapper/322:41:173
508799583,43cyclictest26836-21chrt20:05:473
508799553,41cyclictest26940-21taskset23:10:533
508799544,34cyclictest13041-21taskset19:21:003
508799543,40cyclictest12288-21taskset21:00:353
508799543,35cyclictest17082-21chrt21:07:563
508799542,40cyclictest23526-21chrt22:11:563
508799523,37cyclictest3111-21chrt20:30:583
5087995216,21cyclictest0-21swapper/323:50:303
508799513,39cyclictest22824-21taskset23:04:533
508799513,39cyclictest22629-21chrt19:51:013
508799513,38cyclictest6388-21chrt20:40:593
508799513,37cyclictest18317-21chrt21:10:363
508799513,37cyclictest18317-21chrt21:10:363
508799513,36cyclictest1840-21chrt23:22:483
508799513,35cyclictest15178-21chrt21:58:473
508799513,34cyclictest29982-21chrt20:15:433
508799513,33cyclictest21781-21chrt19:50:283
508799512,32cyclictest10183-21chrt00:30:283
5087995113,27cyclictest8053-21taskset23:33:223
41372517,20sleep30-21swapper/318:56:043
508799504,35cyclictest29348-21chrt20:11:423
508799504,29cyclictest0-21swapper/322:36:233
508799504,29cyclictest0-21swapper/322:36:233
508799503,38cyclictest3283-21chrt00:19:013
508799503,37cyclictest7699-21taskset20:45:483
508799503,35cyclictest30006-21chrt22:23:183
508799503,35cyclictest10891-21chrt23:37:563
508799503,34cyclictest19169-21chrt19:40:573
508799503,34cyclictest17979-21taskset22:03:163
508799503,33cyclictest32557-21chrt22:27:033
508799503,33cyclictest20496-21chrt22:06:573
508799503,33cyclictest18533-21taskset19:40:283
508799502,38cyclictest23950-21chrt23:05:543
508799502,37cyclictest7463-21chrt21:46:393
508799502,36cyclictest3747-21chrt23:25:563
508799502,36cyclictest3747-21chrt23:25:563
508799502,35cyclictest12040-21chrt21:00:263
508799502,35cyclictest11970-21chrt22:46:003
508799493,37cyclictest3756-21taskset22:33:523
508799493,37cyclictest3756-21taskset22:33:523
508799493,36cyclictest21534-21chrt23:55:433
508799493,36cyclictest21534-21chrt23:55:433
508799493,34cyclictest31608-21taskset20:20:413
508799493,34cyclictest31608-21taskset20:20:413
508799493,32cyclictest25638-21chrt20:00:573
508799493,32cyclictest25638-21chrt20:00:573
508799492,36cyclictest15967-21chrt22:53:053
508799492,36cyclictest12463-21chrt23:40:473
508799492,35cyclictest7529-21chrt19:05:493
508799492,35cyclictest29725-21taskset00:09:003
508799492,35cyclictest29725-21taskset00:09:003
508799492,34cyclictest3417-21chrt21:40:113
508799484,12cyclictest0-21swapper/320:26:243
508799483,36cyclictest25918-21chrt21:22:323
508799483,35cyclictest29584-21chrt21:29:253
508799483,31cyclictest10867-21chrt19:15:503
508799482,34cyclictest26609-21chrt22:17:103
508799482,34cyclictest26609-21chrt22:17:103
508799482,33cyclictest9688-21chrt20:50:593
508799482,33cyclictest9688-21chrt20:50:593
508799473,35cyclictest15439-21taskset19:30:403
508799473,35cyclictest15439-21taskset19:30:403
508799473,34cyclictest21368-21chrt21:15:433
508799472,33cyclictest9629-21chrt19:11:163
508799463,34cyclictest23580-21taskset19:55:433
508799462,32cyclictest26020-21taskset00:02:403
508799462,31cyclictest593-21taskset00:14:523
508799452,34cyclictest10727-21chrt21:50:573
508799444,30cyclictest5050-21chrt00:21:003
508799443,25cyclictest31400-21chrt21:31:033
5087994424,14cyclictest451rcuc/319:00:483
508799433,31cyclictest4605-21chrt20:35:523
508799393,27cyclictest14196-21taskset19:25:503
508799393,27cyclictest14196-21taskset19:25:503
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional