You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 23:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Wed May 13, 2026 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15291213514,79sleep015373-21grep06:57:020
15291213514,79sleep015373-21grep06:57:020
1496021199,80sleep214994-21grep06:56:522
1496021199,80sleep214994-21grep06:56:522
159509910435,14cyclictest0-21swapper/209:36:172
15950991043,38cyclictest31501-21cut07:46:362
159509910358,12cyclictest181rcu_preempt10:51:282
15950991013,35cyclictest4069-21kernelversion10:16:402
15950991002,40cyclictest23278-21latency_hist09:01:182
1595099973,38cyclictest20047-21grep07:11:372
15950999733,12cyclictest0-21swapper/211:41:302
15950999733,12cyclictest0-21swapper/211:41:302
15950999732,12cyclictest0-21swapper/209:31:182
15950999732,12cyclictest0-21swapper/209:31:182
15950999730,12cyclictest0-21swapper/208:36:372
15950999718,33cyclictest26366-21sh09:06:262
15950999718,33cyclictest26366-21sh09:06:262
1595099963,40cyclictest7745-21fschecks_count11:17:242
15950999622,32cyclictest27601-21sshd07:33:342
15950999622,32cyclictest27601-21sshd07:33:342
15950999536,11cyclictest0-21swapper/207:36:312
15950999536,11cyclictest0-21swapper/207:36:312
1595099953,39cyclictest10544-21fschecks_time11:21:382
1595099953,39cyclictest10544-21fschecks_time11:21:382
15940999535,44cyclictest1300-21latency_hist12:01:160
15950999434,11cyclictest15927-21cyclictest08:27:582
1595099943,34cyclictest32302-21latency_hist09:16:182
1594099949,37cyclictest0-21swapper/009:21:290
1595099933,38cyclictest26158-21latency_hist07:31:182
1595099933,38cyclictest26158-21latency_hist07:31:182
1595099933,36cyclictest20808-21grep08:51:492
1595099933,36cyclictest20808-21grep08:51:492
1595099924,35cyclictest352-21runrttasks07:20:322
1595099923,43cyclictest949-21netstat07:51:432
15944999272,10cyclictest291rcuc/112:26:281
1595099917,46cyclictest27627-21kworker/u17:2+flush-179:008:31:292
1595099914,39cyclictest13066-21latency10:31:402
1595099914,39cyclictest13066-21latency10:31:402
1595099913,38cyclictest18385-21fschecks_time09:46:372
1595099913,35cyclictest17874-21latency_hist09:46:172
1595099903,42cyclictest12385-21fschecks_count09:36:372
1595099893,38cyclictest626-21/usr/sbin/munin10:11:382
15940998965,18cyclictest21869-21sh11:41:170
15950998834,10cyclictest0-21swapper/210:41:172
1595099882,41cyclictest352-21runrttasks07:05:442
15944998869,13cyclictest544-21cron10:26:151
15944998869,13cyclictest544-21cron10:26:151
15944998856,8cyclictest181rcu_preempt11:55:491
15944998856,8cyclictest181rcu_preempt11:55:491
1594499883,47cyclictest31-21ksoftirqd/111:38:101
1595099862,36cyclictest547-21interrupts09:16:382
1595099862,36cyclictest547-21interrupts09:16:382
15950998611,35cyclictest22064-21timerwakeupswit09:51:512
15944998639,32cyclictest26479-21apt-get07:31:331
15944998639,32cyclictest26479-21apt-get07:31:331
15940998665,15cyclictest3693-21sendmail11:11:140
15944998562,14cyclictest3538-21runrttasks09:21:391
15944998557,8cyclictest181rcu_preempt08:56:391
15940998556,21cyclictest26164-21latency_hist09:06:180
15940998556,21cyclictest26164-21latency_hist09:06:180
1595099844,40cyclictest1228-21latency_hist12:01:172
15944998454,8cyclictest181rcu_preempt11:06:181
15944998450,14cyclictest181rcu_preempt09:54:131
15940998419,12cyclictest21137-21munin-run07:16:150
15950998351,23cyclictest22406-21threads10:46:502
15950998351,23cyclictest22406-21threads10:46:502
15944998324,37cyclictest24722-21ntpq09:56:441
1594099839,31cyclictest0-21swapper/009:31:530
15940998266,10cyclictest23294-21runrttasks07:21:390
15950998150,21cyclictest1-21systemd11:39:442
15944998154,20cyclictest13101-21cron12:21:141
1594499813,55cyclictest13638-21apt-get08:31:291
15944998054,9cyclictest181rcu_preempt11:56:301
1594499803,43cyclictest31-21ksoftirqd/107:41:311
1594499803,43cyclictest31-21ksoftirqd/107:41:311
1594099804,36cyclictest6697-21latency_hist11:16:190
1594099804,36cyclictest6697-21latency_hist11:16:190
15940998027,32cyclictest0-21swapper/007:06:540
1595099792,50cyclictest17272-21grep11:31:512
1595099792,50cyclictest17272-21grep11:31:512
15944997961,13cyclictest27461-21sh10:01:381
15944997959,10cyclictest0-21swapper/110:16:471
15944997958,7cyclictest181rcu_preempt10:47:091
15944997958,7cyclictest181rcu_preempt10:47:091
15944997955,9cyclictest181rcu_preempt08:46:401
15944997955,8cyclictest181rcu_preempt10:30:481
15944997940,8cyclictest181rcu_preempt09:41:371
15944997940,12cyclictest20321-21/usr/sbin/munin07:11:441
15944997940,11cyclictest0-21swapper/111:49:371
15940997955,17cyclictest1820-21fschecks_time12:01:370
15940997953,19cyclictest21276-21sh10:46:170
15940997916,16cyclictest26090-21latency_hist07:31:170
15940997916,16cyclictest26090-21latency_hist07:31:170
1455227926,34sleep30-21swapper/306:56:373
1455227926,34sleep30-21swapper/306:56:373
15950997848,20cyclictest19039-21latency_hist12:31:172
15944997853,9cyclictest181rcu_preempt12:21:501
15944997853,8cyclictest181rcu_preempt07:16:321
15944997849,8cyclictest181rcu_preempt07:31:191
15944997849,8cyclictest181rcu_preempt07:31:191
1594499783,45cyclictest2892-21ssh09:21:171
1594499783,45cyclictest2892-21ssh09:21:171
15944997831,17cyclictest12777-21/usr/sbin/munin10:31:331
15944997831,17cyclictest12777-21/usr/sbin/munin10:31:331
15944997829,10cyclictest0-21swapper/109:01:371
15944997829,10cyclictest0-21swapper/109:01:371
15940997859,14cyclictest15927-21cyclictest09:55:410
15940997825,12cyclictest22303-21mailstats10:46:480
15940997825,12cyclictest22303-21mailstats10:46:480
15950997740,10cyclictest181rcu_preempt08:21:512
15944997755,7cyclictest181rcu_preempt08:01:441
15944997729,14cyclictest181rcu_preempt08:16:501
15944997727,10cyclictest0-21swapper/109:38:321
15944997724,21cyclictest0-21swapper/108:26:181
15940997756,14cyclictest15182-21latency_hist08:36:180
1594099773,37cyclictest13155-21munin-run12:21:160
15950997648,22cyclictest23775-21if_eth009:01:392
15950997648,22cyclictest23775-21if_eth009:01:392
15950997648,20cyclictest300-21rs:main2
15950997640,20cyclictest3569-21kworker/u18:1-rpciod11:30:502
15950997639,14cyclictest181rcu_preempt11:11:302
15950997639,14cyclictest181rcu_preempt11:11:302
1595099763,39cyclictest7331-21latency_hist12:11:182
15944997641,10cyclictest0-21swapper/111:41:371
15944997641,10cyclictest0-21swapper/111:41:371
15944997636,10cyclictest0-21swapper/107:06:391
15944997629,29cyclictest0-21swapper/108:51:301
15944997629,29cyclictest0-21swapper/108:51:301
1594099765,37cyclictest24444-21fschecks_time09:56:370
15940997650,16cyclictest1461-21ssh09:17:250
15940997650,16cyclictest1461-21ssh09:17:250
1594099763,53cyclictest6693-21apt-get11:16:580
15940997615,18cyclictest352-21runrttasks11:35:100
15940997615,18cyclictest352-21runrttasks11:35:100
15940997613,24cyclictest0-21swapper/010:51:300
15950997553,13cyclictest0-21swapper/210:26:352
15950997550,15cyclictest0-21swapper/212:21:472
15944997557,12cyclictest29611-21/usr/sbin/munin09:12:011
15944997553,8cyclictest181rcu_preempt07:21:401
15944997551,10cyclictest181rcu_preempt07:01:511
15944997547,12cyclictest181rcu_preempt08:11:501
1594499754,45cyclictest10520-21fschecks_time11:21:371
1594499754,45cyclictest10520-21fschecks_time11:21:371
15944997542,10cyclictest181rcu_preempt10:56:001
15944997523,13cyclictest0-21swapper/110:37:591
15940997558,11cyclictest16730-21ssh12:26:370
1594099754,36cyclictest352-21runrttasks12:09:430
15940997529,33cyclictest681-21cron11:06:140
15940997524,37cyclictest29049-21cron09:11:150
15940997524,37cyclictest29049-21cron09:11:150
15940997523,22cyclictest22784-21latency_hist07:21:170
15944997459,9cyclictest1545-21timerandwakeup10:11:501
15944997450,9cyclictest181rcu_preempt12:15:351
15944997433,12cyclictest181rcu_preempt08:41:261
15940997455,10cyclictest0-21swapper/012:11:440
15940997413,23cyclictest5176-21latency_hist08:06:180
15940997412,18cyclictest453-21/usr/sbin/munin07:51:430
15940997410,23cyclictest29072-21apt-get09:11:300
1595099736,20cyclictest31132-21grep10:06:522
15950997334,13cyclictest0-21swapper/210:56:312
15944997350,7cyclictest181rcu_preempt09:06:401
15944997350,7cyclictest181rcu_preempt09:06:401
15944997349,8cyclictest181rcu_preempt07:56:181
15944997331,15cyclictest22961-21kworker/u17:0-xprtiod09:28:351
15944997331,15cyclictest22961-21kworker/u17:0-xprtiod09:28:351
15944997329,22cyclictest58-21kcompactd010:45:451
15944997326,12cyclictest0-21swapper/107:56:431
15944997318,36cyclictest9199-21cpu09:31:311
1594099735,55cyclictest19833-21awk08:51:150
1594099732,59cyclictest19676-21snmpd12:21:490
15940997312,24cyclictest4383-21awk10:16:480
15940997311,56cyclictest0-21swapper/010:36:280
116562730,4sleep0211rcuc/010:29:360
58592720,10sleep3451rcuc/308:06:473
1595799723,56cyclictest7429-21taskset11:16:583
15950997247,18cyclictest0-21swapper/210:26:172
15950997247,18cyclictest0-21swapper/210:26:172
15944997250,8cyclictest181rcu_preempt12:01:321
15944997242,8cyclictest181rcu_preempt11:36:171
15944997242,8cyclictest181rcu_preempt11:36:171
15944997233,14cyclictest181rcu_preempt08:36:361
1594499722,50cyclictest7184-21awk12:11:151
15940997246,18cyclictest352-21runrttasks09:46:320
1594099724,56cyclictest352-21runrttasks11:42:380
1594099724,56cyclictest352-21runrttasks11:42:380
1594099723,49cyclictest13474-21ls08:31:180
15940997216,17cyclictest7674-21ssh09:28:060
15940997216,17cyclictest7674-21ssh09:28:060
1594099721,38cyclictest8191-21seq08:15:150
1594099721,13cyclictest12910-21munin-run11:26:150
1594099721,13cyclictest12910-21munin-run11:26:150
15944997148,8cyclictest181rcu_preempt09:46:511
15944997147,8cyclictest181rcu_preempt11:26:381
15944997127,13cyclictest0-21swapper/111:17:421
15944997124,17cyclictest181rcu_preempt11:16:141
15944997124,17cyclictest181rcu_preempt11:16:141
151112718,34sleep10-21swapper/106:56:591
151112718,34sleep10-21swapper/106:56:591
1595799704,46cyclictest13359-21chrt08:31:173
1595799702,55cyclictest29207-21taskset10:04:433
15950997047,17cyclictest18204-21idleruntime-cro08:46:152
1595099703,45cyclictest30999-21latency11:01:432
15950997026,30cyclictest19486-21cron07:11:152
15944997055,10cyclictest29429-21sh07:41:151
15944997055,10cyclictest29429-21sh07:41:151
15944997047,9cyclictest181rcu_preempt07:46:481
15944997047,7cyclictest181rcu_preempt10:56:271
15944997043,12cyclictest181rcu_preempt08:30:081
1594099707,11cyclictest15-21ksoftirqd/010:11:500
1594099704,51cyclictest22384-21mailstats08:56:450
15940997041,19cyclictest25491-21ssh11:46:400
15940997021,16cyclictest352-21runrttasks11:53:180
15940997021,16cyclictest352-21runrttasks11:53:180
15950996947,13cyclictest30527-21sh11:55:442
15950996947,13cyclictest30527-21sh11:55:442
15950996932,15cyclictest352-21runrttasks11:50:232
15944996947,8cyclictest181rcu_preempt10:10:161
15940996949,13cyclictest12124-21ssh09:36:250
15940996944,17cyclictest7017-21sh10:21:400
15940996944,17cyclictest7017-21sh10:21:400
1594099694,35cyclictest30363-21tune2fs10:06:360
1594099693,38cyclictest17796-21modprobe09:46:150
15940996920,9cyclictest27534-21ssh10:56:270
15940996916,44cyclictest26511-21runrttasks07:31:450
15940996916,44cyclictest26511-21runrttasks07:31:450
15940996916,21cyclictest29501-21latency_hist07:41:170
15940996916,21cyclictest29501-21latency_hist07:41:170
1595799684,52cyclictest5226-21taskset08:06:193
1595799683,50cyclictest30595-21taskset10:06:413
1595799683,50cyclictest16191-21chrt08:36:523
1595799682,53cyclictest23721-21taskset09:56:033
1595799682,52cyclictest30983-21taskset09:12:343
1595799682,51cyclictest23210-21taskset11:41:513
1595799682,51cyclictest23210-21taskset11:41:513
1595799682,49cyclictest476-21chrt07:51:283
15950996851,9cyclictest0-21swapper/210:01:362
15950996814,12cyclictest181rcu_preempt09:56:502
1594099686,35cyclictest352-21runrttasks11:27:130
1594099685,39cyclictest352-21runrttasks08:52:480
1594099685,39cyclictest352-21runrttasks08:52:480
1594099683,44cyclictest30610-21unixbench_multi07:41:520
1594099683,44cyclictest30610-21unixbench_multi07:41:520
1594099681,33cyclictest3437-21apt08:01:150
15940996811,28cyclictest6876-21latency_hist08:11:180
15940996810,25cyclictest27606-21/usr/sbin/munin10:01:400
1595799673,50cyclictest25147-21taskset09:03:423
1595799673,50cyclictest25147-21taskset09:03:423
1595799673,50cyclictest23079-21taskset07:21:333
1595799672,51cyclictest32596-21taskset11:04:423
15950996746,7cyclictest181rcu_preempt08:11:332
1595099672,37cyclictest3958-21fschecks_time08:01:372
15944996738,9cyclictest181rcu_preempt08:06:321
15940996747,14cyclictest0-21swapper/008:36:360
15940996715,14cyclictest10971-21cat08:21:500
1595799662,52cyclictest30651-21taskset11:56:053
1595799662,52cyclictest30651-21taskset11:56:053
1595799662,51cyclictest20930-21taskset07:13:473
1595799662,51cyclictest20288-21chrt11:36:573
1595799662,50cyclictest19555-21taskset08:48:063
15950996641,17cyclictest1769-21fschecks_count12:01:372
1595099663,46cyclictest352-21runrttasks08:06:312
15944996651,8cyclictest0-21swapper/111:06:521
15940996622,9cyclictest0-21swapper/008:16:470
1595799653,50cyclictest25818-21chrt09:58:353
1595799653,49cyclictest5887-21taskset12:07:403
1595799653,49cyclictest17281-21chrt12:26:493
1595799653,48cyclictest24762-21taskset07:26:353
1595799653,48cyclictest24762-21taskset07:26:353
1595799653,48cyclictest22193-21chrt10:46:463
1595799653,48cyclictest22193-21chrt10:46:463
1595799653,47cyclictest17982-21taskset07:06:263
1595799653,47cyclictest16448-21chrt09:42:293
1595799652,50cyclictest18158-21taskset10:41:033
1595799652,49cyclictest4315-21chrt09:22:023
1595799652,49cyclictest3016-21chrt11:09:243
1595799652,49cyclictest22742-21chrt08:57:183
1595799652,48cyclictest9597-21chrt08:17:373
1595099654,41cyclictest3478-21latency_hist08:01:172
1595099652,42cyclictest29678-21/usr/sbin/munin07:41:372
1595099652,42cyclictest29678-21/usr/sbin/munin07:41:372
1595099652,34cyclictest19080-21cut08:46:472
1595099652,24cyclictest19269-21find10:41:472
15940996545,14cyclictest12904-21fschecks_time10:31:370
15940996545,14cyclictest12904-21fschecks_time10:31:370
15940996538,19cyclictest17443-21modprobe07:06:140
1595799643,49cyclictest12551-21chrt12:19:403
1595799643,48cyclictest8086-21taskset10:22:553
1595799643,48cyclictest8086-21taskset10:22:553
1595799643,47cyclictest14909-21taskset11:28:343
1595799642,50cyclictest9338-21taskset12:13:563
1595799642,50cyclictest7432-21taskset09:27:253
1595799642,50cyclictest7432-21taskset09:27:253
1595799642,49cyclictest6267-21chrt10:21:123
1595799642,49cyclictest3344-21chrt12:03:513
1595799642,49cyclictest29036-21chrt10:58:013
1595799642,49cyclictest10875-21taskset09:33:383
15950996441,12cyclictest0-21swapper/212:11:372
1595099643,44cyclictest1433-21kernelversion11:06:402
15940996416,22cyclictest18411-21ls08:46:190
1595799634,45cyclictest10146-21chrt08:21:253
1595799633,47cyclictest2211-21chrt10:13:103
1595799632,49cyclictest27290-21chrt11:50:023
1595799632,49cyclictest13620-21taskset08:31:263
1595799632,49cyclictest13439-21chrt12:21:263
1595799632,48cyclictest30814-21chrt07:43:353
1595799632,48cyclictest30814-21chrt07:43:353
1595799632,48cyclictest24965-21chrt10:51:413
1595799632,48cyclictest18076-21chrt09:46:223
1595799632,48cyclictest17436-21chrt11:32:113
1595799632,48cyclictest17436-21chrt11:32:113
15950996339,13cyclictest0-21swapper/212:16:362
15940996317,13cyclictest297-21systemd-logind07:51:160
1595799623,45cyclictest6352-21chrt11:15:203
1595799623,45cyclictest6352-21chrt11:15:203
1595799623,45cyclictest12967-21taskset09:36:503
1595799623,43cyclictest20113-21chrt08:51:263
1595799623,43cyclictest20113-21chrt08:51:263
1595799622,48cyclictest29151-21chrt07:37:483
1595799622,48cyclictest29151-21chrt07:37:483
1595799622,47cyclictest7087-21taskset08:11:333
1595799622,47cyclictest18101-21chrt08:45:073
15950996247,10cyclictest58-21kcompactd008:16:372
1595799613,45cyclictest2616-21chrt09:20:403
1595799613,45cyclictest2616-21chrt09:20:403
1595799613,45cyclictest21403-21chrt07:16:263
1595799613,45cyclictest12617-21chrt10:31:303
1595799613,45cyclictest12617-21chrt10:31:303
1595799613,44cyclictest27493-21chrt09:06:543
1595799613,44cyclictest27493-21chrt09:06:543
1595799613,44cyclictest11724-21taskset11:22:583
1595799613,44cyclictest11724-21taskset11:22:583
1595799612,47cyclictest893-21chrt12:00:193
1595799612,47cyclictest32360-21chrt07:47:483
1595799612,46cyclictest9552-21taskset10:26:233
1595799612,46cyclictest3302-21chrt07:59:473
1595799612,46cyclictest20194-21chrt10:43:223
1595099602,37cyclictest3468-21fschecks_time09:21:372
1595799593,44cyclictest17172-21chrt07:02:463
1595799572,44cyclictest26516-21chrt07:31:493
1595799572,44cyclictest26516-21chrt07:31:493
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional