You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 14:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Sat May 30, 2026 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
309809912371,45cyclictest9406-21fschecks_count21:06:231
309529911826,46cyclictest0-21swapper/023:05:310
309529911826,46cyclictest0-21swapper/023:05:310
30830211615,64sleep130814-21munin-run19:00:121
30781211613,75sleep030829-21cat19:00:110
309529911549,58cyclictest0-21swapper/020:20:300
309529911540,64cyclictest0-21swapper/000:10:290
309529911540,64cyclictest0-21swapper/000:10:290
31006991143,70cyclictest2961-21apt-get21:50:232
31006991143,70cyclictest2961-21apt-get21:50:232
30980991073,73cyclictest11173-21ssh23:50:271
30952991073,93cyclictest0-21swapper/019:55:210
30980991063,75cyclictest14340-21apt-get23:55:261
30952991054,88cyclictest8965-21apt-get19:30:230
30952991052,59cyclictest0-21swapper/023:45:340
30952991052,59cyclictest0-21swapper/023:45:340
30952991044,92cyclictest21466-21apt-get22:20:190
30952991043,91cyclictest0-21swapper/019:35:210
30952991043,91cyclictest0-21swapper/019:35:210
30952991042,73cyclictest4088-21cat22:45:110
30952991042,73cyclictest4088-21cat22:45:110
30980991023,27cyclictest5181-21taskset20:58:461
309529910246,46cyclictest0-21swapper/020:05:200
309529910246,46cyclictest0-21swapper/020:05:200
30952991023,89cyclictest0-21swapper/000:25:210
309529910232,63cyclictest0-21swapper/023:30:190
30952991022,66cyclictest20752-21latency00:05:330
30952991012,89cyclictest24442-21/usr/sbin/munin22:25:250
30952991012,89cyclictest24442-21/usr/sbin/munin22:25:250
309809910012,67cyclictest3562-21kworker/u19:2+ext4-rsv-conversion21:50:571
309809910012,67cyclictest3562-21kworker/u19:2+ext4-rsv-conversion21:50:571
309529910050,43cyclictest0-21swapper/021:15:330
309529910050,43cyclictest0-21swapper/021:15:330
30952991003,88cyclictest30177-21munin-run22:35:100
30952991003,88cyclictest30177-21munin-run22:35:100
30980999976,16cyclictest352-21runrttasks22:30:441
30980999976,16cyclictest352-21runrttasks22:30:441
3095299993,89cyclictest0-21swapper/021:30:240
3095299993,89cyclictest0-21swapper/021:30:240
3095299992,24cyclictest338-21sh22:39:070
3095299992,24cyclictest338-21sh22:39:070
3098099982,30cyclictest25663-21fschecks_count23:20:291
3098099982,30cyclictest15690-21sh23:04:371
30952999850,41cyclictest0-21swapper/020:30:250
3095299983,91cyclictest0-21swapper/022:50:220
3095299983,6cyclictest0-21swapper/022:15:190
3095299983,67cyclictest3604-21sshd21:50:320
3095299983,67cyclictest3604-21sshd21:50:320
3095299983,57cyclictest0-21swapper/021:36:310
3098099972,68cyclictest1312-21ssh22:40:261
3098099972,68cyclictest1312-21ssh22:40:261
30952999745,44cyclictest0-21swapper/019:45:190
3095299974,52cyclictest14340-21apt-get23:56:160
30952999737,51cyclictest0-21swapper/019:25:300
3095299972,7cyclictest0-21swapper/021:45:230
3098099962,32cyclictest7355-21cpuspeed22:50:241
30952999644,45cyclictest23412-21sh21:30:010
30952999633,53cyclictest0-21swapper/023:55:090
3095299962,64cyclictest9705-21ssh22:00:280
3098099952,29cyclictest5616-21sh22:46:301
3095299952,7cyclictest0-21swapper/022:05:190
17742950,8sleep21778-21unixbench_multi19:05:472
3098099942,78cyclictest32269-21ssh00:25:211
3098099942,6cyclictest0-21swapper/120:05:211
3098099942,6cyclictest0-21swapper/120:05:211
3095299943,80cyclictest29263-21apt-get00:20:190
30952999434,54cyclictest0-21swapper/023:15:460
30952999431,53cyclictest0-21swapper/020:27:430
3095299942,7cyclictest0-21swapper/021:00:230
3095299942,7cyclictest0-21swapper/021:00:230
3098099932,33cyclictest13679-21sh21:12:481
30952999369,15cyclictest9556-21diskmemload21:06:260
3095299933,78cyclictest749-21cpu20:45:240
3095299933,78cyclictest749-21cpu20:45:240
30952999337,50cyclictest0-21swapper/022:50:080
30952999334,48cyclictest0-21swapper/019:15:400
3095299932,65cyclictest27523-21ssh00:16:080
30952999321,44cyclictest0-21swapper/022:59:360
30952999321,44cyclictest0-21swapper/022:59:360
30980999267,11cyclictest0-21swapper/120:35:301
3095299922,82cyclictest5504-21latency_hist21:00:120
30952999217,49cyclictest5653-21mailstats23:40:410
30952999217,49cyclictest5653-21mailstats23:40:410
3098099912,30cyclictest23717-21ssh21:30:121
3098099912,30cyclictest23717-21ssh21:30:121
3095299913,82cyclictest0-21swapper/021:20:210
3095299913,81cyclictest5306-21diskmemload21:40:220
30952999135,48cyclictest0-21swapper/020:00:270
318862900,25sleep0161ktimers/019:02:010
3098099903,8cyclictest0-21swapper/121:00:231
3098099903,8cyclictest0-21swapper/121:00:231
30952999032,46cyclictest0-21swapper/019:10:110
31006998971,12cyclictest10557-40ogrotate23:50:092
31006998971,12cyclictest10557-40ogrotate23:50:092
30952998939,44cyclictest5306-21diskmemload23:02:590
3095299892,77cyclictest7176-21/usr/sbin/munin21:55:400
3095299892,58cyclictest30827-21ssh23:29:310
3095299892,58cyclictest30827-21ssh23:29:310
30952998923,59cyclictest4711-1kworker/0:2H+mmc_complete21:15:110
30980998873,8cyclictest0-21swapper/120:40:441
30980998872,9cyclictest0-21swapper/120:30:321
3098099884,74cyclictest0-21swapper/119:35:211
3098099884,74cyclictest0-21swapper/119:35:211
3098099882,25cyclictest17547-21ssh23:06:261
3098099882,25cyclictest17547-21ssh23:06:261
3095299883,75cyclictest19448-21apt-get23:11:070
3095299882,76cyclictest1918-21apt-get23:35:230
3095299873,77cyclictest0-21swapper/022:10:210
3095299872,72cyclictest25425-21ssh23:20:200
3095299872,72cyclictest15418-21apt-get19:50:180
30952998713,68cyclictest4220-21kworker/u17:2+rpciod00:05:100
30952998713,68cyclictest4220-21kworker/u17:2+rpciod00:05:100
3100699862,45cyclictest39-21ksoftirqd/223:34:132
30980998666,11cyclictest0-21swapper/119:50:341
30980998644,35cyclictest0-21swapper/119:45:081
30980998644,35cyclictest0-21swapper/119:45:081
30980998637,43cyclictest0-21swapper/122:05:421
3095299863,68cyclictest0-21swapper/020:50:230
3098099854,68cyclictest29276-21ssh00:20:201
31006998464,13cyclictest544-21cron19:30:092
30980998468,11cyclictest30940-21cyclictest19:05:171
30980998459,16cyclictest688-21/usr/sbin/munin20:45:331
30980998459,16cyclictest688-21/usr/sbin/munin20:45:331
30980998443,33cyclictest0-21swapper/122:25:411
30980998443,33cyclictest0-21swapper/122:25:411
3098099842,36cyclictest28071-21basename20:30:081
30952998430,49cyclictest0-21swapper/020:10:440
30952998430,49cyclictest0-21swapper/020:10:440
3095299842,74cyclictest3847-21modprobe19:15:080
31006998359,18cyclictest16984-21latency_hist00:00:102
31006998355,21cyclictest14210-21kworker/u19:1+nfsiod22:09:172
3100699833,71cyclictest1831-21cstates22:41:392
3100699833,71cyclictest1831-21cstates22:41:392
30980998347,27cyclictest18818-21cut22:15:291
3098099834,68cyclictest0-21swapper/100:15:261
3098099834,31cyclictest31605-21taskset23:30:281
3098099833,65cyclictest19702-21ssh23:11:011
3098099832,68cyclictest0-21swapper/122:10:211
3098099832,36cyclictest26444-21latency_hist20:25:111
3098099832,33cyclictest31605-21fschecks_count19:01:541
3095299833,69cyclictest31560-21apt-get20:40:190
3095299832,6cyclictest0-21swapper/020:35:430
31006998263,12cyclictest300-21rs:main2
3098099825,70cyclictest0-21swapper/119:30:231
3098099825,67cyclictest0-21swapper/123:45:221
3098099825,67cyclictest0-21swapper/123:45:221
3098099823,68cyclictest0-21swapper/120:00:211
3103199819,59cyclictest11151-21chrt23:50:243
31006998157,17cyclictest5306-21diskmemload00:20:312
3100699813,39cyclictest3276-21gpgv21:50:112
30980998166,8cyclictest0-21swapper/119:10:331
3098099814,70cyclictest0-21swapper/122:00:191
3098099813,31cyclictest14059-21fschecks_time19:45:311
3095299813,6cyclictest0-21swapper/019:40:210
3095299813,6cyclictest0-21swapper/019:40:210
30952998113,55cyclictest20837-1kworker/0:1H+mmc_complete20:20:110
31006998058,15cyclictest5306-21diskmemload21:15:312
31006998058,15cyclictest5306-21diskmemload21:15:312
3100699802,61cyclictest19912-21/usr/sbin/munin23:11:152
30980998059,11cyclictest0-21swapper/123:35:291
3098099803,7cyclictest0-21swapper/122:20:191
3098099803,72cyclictest0-21swapper/120:15:241
3098099803,71cyclictest0-21swapper/119:15:201
3098099803,34cyclictest31602-21vmstat22:35:461
3098099803,34cyclictest31602-21vmstat22:35:461
3098099802,35cyclictest3115-21apt-config21:50:101
3098099802,34cyclictest6924-21ssh21:55:321
3098099802,32cyclictest18050-21expr21:20:281
3098099802,29cyclictest22569-21rm00:09:031
3095299803,64cyclictest0-21swapper/019:20:190
31006997947,13cyclictest181rcu_preempt22:49:252
3098099794,33cyclictest11113-21switchtime22:55:461
3098099794,33cyclictest11113-21switchtime22:55:461
30980997936,36cyclictest0-21swapper/120:00:081
3098099792,34cyclictest5641-21mailstats23:40:401
3098099792,34cyclictest5641-21mailstats23:40:401
3098099792,32cyclictest17594-21latency_hist21:20:111
3098099792,32cyclictest17594-21latency_hist21:20:111
31006997850,19cyclictest11596-21sh21:10:112
31006997837,18cyclictest28176-21find22:30:412
31006997837,18cyclictest28176-21find22:30:412
3098099783,32cyclictest29136-21grep23:25:421
3098099783,32cyclictest29136-21grep23:25:421
3098099782,34cyclictest6008-21/usr/sbin/munin19:20:301
3098099782,32cyclictest17736-21ssh00:00:361
3098099782,32cyclictest17736-21ssh00:00:361
3098099782,31cyclictest21927-21ssh21:25:501
3098099782,30cyclictest28017-21rm21:36:091
31006997757,14cyclictest25452-21kernelversion20:20:342
31006997730,35cyclictest5449-21modprobe19:20:082
31006997718,51cyclictest11299-21sh23:50:492
3098099773,31cyclictest8246-21unixbench_multi19:25:451
3098099772,60cyclictest29758-21apt-get21:40:191
3098099772,32cyclictest23539-21awk23:15:461
3075927732,29sleep30-21swapper/319:00:103
31006997612,53cyclictest31510-21df19:01:232
30980997638,31cyclictest0-21swapper/120:50:331
3098099763,29cyclictest25621-21taskset00:14:111
3098099763,29cyclictest25621-21taskset00:14:111
3071827610,41sleep230739-21apt-get19:00:092
3103199753,56cyclictest18880-21chrt23:10:083
3103199753,56cyclictest18880-21chrt23:10:083
31006997556,14cyclictest58-21kcompactd020:40:252
3100699755,46cyclictest18706-21grep22:15:282
31006997549,9cyclictest18072-21tune2fs21:20:282
3098099753,34cyclictest22869-21/usr/sbin/munin20:10:451
3098099753,34cyclictest22869-21/usr/sbin/munin20:10:451
31006997453,15cyclictest803-21sendmail-mta23:08:142
31006997453,15cyclictest803-21sendmail-mta23:08:142
31006997453,13cyclictest5306-21diskmemload00:16:452
31006997444,21cyclictest0-21swapper/223:20:422
31006997440,12cyclictest181rcu_preempt21:25:582
31006997437,20cyclictest15737-21ssh23:04:442
31006997432,18cyclictest181rcu_preempt21:35:322
31006997356,11cyclictest2704-21/usr/sbin/munin23:35:412
31006997353,12cyclictest0-21swapper/223:15:192
3100699733,40cyclictest7647-21cat23:45:102
3100699733,40cyclictest7647-21cat23:45:102
1787927351,9sleep20-21swapper/219:55:462
31006997256,8cyclictest58-21kcompactd020:48:352
31006997256,8cyclictest58-21kcompactd020:48:352
31006997254,11cyclictest26513-21ssh21:35:112
31006997254,11cyclictest26513-21ssh21:35:112
31006997251,14cyclictest32704-21latency00:25:332
31006997249,16cyclictest30940-21cyclictest21:45:112
3100699723,49cyclictest18815-21fschecks_count20:00:292
3100699722,41cyclictest14729-21unixbench_singl19:45:452
31006997150,11cyclictest0-21swapper/223:28:382
31006997150,11cyclictest0-21swapper/223:28:382
3100699714,45cyclictest30739-21fschecks_count22:35:292
3100699714,45cyclictest30739-21fschecks_count22:35:292
31006997138,11cyclictest181rcu_preempt20:50:292
3100699713,30cyclictest12427-21chrt19:40:302
3100699713,30cyclictest12427-21chrt19:40:302
31006997126,16cyclictest181rcu_preempt22:52:552
31031997016,31cyclictest0-21swapper/323:55:253
3100699704,44cyclictest4173-21fschecks_count20:55:302
31006997034,12cyclictest181rcu_preempt20:35:412
3100699702,40cyclictest24693-21fschecks_count22:25:302
3100699702,40cyclictest24693-21fschecks_count22:25:302
3103199693,52cyclictest28593-21taskset23:25:283
3103199693,52cyclictest28593-21taskset23:25:283
31031996917,30cyclictest0-21swapper/321:05:273
31006996949,11cyclictest0-21swapper/200:10:262
31006996949,11cyclictest0-21swapper/200:10:262
31006996948,11cyclictest0-21swapper/219:30:282
3100699694,41cyclictest352-21runrttasks19:22:132
31006996937,17cyclictest181rcu_preempt22:20:182
31006996928,13cyclictest181rcu_preempt00:02:572
31006996928,13cyclictest181rcu_preempt00:02:572
31006996919,29cyclictest3464-21unixbench_singl19:10:432
3103199683,51cyclictest3035-21taskset20:50:423
3103199683,51cyclictest12193-21taskset21:10:323
3103199683,50cyclictest7946-21taskset21:04:113
3103199683,50cyclictest7946-21taskset21:04:113
3103199683,49cyclictest0-21swapper/300:10:203
3103199683,49cyclictest0-21swapper/300:10:203
31006996850,9cyclictest0-21swapper/222:10:462
31006996850,12cyclictest0-21swapper/200:05:412
3100699683,46cyclictest10483-21ssh22:00:442
3100699683,36cyclictest28525-21fschecks_count20:30:302
3103199674,49cyclictest10233-21chrt22:55:203
3103199674,49cyclictest10233-21chrt22:55:203
3103199673,51cyclictest8466-21chrt22:51:033
3103199673,51cyclictest609-21chrt23:32:213
3103199673,51cyclictest3681-21taskset21:50:443
3103199673,51cyclictest3681-21taskset21:50:443
3103199673,49cyclictest23969-21chrt23:16:573
3103199672,51cyclictest3263-21taskset22:43:163
3103199672,51cyclictest3263-21taskset22:43:163
3100699675,35cyclictest5893-21ssh21:00:302
3100699675,35cyclictest5893-21ssh21:00:302
31006996747,13cyclictest13737-21ssh21:12:572
3103199663,50cyclictest31477-21chrt21:42:293
3103199663,50cyclictest28577-21chrt21:37:433
3103199663,50cyclictest2455-21chrt00:30:083
3103199663,50cyclictest19859-21taskset20:03:443
3103199663,50cyclictest12676-21chrt22:05:273
3103199663,49cyclictest0-21swapper/322:20:193
3103199662,53cyclictest8407-21taskset21:57:583
3103199662,53cyclictest28241-21taskset23:25:123
3103199662,51cyclictest22474-21chrt20:10:403
3103199662,51cyclictest22474-21chrt20:10:403
3103199662,51cyclictest17043-21taskset19:55:263
3103199662,50cyclictest7490-21chrt19:25:293
3103199662,50cyclictest29816-21chrt22:34:163
3103199662,50cyclictest29816-21chrt22:34:163
3103199662,49cyclictest31847-21chrt19:01:593
3100699663,41cyclictest11364-21vmstat22:55:502
3100699663,41cyclictest11364-21vmstat22:55:502
3100699663,39cyclictest39-21ksoftirqd/220:25:412
3100699663,26cyclictest24960-21latency_hist20:20:112
31006996631,12cyclictest181rcu_preempt19:36:152
31006996631,12cyclictest181rcu_preempt19:36:152
3103199653,49cyclictest24473-21chrt22:25:213
3103199653,49cyclictest24473-21chrt22:25:213
3103199653,48cyclictest5228-21chrt20:59:253
3103199653,48cyclictest29360-21taskset20:31:103
3103199653,48cyclictest11866-21chrt19:39:423
3103199653,48cyclictest11866-21chrt19:39:423
3103199653,48cyclictest0-21swapper/320:20:203
3103199653,47cyclictest5608-21chrt22:46:293
3103199653,47cyclictest30515-21taskset22:35:223
3103199653,47cyclictest30515-21taskset22:35:223
3103199652,51cyclictest22074-21taskset23:15:023
3103199652,50cyclictest16841-21taskset21:18:183
3103199652,50cyclictest16841-21taskset21:18:183
3103199652,49cyclictest5496-21chrt19:20:103
3103199652,47cyclictest0-21swapper/320:05:213
3103199652,47cyclictest0-21swapper/320:05:213
3100699654,41cyclictest803-21sendmail-mta19:54:042
3100699654,31cyclictest352-21runrttasks20:12:212
3100699654,31cyclictest352-21runrttasks20:12:212
3103199643,48cyclictest24220-21chrt21:30:343
3103199643,48cyclictest24220-21chrt21:30:343
3103199643,48cyclictest10019-21chrt23:48:423
3103199643,48cyclictest10019-21chrt23:48:423
3103199643,48cyclictest0-21swapper/300:05:193
3103199643,47cyclictest5733-21chrt19:20:223
3103199643,46cyclictest0-21swapper/319:30:233
3103199642,50cyclictest17949-21taskset00:00:423
3103199642,50cyclictest17949-21taskset00:00:423
3103199642,49cyclictest19644-21taskset22:15:533
3103199642,49cyclictest0-21swapper/322:00:193
3103199642,48cyclictest22902-21chrt21:28:353
3103199642,48cyclictest13873-21chrt19:45:253
3103199642,48cyclictest12188-21chrt19:40:233
3103199642,48cyclictest12188-21chrt19:40:233
3100699644,46cyclictest20418-21fschecks_count20:05:292
3100699644,46cyclictest20418-21fschecks_count20:05:292
3103199633,48cyclictest27942-21chrt00:17:193
3103199633,48cyclictest2435-21chrt21:48:413
3103199633,47cyclictest27729-21chrt20:26:123
3103199633,47cyclictest16533-21chrt19:52:183
3103199633,45cyclictest0-21swapper/323:00:183
3103199632,48cyclictest18943-21taskset21:20:493
3103199632,46cyclictest23942-21chrt20:15:343
3103199623,47cyclictest32633-21taskset20:41:513
3103199623,47cyclictest29917-21chrt00:20:403
3103199623,47cyclictest0-21swapper/323:35:183
3103199623,46cyclictest6209-21chrt23:41:203
3103199623,46cyclictest6209-21chrt23:41:203
3103199623,46cyclictest2489-21chrt19:10:143
3103199622,48cyclictest16837-21taskset22:11:233
3103199622,47cyclictest30704-21taskset20:35:433
3103199622,47cyclictest2051-21chrt19:08:073
3103199602,48cyclictest0-21swapper/320:45:193
3103199602,48cyclictest0-21swapper/320:45:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional