You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-25 - 16:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Sat Jul 25, 2026 12:44:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
37899911192,13cyclictest23886-21latency_hist11:06:201
287121089,34sleep12872-21ntp_states06:51:581
37859910792,8cyclictest0-21swapper/010:01:370
37899910646,36cyclictest0-21swapper/110:01:001
37899910561,35cyclictest0-21swapper/111:21:551
37859910310,84cyclictest4228-21ssh12:21:510
37859910310,84cyclictest4228-21ssh12:21:510
3785991023,75cyclictest11490-21apt-get08:56:330
3785991023,75cyclictest11490-21apt-get08:56:330
3789999874,15cyclictest0-21swapper/109:56:191
378999983,50cyclictest6018-21apt-get07:01:321
378999983,50cyclictest6018-21apt-get07:01:321
3789999752,37cyclictest4620-21latency_hist08:36:201
378599967,36cyclictest0-21swapper/012:11:380
3795999557,32cyclictest0-21swapper/210:26:462
3795999544,41cyclictest2876-21runrttasks11:25:302
3789999551,39cyclictest3777-21cyclictest10:33:531
3789999551,39cyclictest3777-21cyclictest10:33:531
3789999475,12cyclictest0-21swapper/108:56:201
3789999474,14cyclictest10927-21diskmemload12:21:371
3789999474,14cyclictest10927-21diskmemload12:21:371
3789999359,24cyclictest6057-21sh08:41:171
3789999352,33cyclictest0-21swapper/110:16:431
3795999269,12cyclictest0-21swapper/211:01:212
3795999269,12cyclictest0-21swapper/211:01:212
3795999252,33cyclictest0-21swapper/210:47:172
3795999252,33cyclictest0-21swapper/210:47:172
3795999220,29cyclictest21223-21ls11:01:352
3789999276,12cyclictest3777-21cyclictest08:57:351
3789999276,12cyclictest3777-21cyclictest08:57:351
36422928,55sleep23679-21munin-run06:56:192
3795999146,37cyclictest0-21swapper/209:05:152
3795999146,37cyclictest0-21swapper/209:05:152
3795999136,47cyclictest0-21swapper/209:56:542
3789999172,12cyclictest11456-21processes07:16:501
3789999172,12cyclictest11456-21processes07:16:501
3789999146,38cyclictest10927-21diskmemload11:12:131
3789999115,31cyclictest24712-21sh11:06:471
3795999053,31cyclictest3777-21cyclictest12:21:112
3789999048,34cyclictest0-21swapper/109:46:421
3789999048,34cyclictest0-21swapper/109:46:421
378599903,47cyclictest23512-21apt07:56:170
3785999011,70cyclictest30711-21apt-get08:16:320
3789998946,37cyclictest10927-21diskmemload11:35:391
378999894,31cyclictest10927-21diskmemload11:56:331
3789998929,53cyclictest7657-21modprobe08:46:171
378599894,39cyclictest15-21ksoftirqd/009:37:330
379599884,38cyclictest4537-21latency_hist08:36:202
379599884,28cyclictest11505-21ssh08:56:332
379599884,28cyclictest11505-21ssh08:56:332
379599883,26cyclictest26896-21modprobe12:06:172
3789998839,41cyclictest10927-21diskmemload11:53:481
3789998839,41cyclictest10927-21diskmemload11:53:481
3785998859,20cyclictest10927-21diskmemload09:11:420
3785998859,20cyclictest10927-21diskmemload09:11:420
379599875,29cyclictest30286-21latency_hist12:11:212
3795998746,34cyclictest0-21swapper/209:46:422
3795998746,34cyclictest0-21swapper/209:46:422
379599874,26cyclictest4737-21grep08:36:362
379599873,39cyclictest23835-21apt-key07:56:212
3789998763,14cyclictest0-21swapper/108:26:411
3789998763,14cyclictest0-21swapper/108:26:411
378999872,35cyclictest2372-21ssh10:30:071
3785998759,19cyclictest7552-21latency_hist07:06:210
3785998759,19cyclictest7552-21latency_hist07:06:210
3785998719,55cyclictest17077-21apt09:06:170
3785998719,55cyclictest17077-21apt09:06:170
379599864,34cyclictest24571-21ntp_kernel_pll_07:57:312
379599864,34cyclictest24571-21ntp_kernel_pll_07:57:312
3795998640,39cyclictest0-21swapper/211:21:172
3795998636,43cyclictest0-21swapper/210:16:412
3795998634,45cyclictest0-21swapper/210:36:562
379599863,40cyclictest18014-21idleruntime-cro11:51:172
379599863,39cyclictest21785-21latency11:56:452
379599863,31cyclictest12969-21/usr/sbin/munin07:21:472
379599862,38cyclictest352-21runrttasks09:10:482
378999864,35cyclictest3976-21latency11:26:451
378999863,40cyclictest30068-21sh10:21:451
378999863,40cyclictest30068-21sh10:21:451
3789998615,27cyclictest29717-21ls09:26:441
3785998663,16cyclictest10927-21diskmemload10:46:550
3785998663,16cyclictest10927-21diskmemload10:46:550
3785998658,18cyclictest3665-21apt-key12:21:200
3785998641,24cyclictest15-21ksoftirqd/007:09:560
378599863,55cyclictest15-21ksoftirqd/007:25:570
3795998562,12cyclictest0-21swapper/208:26:402
3795998562,12cyclictest0-21swapper/208:26:402
379599855,28cyclictest27018-21/usr/sbin/munin11:11:402
379599854,37cyclictest32670-21latency09:31:452
379599854,36cyclictest30848-21sort08:17:332
379599853,26cyclictest29594-21fschecks_time09:26:412
3789998566,9cyclictest0-21swapper/107:26:511
3789998563,11cyclictest0-21swapper/109:41:541
378999853,36cyclictest352-21runrttasks10:05:261
378599859,45cyclictest15-21ksoftirqd/012:03:000
379599844,37cyclictest30565-21latency_hist08:16:222
379599844,35cyclictest23404-21modprobe10:11:172
3795998442,35cyclictest4618-21sh11:26:582
379599843,28cyclictest19418-21grep11:51:572
379599843,28cyclictest19418-21grep11:51:572
379599842,30cyclictest352-21runrttasks08:25:342
379599842,30cyclictest352-21runrttasks08:25:342
378999843,26cyclictest19398-21/usr/sbin/munin10:57:591
378999843,26cyclictest19398-21/usr/sbin/munin10:57:591
3789998432,45cyclictest0-21swapper/112:16:421
3789998432,44cyclictest0-21swapper/110:06:531
378599848,58cyclictest30215-21munin-run08:16:180
3785998462,17cyclictest3777-21cyclictest11:51:570
3785998462,17cyclictest3777-21cyclictest11:51:570
3795998361,15cyclictest6059-21sh09:41:432
379599835,36cyclictest352-21runrttasks12:23:062
379599835,36cyclictest352-21runrttasks12:23:062
379599834,35cyclictest14151-21cpu07:26:352
379599834,26cyclictest16114-21runrttasks10:52:012
3795998340,36cyclictest0-21swapper/209:16:422
3795998333,42cyclictest0-21swapper/210:34:322
3795998333,42cyclictest0-21swapper/210:34:322
379599833,33cyclictest10808-21taskset10:43:502
379599833,31cyclictest17341-21cpu07:36:342
379599833,27cyclictest9557-21/usr/sbin/munin08:51:522
379599833,27cyclictest11892-21fschecks_count09:51:412
3795998317,29cyclictest5517-21latency_hist07:01:212
3789998367,9cyclictest0-21swapper/108:06:551
3789998362,16cyclictest3777-21cyclictest07:57:371
3789998362,16cyclictest3777-21cyclictest07:57:371
378999835,38cyclictest17782-21latency_hist10:56:201
3789998347,31cyclictest5966-21kworker/u17:0-rpciod09:16:241
378999834,37cyclictest352-21runrttasks11:48:271
378999834,34cyclictest26909-21sendmail-msp12:06:171
378999833,39cyclictest8726-21grep10:41:171
378599835,41cyclictest15-21ksoftirqd/010:20:530
378599834,38cyclictest15-21ksoftirqd/012:06:370
378599832,53cyclictest15-21ksoftirqd/008:37:390
379599823,40cyclictest23713-21/usr/sbin/munin10:11:402
379599823,40cyclictest23713-21/usr/sbin/munin10:11:402
3795998217,26cyclictest388-21sh12:15:482
3789998265,9cyclictest0-21swapper/108:17:501
3789998262,14cyclictest16653-21unixbench_singl07:31:571
378999823,23cyclictest22378-21fschecks_count07:51:431
3789998228,46cyclictest0-21swapper/110:41:341
3789998227,48cyclictest0-21swapper/112:06:351
378999822,29cyclictest14886-21/usr/sbin/munin09:01:441
378999822,29cyclictest14886-21/usr/sbin/munin09:01:441
3789998215,29cyclictest24362-21diskmemload10:11:501
3789998215,29cyclictest24362-21diskmemload10:11:501
3785998254,7cyclictest181rcu_preempt10:35:270
3785998254,7cyclictest181rcu_preempt10:35:270
378599822,54cyclictest15-21ksoftirqd/010:12:250
378599822,54cyclictest15-21ksoftirqd/010:12:250
378599822,49cyclictest15-21ksoftirqd/009:54:250
3785998224,35cyclictest15-21ksoftirqd/007:20:500
3785998224,35cyclictest15-21ksoftirqd/007:20:500
379599814,25cyclictest12660-21fschecks_time11:41:402
379599814,25cyclictest12660-21fschecks_time11:41:402
379599814,22cyclictest20359-21/usr/sbin/munin09:11:412
379599814,22cyclictest20359-21/usr/sbin/munin09:11:412
379599813,33cyclictest352-21runrttasks07:46:132
379599813,29cyclictest9463-21sshd11:36:352
379599813,29cyclictest9463-21sshd11:36:352
379599813,27cyclictest3082-21sort09:36:432
3795998128,41cyclictest0-21swapper/208:46:362
3795998128,41cyclictest0-21swapper/208:46:362
378999813,39cyclictest12996-21ntpq07:21:471
378999813,37cyclictest352-21runrttasks09:12:431
378999813,37cyclictest352-21runrttasks09:12:431
3789998128,45cyclictest0-21swapper/111:41:441
3789998128,45cyclictest0-21swapper/111:41:441
378599814,61cyclictest15010-21modprobe11:46:160
378599814,61cyclictest15010-21modprobe11:46:160
378599813,54cyclictest15-21ksoftirqd/008:51:070
378599813,54cyclictest15-21ksoftirqd/008:51:070
378599813,51cyclictest15-21ksoftirqd/007:58:450
378599813,51cyclictest15-21ksoftirqd/007:58:450
378599812,52cyclictest15-21ksoftirqd/009:51:210
378599812,52cyclictest15-21ksoftirqd/009:51:210
379599804,26cyclictest6664-21/usr/sbin/munin11:31:382
379599803,36cyclictest15977-21fschecks_time07:31:422
379599803,33cyclictest8242-21processes07:06:512
379599803,26cyclictest20623-21cpuspeed07:46:352
379599803,26cyclictest20623-21cpuspeed07:46:352
379599803,25cyclictest30309-21grep10:21:522
379599803,25cyclictest30309-21grep10:21:522
3795998028,44cyclictest0-21swapper/211:08:272
379599802,26cyclictest9679-21grep07:11:452
379599802,26cyclictest6326-21apt-get08:41:332
378999804,35cyclictest31120-21runrttasks12:12:421
378999803,37cyclictest32370-21ssh09:31:371
378999803,27cyclictest27318-21unixbench_multi09:21:551
378999803,25cyclictest17492-21cpu09:06:351
378999802,29cyclictest11509-21runrttasks11:39:431
378999802,29cyclictest11509-21runrttasks11:39:431
3789998012,29cyclictest21590-21seq07:47:201
3789998012,29cyclictest21590-21seq07:47:201
3785998054,20cyclictest10927-21diskmemload09:26:460
378599803,44cyclictest15-21ksoftirqd/006:57:370
378599803,39cyclictest3485-21/usr/sbin/munin11:26:550
378599803,37cyclictest15-21ksoftirqd/008:02:180
378599802,50cyclictest15-21ksoftirqd/011:15:590
3785998023,32cyclictest15-21ksoftirqd/010:00:110
379599794,40cyclictest18946-21/usr/sbin/munin10:03:172
379599794,37cyclictest26269-21ssh09:21:292
379599794,24cyclictest10849-21/usr/sbin/munin07:16:462
379599794,24cyclictest10849-21/usr/sbin/munin07:16:462
379599793,28cyclictest27747-21ntp_kernel_err08:06:472
379599792,31cyclictest5581-21cat07:01:242
379599792,31cyclictest5581-21cat07:01:242
378999793,35cyclictest28961-21cstates08:11:361
378999793,35cyclictest10782-21latency_hist07:16:211
378999793,25cyclictest26524-21tail08:01:551
378999793,24cyclictest18741-21cat07:41:201
378999792,25cyclictest32671-21awk11:21:171
3789997915,28cyclictest4425-21seq09:38:381
378599795,34cyclictest15-21ksoftirqd/007:26:300
3785997951,23cyclictest3777-21cyclictest11:21:200
378599793,41cyclictest15-21ksoftirqd/011:47:580
378599793,37cyclictest15-21ksoftirqd/009:21:470
378599792,44cyclictest15-21ksoftirqd/008:10:450
3795997817,52cyclictest26068-21needreboot08:01:462
3789997864,7cyclictest0-21swapper/107:41:551
378999782,23cyclictest12366-21/usr/sbin/munin10:46:421
378999782,23cyclictest12366-21/usr/sbin/munin10:46:421
3789997815,21cyclictest8082-21expr08:46:361
3789997815,21cyclictest8082-21expr08:46:361
3785997855,15cyclictest21494-21diskstats11:56:400
378599782,48cyclictest15-21ksoftirqd/011:23:150
3785997810,29cyclictest15-21ksoftirqd/007:13:040
378999773,24cyclictest32490-21fschecks_count08:21:411
378999773,24cyclictest32490-21fschecks_count08:21:411
378999773,23cyclictest3979-21apt-get06:56:301
378999772,25cyclictest7923-21fschecks_time07:06:411
378599774,40cyclictest15-21ksoftirqd/009:07:110
378599774,30cyclictest15-21ksoftirqd/011:03:520
378599773,49cyclictest15-21ksoftirqd/011:07:240
3785997728,26cyclictest15-21ksoftirqd/009:43:290
378599772,24cyclictest476-21fschecks_time10:26:410
378599763,44cyclictest15-21ksoftirqd/007:46:060
378599763,43cyclictest18196-21ssh10:56:330
378599763,43cyclictest18196-21ssh10:56:330
3785997634,25cyclictest15-21ksoftirqd/008:43:220
3785997630,22cyclictest15-21ksoftirqd/011:33:000
3785997554,15cyclictest352-21runrttasks10:41:510
378599753,51cyclictest15-21ksoftirqd/010:51:390
3785997534,31cyclictest0-21swapper/009:16:560
378599752,30cyclictest15-21ksoftirqd/011:36:590
378599752,30cyclictest15-21ksoftirqd/011:36:590
378599744,37cyclictest21826-21unixbench_singl10:06:570
3785997440,24cyclictest18650-21modprobe07:41:170
378599743,59cyclictest32291-21readlink10:26:170
378599743,59cyclictest32291-21readlink10:26:170
378599743,43cyclictest15-21ksoftirqd/008:35:280
24592749,41sleep019676-21snmpd06:51:400
378599738,41cyclictest15-21ksoftirqd/008:30:340
378599738,41cyclictest15-21ksoftirqd/008:30:340
378599734,37cyclictest15-21ksoftirqd/007:50:230
378599734,37cyclictest15-21ksoftirqd/007:50:230
378599733,27cyclictest15-21ksoftirqd/010:38:420
3785997321,26cyclictest15-21ksoftirqd/009:33:190
61382720,8sleep30-21swapper/312:24:373
61382720,8sleep30-21swapper/312:24:373
380299723,38cyclictest0-21swapper/312:11:323
3785997210,42cyclictest15-21ksoftirqd/008:53:280
378599713,36cyclictest15-21ksoftirqd/007:33:340
380299693,54cyclictest11507-21chrt08:56:333
380299693,54cyclictest11507-21chrt08:56:333
378599693,51cyclictest1211-21idleruntime-cro08:26:180
378599693,51cyclictest1211-21idleruntime-cro08:26:180
3802996715,34cyclictest0-21swapper/310:01:383
35332674,15sleep30-21swapper/306:54:533
3802996312,28cyclictest0-21swapper/308:17:113
380299573,39cyclictest23985-21taskset07:56:223
380299573,39cyclictest23985-21taskset07:56:223
380299563,38cyclictest17376-21chrt10:01:203
380299554,37cyclictest22941-21chrt07:51:563
380299553,40cyclictest32302-21taskset08:21:293
380299553,40cyclictest32302-21taskset08:21:293
380299553,39cyclictest7546-21taskset10:37:573
380299543,38cyclictest29969-21chrt12:11:193
380299543,36cyclictest6009-21chrt09:41:413
380299543,36cyclictest17368-21taskset11:49:303
380299542,42cyclictest2275-21taskset10:29:503
380299542,37cyclictest11657-21taskset09:51:313
380299535,36cyclictest3108-21chrt08:31:303
380299533,37cyclictest26157-21chrt11:09:403
380299532,41cyclictest10682-21chrt07:16:203
3802995314,21cyclictest0-21swapper/310:06:223
380299523,37cyclictest27750-21chrt10:16:573
380299523,36cyclictest1798-21chrt11:22:253
380299522,32cyclictest5495-21chrt07:01:213
380299514,33cyclictest30014-21chrt10:21:433
380299514,33cyclictest30014-21chrt10:21:433
380299513,38cyclictest9704-21taskset10:41:503
380299513,36cyclictest8058-21chrt11:33:203
380299513,34cyclictest23437-21chrt09:16:363
380299513,33cyclictest20560-21chrt07:46:293
380299513,33cyclictest20560-21chrt07:46:293
380299513,33cyclictest18176-21taskset10:56:303
380299513,33cyclictest18176-21taskset10:56:303
380299512,39cyclictest26063-21chrt08:01:453
380299512,36cyclictest10940-21taskset11:38:063
380299512,36cyclictest10940-21taskset11:38:063
380299512,33cyclictest22232-21chrt09:14:043
380299512,33cyclictest22232-21chrt09:14:043
380299503,38cyclictest15481-21taskset07:31:183
380299503,36cyclictest6376-21chrt07:01:463
380299503,36cyclictest6376-21chrt07:01:463
380299503,34cyclictest24802-21taskset12:01:463
380299503,34cyclictest23923-21chrt10:11:383
380299503,34cyclictest23923-21chrt10:11:383
380299503,32cyclictest13555-21chrt11:42:083
380299503,32cyclictest13555-21chrt11:42:083
380299502,34cyclictest2117-21chrt12:17:153
3802994936,9cyclictest451rcuc/311:05:483
3802994935,10cyclictest451rcuc/311:20:073
3802994934,11cyclictest451rcuc/307:36:303
380299493,35cyclictest10904-21chrt08:56:173
380299493,32cyclictest3094-21chrt10:31:283
380299493,32cyclictest3094-21chrt10:31:283
380299492,37cyclictest6485-21chrt08:41:393
380299492,35cyclictest22271-21chrt11:56:563
380299492,32cyclictest16522-21chrt07:31:553
380299483,36cyclictest8647-21taskset09:46:283
380299483,36cyclictest8647-21taskset09:46:283
380299483,33cyclictest17459-21chrt09:06:343
380299483,32cyclictest28760-21chrt08:11:213
380299483,32cyclictest28222-21chrt11:12:073
380299483,31cyclictest15718-21chrt09:02:373
380299483,31cyclictest15718-21chrt09:02:373
380299482,37cyclictest4392-21chrt11:26:553
380299482,34cyclictest26852-21chrt09:21:473
380299482,33cyclictest13457-21taskset10:47:513
380299482,33cyclictest13457-21taskset10:47:513
380299465,31cyclictest17668-21taskset10:56:183
3802994633,8cyclictest451rcuc/311:55:103
3802994633,8cyclictest451rcuc/311:55:103
380299463,33cyclictest5349-21chrt09:41:173
380299463,33cyclictest461-21chrt09:31:523
380299453,33cyclictest7835-21chrt07:06:403
3802994529,9cyclictest451rcuc/308:50:343
3802994529,9cyclictest451rcuc/308:50:343
380299444,26cyclictest20489-21chrt07:46:213
380299423,25cyclictest13382-21chrt07:21:553
380299413,23cyclictest31961-21chrt09:31:193
3802994129,8cyclictest451rcuc/308:26:493
3802994129,8cyclictest451rcuc/308:26:493
3802994128,8cyclictest451rcuc/308:40:553
380299412,27cyclictest12302-21taskset07:21:223
380299412,27cyclictest12302-21taskset07:21:223
380299393,27cyclictest30033-21taskset08:14:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional