You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 17:35
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Thu Mar 19, 2026 12:44:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30148213667,36sleep10-21swapper/107:00:151
30148213667,36sleep10-21swapper/107:00:151
31573991153,78cyclictest607-21apt-get12:30:092
3138921157,16sleep00-21swapper/007:04:380
3138921157,16sleep00-21swapper/007:04:380
30382211210,73sleep230429-21/usr/sbin/munin07:00:242
30382211210,73sleep230429-21/usr/sbin/munin07:00:242
31571991113,98cyclictest3122-21apt-get11:40:091
31571991046,74cyclictest1619-21apt-get09:50:111
31571991046,74cyclictest1619-21apt-get09:50:111
30953210218,49sleep30-21swapper/307:00:423
30953210218,49sleep30-21swapper/307:00:423
31571991003,51cyclictest29332-21ssh11:30:081
31571999651,37cyclictest0-21swapper/112:25:101
31571999651,37cyclictest0-21swapper/112:25:101
31571999551,38cyclictest8010-21sh09:05:491
31571999550,38cyclictest0-21swapper/111:24:551
31571999550,38cyclictest0-21swapper/111:24:551
78742940,7sleep00-21swapper/011:46:420
78742940,7sleep00-21swapper/011:46:420
3157199934,36cyclictest352-21runrttasks08:04:561
3157199934,36cyclictest352-21runrttasks08:04:561
3157199929,68cyclictest6397-21ssh11:45:141
3157199929,68cyclictest6397-21ssh11:45:141
31571999113,68cyclictest1886-21apt-get08:50:181
3157199893,39cyclictest21210-21ssh12:09:581
3157199893,32cyclictest352-21runrttasks10:06:551
3157199893,30cyclictest352-21runrttasks10:35:501
3157199892,29cyclictest17464-21ssh10:15:291
31566998963,17cyclictest277-21dbus-daemon09:49:560
31566998944,29cyclictest607-21apt-get12:30:160
31573998860,11cyclictest181rcu_preempt09:10:102
31571998869,13cyclictest365-21ntpd12:18:061
3157199885,39cyclictest12640-21unin-run07:44:561
3157199885,39cyclictest12640-21unin-run07:44:561
3157199882,40cyclictest22644-21/usr/sbin/munin10:25:281
3157199882,40cyclictest22644-21/usr/sbin/munin10:25:281
3157199882,31cyclictest23063-21latency_hist11:19:571
31566998863,17cyclictest2393-21sh09:50:460
31566998863,17cyclictest2393-21sh09:50:460
3157399876,64cyclictest2571-21apt-get11:40:092
3157199874,39cyclictest7478-21latency_hist09:59:581
3157199874,32cyclictest22324-21cat12:10:301
31571998742,37cyclictest0-21swapper/110:10:271
31571998742,36cyclictest0-21swapper/109:46:031
3157199873,41cyclictest12058-21readlink11:54:551
3157199873,40cyclictest24674-21fschecks_time08:20:171
3157199873,40cyclictest24674-21fschecks_time08:20:171
3157199872,66cyclictest28349-21kworker/u19:0+ext4-rsv-conversion08:16:081
31566998764,16cyclictest31553-21cyclictest07:55:230
3157199869,35cyclictest31351-21netstat08:40:221
31571998615,26cyclictest32568-21timerwakeupswit07:05:321
31571998615,26cyclictest32568-21timerwakeupswit07:05:321
31573998562,16cyclictest17946-21fschecks_time08:00:172
31573998562,16cyclictest17946-21fschecks_time08:00:172
31573998549,18cyclictest181rcu_preempt12:20:292
31573998549,18cyclictest181rcu_preempt12:20:292
3157199854,37cyclictest16095-21/usr/sbin/munin07:55:051
3157199854,34cyclictest352-21runrttasks12:22:041
3157199854,34cyclictest352-21runrttasks12:22:041
31566998558,19cyclictest1498-21apt-key08:49:580
31566998558,19cyclictest1498-21apt-key08:49:580
3157199844,36cyclictest10408-21latency_hist10:04:581
3157199843,30cyclictest7681-21latency_hist07:29:581
3157199843,30cyclictest7681-21latency_hist07:29:581
3157199842,38cyclictest10363-21ntpq09:10:231
3157199842,37cyclictest880-21munin-node10:40:051
31566998434,31cyclictest26893-21ssh09:38:120
31573998366,11cyclictest21532-21df12:10:122
3157199834,37cyclictest4859-21modprobe08:59:551
3157199834,26cyclictest12828-21/usr/sbin/munin09:15:241
3157199834,26cyclictest11001-21ls07:39:551
3157199834,26cyclictest11001-21ls07:39:551
3157199833,42cyclictest20100-21grep08:05:301
3157199833,42cyclictest20100-21grep08:05:301
3157199833,35cyclictest21247-21if_err_eth008:10:181
31571998329,47cyclictest0-21swapper/111:39:571
31571998329,47cyclictest0-21swapper/111:39:571
3157199832,38cyclictest26320-21sort08:25:181
3157199832,38cyclictest26320-21sort08:25:181
3157199825,33cyclictest26285-21/usr/sbin/munin11:25:301
3157199824,39cyclictest15805-21cron07:54:551
3157199824,39cyclictest15805-21cron07:54:551
3157199824,38cyclictest13127-21mailstats11:55:281
3157199824,38cyclictest13127-21mailstats11:55:281
3157199824,34cyclictest2150-21mailstats12:31:241
3157199824,33cyclictest15594-21rm12:00:141
3157199823,37cyclictest11274-21/usr/sbin/munin11:00:291
3157199823,29cyclictest16358-21taskset11:08:131
3157199822,24cyclictest27766-21date09:39:591
31571998221,53cyclictest0-21swapper/107:20:321
31571998221,53cyclictest0-21swapper/107:20:321
31571998212,30cyclictest2739-21sort10:45:221
31566998251,22cyclictest12228-21grep07:40:320
31566998251,22cyclictest12228-21grep07:40:320
31566998247,26cyclictest352-21runrttasks12:05:260
3157199816,11cyclictest0-21swapper/109:25:091
3157199814,36cyclictest7998-21fschecks_count07:30:171
3157199814,25cyclictest10986-21apt10:59:551
3157199814,24cyclictest13367-21ntp_kernel_pll_07:45:231
3157199814,24cyclictest13367-21ntp_kernel_pll_07:45:231
3157199813,35cyclictest1624-21idleruntime07:10:211
3157199813,28cyclictest32708-21fschecks_count08:45:161
3157199813,28cyclictest32708-21fschecks_count08:45:161
3157199813,27cyclictest30715-21latency_hist09:44:591
3157199813,27cyclictest30715-21latency_hist09:44:591
3157199813,27cyclictest22417-21ntpq09:30:231
3157199813,25cyclictest29257-21/usr/sbin/munin08:35:231
3157199813,24cyclictest18523-21latency_hist09:24:581
31571998130,43cyclictest0-21swapper/110:54:581
3157199812,32cyclictest26725-21unixbench_singl10:30:311
3157199812,22cyclictest17417-21/usr/sbin/munin11:10:121
31573998055,17cyclictest19944-21sh09:25:332
31573998049,23cyclictest19741-21latency08:05:212
31573998049,23cyclictest19741-21latency08:05:212
31573998047,12cyclictest181rcu_preempt07:44:582
31573998047,12cyclictest181rcu_preempt07:44:582
31573998031,8cyclictest181rcu_preempt09:40:342
31573998031,8cyclictest181rcu_preempt09:40:342
3157199804,39cyclictest3190-21fschecks_time07:15:171
3157199803,26cyclictest19999-21fschecks_time10:20:171
3156699805,43cyclictest15420-21grep07:50:300
3156699805,43cyclictest15420-21grep07:50:300
31573997933,16cyclictest181rcu_preempt09:55:422
3157199795,33cyclictest5334-21fschecks_count09:00:181
3157399787,55cyclictest1647-21ssh09:50:102
3157399787,55cyclictest1647-21ssh09:50:102
31573997842,16cyclictest181rcu_preempt10:40:112
3157199783,27cyclictest27828-21fschecks_count08:30:171
31566997848,20cyclictest13981-21latency_hist11:04:590
31573997762,10cyclictest1650-21kworker/u17:4-rpciod10:30:232
31573997760,11cyclictest23338-21ssh09:31:402
31573997746,16cyclictest18349-21/usr/sbin/munin12:05:212
3157399773,34cyclictest8434-21taskset10:55:112
31573997723,13cyclictest0-21swapper/207:55:152
31566997755,13cyclictest23261-21diskmemload10:25:240
31566997755,13cyclictest23261-21diskmemload10:25:240
31566997752,18cyclictest31553-21cyclictest10:51:250
31566997748,20cyclictest16947-21kworker/u17:2+nfsiod11:29:110
3156699772,54cyclictest9130-21apt-get07:35:150
3156699772,54cyclictest9130-21apt-get07:35:150
31573997656,14cyclictest24781-21kernelversion08:20:202
31573997656,14cyclictest24781-21kernelversion08:20:202
31573997654,14cyclictest200-21systemd-journal12:19:222
31573997628,37cyclictest14855-21ntp_kernel_err11:05:222
31573997557,13cyclictest1-21systemd12:04:562
31573997553,16cyclictest15885-21latency_hist07:54:572
31573997553,16cyclictest15885-21latency_hist07:54:572
31573997550,17cyclictest20539-21fschecks_count11:15:162
31573997548,10cyclictest181rcu_preempt09:49:582
31573997545,9cyclictest181rcu_preempt11:31:542
31573997532,25cyclictest21839-21unixbench_multi08:10:302
3157399752,45cyclictest39-21ksoftirqd/209:09:592
31566997557,12cyclictest277-21dbus-daemon12:19:560
31566997547,22cyclictest3143-21ssh11:40:090
3156699754,44cyclictest352-21runrttasks12:28:570
3156699754,44cyclictest352-21runrttasks12:28:570
3156699754,44cyclictest32664-21sort10:40:310
3156699753,44cyclictest1886-21apt-get08:50:150
3156699752,49cyclictest15-21ksoftirqd/007:06:030
3156699752,49cyclictest15-21ksoftirqd/007:06:030
103742750,8sleep0211rcuc/009:10:230
31573997451,10cyclictest181rcu_preempt11:45:502
31573997451,10cyclictest181rcu_preempt11:45:502
31573997443,14cyclictest181rcu_preempt07:25:302
31573997443,14cyclictest181rcu_preempt07:25:302
31573997433,16cyclictest181rcu_preempt08:16:262
31566997460,8cyclictest9539-21ssh09:09:570
31566997457,11cyclictest297-21systemd-logind09:20:290
31566997456,12cyclictest21742-21latency_hist09:29:590
31566997449,17cyclictest23003-21latency_hist11:19:580
31566997435,27cyclictest0-21swapper/007:34:590
3156699743,33cyclictest19423-21cstates08:05:130
3156699743,33cyclictest19423-21cstates08:05:130
3156699742,41cyclictest12030-21modprobe11:54:540
31573997353,14cyclictest20495-21switchtime10:20:292
31573997353,13cyclictest29753-21diskmemload10:35:312
3157399734,52cyclictest25878-21unixbench_multi09:35:322
31573997341,15cyclictest181rcu_preempt10:45:092
31573997335,15cyclictest181rcu_preempt07:21:262
31573997335,15cyclictest181rcu_preempt07:21:262
3157399733,32cyclictest1-21systemd08:49:572
3157399733,32cyclictest1-21systemd08:49:572
31573997333,12cyclictest181rcu_preempt11:35:102
31573997333,12cyclictest181rcu_preempt11:35:102
31573997330,11cyclictest181rcu_preempt11:51:212
31566997356,11cyclictest20284-21ssh10:20:240
31566997354,13cyclictest6488-21diskmemload09:32:210
31566997354,12cyclictest31490-21ssh10:39:580
3156699734,28cyclictest15-21ksoftirqd/010:16:580
31573997244,8cyclictest181rcu_preempt08:25:162
31573997244,8cyclictest181rcu_preempt08:25:162
31573997244,11cyclictest181rcu_preempt09:15:072
31573997240,15cyclictest5230-21/usr/sbin/munin10:50:312
31573997238,11cyclictest181rcu_preempt11:24:452
31573997238,11cyclictest181rcu_preempt11:24:452
3157399722,28cyclictest14615-21grep10:10:312
3157399722,15cyclictest6664-21ssh09:04:592
31573997217,12cyclictest11423-21cstates11:00:112
3156699726,20cyclictest14286-21/usr/sbin/munin10:10:260
31566997250,15cyclictest200-21systemd-journal07:25:330
31566997250,15cyclictest200-21systemd-journal07:25:330
3156699724,33cyclictest32634-21fschecks_time11:35:150
3156699724,33cyclictest32634-21fschecks_time11:35:150
3156699723,37cyclictest22339-21http08:14:560
31566997213,35cyclictest15-21ksoftirqd/011:11:390
31573997151,13cyclictest7903-21ssh10:00:172
31573997148,8cyclictest181rcu_preempt11:55:172
31573997148,8cyclictest181rcu_preempt11:55:172
31573997129,13cyclictest181rcu_preempt11:26:552
31566997153,11cyclictest13236-21ssh09:15:200
31566997151,14cyclictest12679-21perl11:55:170
31566997151,14cyclictest12679-21perl11:55:170
31566997120,27cyclictest31443-21/usr/sbin/munin08:40:270
31573997049,15cyclictest16057-21fschecks_count09:20:172
3157399703,45cyclictest1568-21fschecks_time07:10:202
31573997026,14cyclictest0-21swapper/210:09:132
3156699706,36cyclictest27571-21diskstats12:20:150
3156699706,36cyclictest27571-21diskstats12:20:150
31566997054,10cyclictest29434-21sh11:30:110
31566997053,11cyclictest300-21rs:main0
31566997053,11cyclictest300-21rs:main0
31566997047,16cyclictest28609-21latency_hist10:34:590
31573996948,16cyclictest16718-21cpu10:15:112
31566996956,8cyclictest0-21swapper/010:00:310
31566996951,12cyclictest7536-21latency_hist09:59:590
3156699693,56cyclictest3776-21fschecks_time08:55:190
3156699693,53cyclictest4943-21latency_hist10:49:580
3156699693,38cyclictest17136-21latency_hist11:09:580
31566996930,24cyclictest15-21ksoftirqd/010:55:230
31566996915,13cyclictest17945-21/usr/sbin/munin08:00:170
31566996915,13cyclictest17945-21/usr/sbin/munin08:00:170
31573996846,16cyclictest765-21latency_hist07:09:582
31573996846,16cyclictest765-21latency_hist07:09:582
3157399682,46cyclictest352-21runrttasks08:59:582
3156699684,50cyclictest352-21runrttasks08:21:180
3156699684,50cyclictest352-21runrttasks08:21:180
31566996839,22cyclictest0-21swapper/012:10:270
3156699683,53cyclictest15353-21/usr/sbin/munin12:00:250
3156699682,46cyclictest1500-21fschecks_count07:10:190
31566996818,9cyclictest10642-21apt-get10:05:060
31573996749,10cyclictest31553-21cyclictest08:51:092
31573996747,13cyclictest17596-21perl11:10:182
31573996713,44cyclictest9654-21apt-get07:35:152
31573996713,44cyclictest9654-21apt-get07:35:152
31573996712,21cyclictest39-21ksoftirqd/208:39:572
3156699677,27cyclictest23296-21/usr/sbin/munin11:20:350
3156699677,27cyclictest23296-21/usr/sbin/munin11:20:350
31566996726,12cyclictest0-21swapper/009:40:320
31566996726,12cyclictest0-21swapper/009:40:320
31566996714,17cyclictest23566-21cat08:16:360
31573996648,11cyclictest31553-21cyclictest07:34:592
31573996633,13cyclictest181rcu_preempt08:30:292
31573996623,14cyclictest181rcu_preempt07:49:242
31573996623,14cyclictest181rcu_preempt07:49:242
31573996623,13cyclictest181rcu_preempt12:24:592
31573996623,13cyclictest181rcu_preempt12:24:592
3156699669,36cyclictest0-21swapper/008:25:200
3156699669,36cyclictest0-21swapper/008:25:200
31566996616,13cyclictest29660-21grep08:35:210
31566996613,16cyclictest28086-21chrt08:30:210
31573996540,8cyclictest181rcu_preempt08:40:192
31573996511,17cyclictest31470-21kworker/2:2-events07:15:122
3156699659,25cyclictest0-21swapper/009:00:170
31566996525,11cyclictest161ktimers/007:20:320
31566996525,11cyclictest161ktimers/007:20:320
31573996431,16cyclictest181rcu_preempt10:27:462
31573996431,16cyclictest181rcu_preempt10:27:462
3156699644,36cyclictest15-21ksoftirqd/007:19:280
295822640,3sleep30-21swapper/308:35:193
31580995945,10cyclictest451rcuc/310:28:373
31580995945,10cyclictest451rcuc/310:28:373
31580995818,26cyclictest0-21swapper/311:45:423
31580995818,26cyclictest0-21swapper/311:45:423
31580995744,8cyclictest451rcuc/311:56:223
31580995744,8cyclictest451rcuc/311:56:223
31580995642,9cyclictest451rcuc/312:20:343
31580995642,9cyclictest451rcuc/312:20:343
3158099553,41cyclictest3458-21chrt12:34:103
3158099553,39cyclictest5432-21chrt07:20:323
3158099553,39cyclictest5432-21chrt07:20:323
3158099553,39cyclictest27961-21taskset09:40:133
3158099553,39cyclictest27961-21taskset09:40:133
3158099547,34cyclictest8412-21taskset07:30:283
31580995441,9cyclictest451rcuc/309:37:193
31580995439,10cyclictest451rcuc/311:27:253
31580995439,10cyclictest451rcuc/308:04:103
31580995439,10cyclictest451rcuc/308:04:103
3158099543,41cyclictest3286-21chrt11:40:333
31580995339,9cyclictest451rcuc/310:50:413
31580995339,9cyclictest451rcuc/309:52:263
31580995339,9cyclictest451rcuc/309:52:263
31580995339,9cyclictest451rcuc/307:41:183
31580995339,9cyclictest451rcuc/307:41:183
31580995339,9cyclictest451rcuc/307:36:083
31580995339,9cyclictest451rcuc/307:36:083
3158099533,39cyclictest22642-21taskset09:30:293
3158099533,38cyclictest2011-21taskset11:38:223
3158099533,38cyclictest2011-21taskset11:38:223
3158099533,36cyclictest862-21taskset07:09:593
3158099524,37cyclictest16473-21chrt10:14:583
31580995239,8cyclictest451rcuc/311:17:243
31580995238,9cyclictest451rcuc/312:03:233
31580995238,10cyclictest451rcuc/310:04:053
31580995237,9cyclictest451rcuc/309:00:273
31580995236,11cyclictest451rcuc/307:45:113
31580995236,11cyclictest451rcuc/307:45:113
3158099522,39cyclictest31639-21taskset12:26:003
3158099522,39cyclictest31639-21taskset12:26:003
3158099522,37cyclictest8737-21taskset09:07:513
31580995139,8cyclictest451rcuc/312:13:323
31580995137,9cyclictest451rcuc/308:57:513
31580995135,11cyclictest451rcuc/307:07:293
3158099513,33cyclictest15211-21chrt07:50:283
3158099513,33cyclictest15211-21chrt07:50:283
31580995132,11cyclictest0-21swapper/308:50:123
3158099512,31cyclictest21627-21chrt08:10:283
259472510,8sleep30-21swapper/312:16:483
3158099503,36cyclictest23672-21taskset11:20:193
3158099503,36cyclictest23672-21taskset11:20:193
3158099503,35cyclictest19215-21chrt12:05:293
3158099503,34cyclictest17085-21taskset09:20:563
3158099503,33cyclictest24785-21chrt08:20:203
3158099503,33cyclictest24785-21chrt08:20:203
3158099503,33cyclictest18091-21taskset11:10:303
3158099503,30cyclictest15727-21chrt09:19:593
3158099502,37cyclictest11023-21taskset10:59:563
31580994935,9cyclictest451rcuc/308:19:083
3158099493,37cyclictest12258-21chrt10:06:563
3158099493,36cyclictest4218-21chrt10:47:573
3158099493,36cyclictest10433-21chrt09:10:263
3158099493,35cyclictest5943-21chrt09:56:023
3158099493,35cyclictest10712-21chrt11:51:153
3158099493,33cyclictest3417-21chrt07:15:223
3158099493,32cyclictest22355-21chrt10:24:553
3158099492,34cyclictest14148-21chrt11:04:593
3158099484,32cyclictest26609-21chrt08:25:263
3158099484,32cyclictest26609-21chrt08:25:263
31580994836,8cyclictest451rcuc/308:45:263
31580994836,8cyclictest451rcuc/308:45:263
31580994834,9cyclictest451rcuc/310:17:143
3158099483,36cyclictest21308-21chrt09:29:233
3158099483,35cyclictest19783-21taskset08:05:223
3158099483,35cyclictest19783-21taskset08:05:223
3158099483,32cyclictest1277-21chrt09:49:583
3158099482,36cyclictest7027-21taskset07:25:383
3158099482,36cyclictest7027-21taskset07:25:383
3158099482,34cyclictest31572-21taskset08:40:293
3158099473,34cyclictest15711-21chrt11:06:253
3158099463,33cyclictest28413-21chrt10:34:563
3158099463,33cyclictest1461-21chrt07:10:153
3158099463,31cyclictest31933-21chrt11:34:453
31580994626,13cyclictest451rcuc/310:35:503
3158099453,32cyclictest1993-21chrt10:44:573
31580994431,9cyclictest451rcuc/307:55:353
3158099423,29cyclictest28301-21chrt08:30:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional