You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 21:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sun Apr 19, 2026 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27713210121,56sleep21-21systemd07:02:532
2370621010,2sleep2351irq_work/210:31:052
2370621010,2sleep2351irq_work/210:31:052
2669829810,61sleep026774-21cat06:58:230
27827999742,48cyclictest22066-21apt-key08:22:551
27823999750,9cyclictest181rcu_preempt10:57:120
27823999750,9cyclictest181rcu_preempt10:57:120
27835999652,12cyclictest1912-21ntp_kernel_pll_08:58:202
2782799943,35cyclictest4169-21/usr/sbin/munin10:53:121
2782799943,35cyclictest4169-21/usr/sbin/munin10:53:121
2782799942,36cyclictest21907-21ssh10:28:141
2782799942,36cyclictest21907-21ssh10:28:141
2783599939,53cyclictest39-21ksoftirqd/211:38:032
27827999268,16cyclictest10508-21apt-key07:47:551
2782799924,33cyclictest502-21/usr/sbin/munin09:53:131
27827999242,43cyclictest9186-21idleruntime-cro10:07:511
2782799922,37cyclictest14320-21rm11:08:281
2782399923,71cyclictest22300-21apt-get08:23:150
2782799916,64cyclictest7400-21apt-get10:59:001
2782799914,31cyclictest22301-21/usr/sbin/munin11:23:211
2782799914,31cyclictest22301-21/usr/sbin/munin11:23:211
27827999135,48cyclictest0-21swapper/112:18:291
27827999135,48cyclictest0-21swapper/112:18:291
27835999071,13cyclictest14066-21processes11:08:252
2782799904,32cyclictest13973-21/usr/sbin/munin07:58:221
2782799903,36cyclictest26969-21/usr/sbin/munin09:43:141
2782799903,33cyclictest9657-21awk09:13:191
2782799902,26cyclictest2957350munin-node07:07:561
27827998941,41cyclictest0-21swapper/110:23:211
2782799893,32cyclictest7057-21latency_hist07:37:531
2782799893,32cyclictest7057-21latency_hist07:37:531
2782799893,31cyclictest14005-21/usr/sbin/munin12:03:291
2782799893,27cyclictest19046-21fschecks_count08:13:141
27823998935,10cyclictest181rcu_preempt10:22:210
27835998827,28cyclictest27389-21apt-get08:38:172
27835998827,28cyclictest27389-21apt-get08:38:172
2782799884,33cyclictest31522-21unixbench_multi08:48:281
2782799883,67cyclictest11171-21ssh11:58:201
2782799883,67cyclictest11171-21ssh11:58:201
2782799883,34cyclictest22599-21fschecks_time08:23:371
2782799883,33cyclictest9779-21grep10:08:141
2782799883,33cyclictest9779-21grep10:08:141
2782799883,33cyclictest15818-21sort10:18:091
2782399884,43cyclictest15-21ksoftirqd/007:20:010
2782399883,71cyclictest11089-21rm11:58:060
2782399883,71cyclictest11089-21rm11:58:060
2782799877,69cyclictest27389-21apt-get08:38:161
2782799877,69cyclictest27389-21apt-get08:38:161
2782799874,32cyclictest22875-21latency_hist12:17:541
2782799873,35cyclictest2265-21timerwakeupswit08:58:271
27827998733,48cyclictest0-21swapper/109:08:191
27827998733,48cyclictest0-21swapper/109:08:191
2782799873,32cyclictest13124-21fschecks_count10:13:351
2782799873,31cyclictest3984-21latency_hist07:27:551
2782799873,29cyclictest30702-21awk09:48:251
2782799873,29cyclictest30702-21awk09:48:251
2782799872,33cyclictest2218-21basename11:43:151
2782799872,33cyclictest2218-21basename11:43:151
2782799872,32cyclictest19048-21taskset12:11:071
27823998747,31cyclictest30973-21grep08:48:170
2782799863,35cyclictest18891-21timerandwakeup09:28:271
2782799863,34cyclictest26904-21http08:37:521
2782799863,32cyclictest31618-21latency_hist12:32:541
2782799862,32cyclictest30642-21/usr/sbin/munin10:43:151
2782799862,32cyclictest23893-21/usr/sbin/munin08:28:191
2782799862,32cyclictest16979-21latency_hist08:07:541
27823998649,9cyclictest29825-21ssh12:28:280
27823998616,34cyclictest21569-21awk09:33:190
2783599853,56cyclictest11063-21apt-get11:58:232
2783599853,56cyclictest11063-21apt-get11:58:232
2782799853,35cyclictest28289-21/usr/sbin/munin11:33:091
2782799853,35cyclictest28289-21/usr/sbin/munin11:33:091
2782799853,34cyclictest8932-21head11:53:291
2782799853,34cyclictest8932-21head11:53:291
2782799853,34cyclictest6595-21fschecks_count09:09:011
2782799853,34cyclictest6595-21fschecks_count09:09:011
2782799853,32cyclictest32116-21latency07:13:171
2782799853,32cyclictest11709-21grep07:49:281
2782799853,30cyclictest5000-21grep07:28:281
2782799853,30cyclictest25980-21taskset12:23:031
2782799853,22cyclictest5556-21/usr/sbin/munin11:48:251
27827998524,53cyclictest0-21swapper/109:27:511
2782799852,37cyclictest14886-21taskset09:22:551
2782799852,37cyclictest14886-21taskset09:22:551
2782799852,33cyclictest32229-21/usr/sbin/munin08:53:271
27835998451,8cyclictest181rcu_preempt10:54:342
27835998451,8cyclictest181rcu_preempt10:54:342
2782799843,36cyclictest18663-21grep08:12:531
27827998433,44cyclictest30331-21cron08:47:511
2782799843,32cyclictest31356-21cpu11:38:091
27827998428,7cyclictest0-21swapper/110:48:131
27827998424,6cyclictest0-21swapper/111:18:021
27827998424,52cyclictest0-21swapper/111:17:551
2782799842,34cyclictest2252-21taskset07:22:541
27823998444,23cyclictest27367-21kworker/u18:2-flush-0:3209:18:250
27823998444,23cyclictest27367-21kworker/u18:2-flush-0:3209:18:250
2782799834,31cyclictest10669-21fschecks_time11:03:141
2782799834,31cyclictest10669-21fschecks_time11:03:141
2782799833,34cyclictest27004-21sshd10:36:541
2782799833,33cyclictest28950-21unixbench_singl07:03:281
2782799833,33cyclictest28950-21unixbench_singl07:03:281
2782799833,32cyclictest27938-21fschecks_count10:38:141
27823998370,8cyclictest0-21swapper/009:13:170
27823998346,10cyclictest14001-21awk11:08:240
27835998259,15cyclictest22066-21apt-key08:22:562
2782799823,34cyclictest4545-21unixbench_multi09:58:291
2782799823,34cyclictest4545-21unixbench_multi09:58:291
2782799823,31cyclictest27680-21latency_hist07:02:561
2782799823,30cyclictest26359-21runrttasks09:42:001
27827998214,28cyclictest21211-21df_inode09:33:111
27823998256,19cyclictest8795-21sh09:12:530
27823998256,19cyclictest8795-21sh09:12:530
2783599819,52cyclictest12747-21apt-get10:13:102
27835998162,13cyclictest1-21systemd12:09:542
27835998161,13cyclictest6373-21latency_hist10:02:552
27835998161,13cyclictest6373-21latency_hist10:02:552
2782799813,35cyclictest25602-21fschecks_time11:28:141
2782799813,33cyclictest7266-21/usr/sbin/munin07:38:141
2782799813,33cyclictest7266-21/usr/sbin/munin07:38:141
2782799813,31cyclictest19131-21latency_hist11:17:541
27827998120,11cyclictest0-21swapper/107:55:231
27823998165,12cyclictest27809-21cyclictest11:39:040
27823998162,13cyclictest1-21systemd10:27:520
2782399814,33cyclictest15-21ksoftirqd/009:26:110
27823998130,16cyclictest181rcu_preempt11:14:480
27823998130,16cyclictest181rcu_preempt11:14:480
27835998063,11cyclictest5976-21ssh09:07:592
27835998063,11cyclictest5976-21ssh09:07:592
27835998061,13cyclictest11373-21kworker/u18:1+flush-179:010:58:332
27835998028,41cyclictest30838-21df10:43:102
2782399806,65cyclictest19676-21snmpd10:13:100
27823998037,11cyclictest181rcu_preempt09:30:410
27835997958,7cyclictest181rcu_preempt12:17:162
27823997959,14cyclictest27809-21cyclictest10:07:570
27823997959,14cyclictest27809-21cyclictest10:07:570
27823997957,15cyclictest27809-21cyclictest10:42:500
27835997861,9cyclictest0-21swapper/210:23:252
27835997860,11cyclictest17012-21ssh10:19:002
27823997860,13cyclictest21913-21apt-key08:22:540
27823997855,17cyclictest31121-21latency_hist11:37:540
27823997855,17cyclictest31121-21latency_hist11:37:540
27823997852,17cyclictest4369-21switchtime09:58:270
27823997852,17cyclictest4369-21switchtime09:58:270
27823997850,19cyclictest22988-21/usr/sbin/munin12:18:180
27823997850,19cyclictest22988-21/usr/sbin/munin12:18:180
27823997849,21cyclictest8493-21mailstats10:59:220
27823997848,21cyclictest27400-21ls10:37:530
27823997840,9cyclictest181rcu_preempt07:34:030
27823997840,9cyclictest181rcu_preempt07:34:030
27823997835,12cyclictest181rcu_preempt07:49:260
27835997761,10cyclictest32310-21cpuspeed08:53:082
27835997761,10cyclictest10647-21ssh09:14:482
27835997755,16cyclictest30793-21threads07:08:292
27835997755,16cyclictest16069-21netstat08:03:182
27835997752,7cyclictest181rcu_preempt09:48:182
27835997752,7cyclictest181rcu_preempt09:48:182
27835997727,13cyclictest181rcu_preempt09:57:532
2782399775,48cyclictest15-21ksoftirqd/010:51:560
27823997728,12cyclictest181rcu_preempt11:51:560
27823997710,39cyclictest15-21ksoftirqd/008:58:460
27835997656,14cyclictest5555-21latency_hist07:32:552
27835997651,11cyclictest181rcu_preempt11:19:382
27835997646,9cyclictest181rcu_preempt08:48:272
27835997628,13cyclictest181rcu_preempt11:48:272
2782399764,61cyclictest27417-21sshd08:38:200
2782399764,61cyclictest27417-21sshd08:38:200
27823997639,15cyclictest181rcu_preempt09:47:060
27823997616,11cyclictest181rcu_preempt11:32:500
27835997558,12cyclictest2764-21diskmemload09:38:272
27835997558,11cyclictest298-21in:imuxsock08:27:512
27835997554,15cyclictest12475-21kernelversion09:18:182
27835997554,15cyclictest12475-21kernelversion09:18:182
27835997552,16cyclictest27367-21kworker/u18:2-rpciod09:28:282
27823997553,18cyclictest27809-21cyclictest12:23:290
27823997547,12cyclictest181rcu_preempt09:54:270
27823997534,9cyclictest0-21swapper/012:12:540
2782399752,48cyclictest8183-21grep11:53:150
2782399752,48cyclictest8183-21grep11:53:150
27835997456,12cyclictest297-21systemd-logind11:32:522
27835997453,8cyclictest181rcu_preempt07:13:052
27835997449,7cyclictest181rcu_preempt11:33:022
27835997449,7cyclictest181rcu_preempt11:33:022
27835997446,21cyclictest16427-21kworker/u19:0+nfsiod09:33:172
27835997439,10cyclictest181rcu_preempt08:33:042
27835997425,15cyclictest181rcu_preempt10:12:552
27835997425,15cyclictest181rcu_preempt10:12:552
27835997418,15cyclictest181rcu_preempt07:33:142
27835997418,15cyclictest181rcu_preempt07:33:142
27823997457,11cyclictest15081-21unixbench_singl12:03:280
27823997455,11cyclictest31472-21latency_hist07:12:540
27823997447,8cyclictest181rcu_preempt09:07:330
27823997447,8cyclictest181rcu_preempt09:07:330
27823997438,8cyclictest181rcu_preempt10:43:230
27823997438,29cyclictest0-21swapper/011:43:170
27823997438,29cyclictest0-21swapper/011:43:170
2782399743,37cyclictest6495-21/usr/sbin/munin10:03:290
27823997425,26cyclictest15-21ksoftirqd/009:38:310
27835997357,10cyclictest29870-21ssh10:41:332
27835997352,14cyclictest298-21in:imuxsock08:17:512
27835997350,13cyclictest10979-21latency_hist11:57:552
27835997349,14cyclictest0-21swapper/207:57:542
27835997346,8cyclictest181rcu_preempt12:25:152
2783599733,34cyclictest29260-21fschecks_time08:43:132
27835997333,14cyclictest10475-21latency_hist07:47:542
27823997348,19cyclictest26879-21latency_hist08:37:550
27823997337,15cyclictest181rcu_preempt07:27:310
27835997253,13cyclictest2374-21systemd-journal07:07:532
27835997253,13cyclictest2374-21systemd-journal07:07:532
27835997251,14cyclictest27809-21cyclictest11:47:542
27835997251,14cyclictest27809-21cyclictest11:47:542
27835997251,11cyclictest0-21swapper/210:07:542
27835997245,18cyclictest352-21runrttasks11:23:262
27835997245,18cyclictest352-21runrttasks11:23:262
27835997222,13cyclictest181rcu_preempt11:53:092
27823997247,8cyclictest181rcu_preempt09:51:270
27823997247,8cyclictest181rcu_preempt09:51:270
27835997156,10cyclictest27809-21cyclictest09:47:502
27835997149,16cyclictest298-21in:imuxsock10:37:502
27835997129,12cyclictest181rcu_preempt07:23:242
27823997152,13cyclictest12547-21/usr/sbin/munin07:53:130
2782399714,55cyclictest12066-21ssh11:05:070
2782399714,55cyclictest12066-21ssh11:05:070
27823997134,13cyclictest181rcu_preempt08:12:550
2782399713,28cyclictest15816-21fschecks_time08:03:130
27823997125,34cyclictest20398-21if_eth012:13:160
27835997052,8cyclictest0-21swapper/209:09:182
27835997052,8cyclictest0-21swapper/209:09:182
27835997050,11cyclictest0-21swapper/212:03:142
27835997048,16cyclictest24138-21fschecks_time08:28:142
27835997030,9cyclictest181rcu_preempt11:16:332
27835997030,9cyclictest181rcu_preempt11:16:332
2782399705,45cyclictest5001-21unixbench_singl07:28:280
2782399704,45cyclictest14090-21cstates07:58:110
27823997042,20cyclictest1133-21modprobe08:57:510
27823997036,17cyclictest181rcu_preempt11:27:440
27823997036,17cyclictest181rcu_preempt11:27:440
2782399703,38cyclictest20350-21latency_hist08:17:540
2782399702,42cyclictest15-21ksoftirqd/007:13:200
27835996951,12cyclictest15181-21sh09:23:092
27835996947,17cyclictest0-21swapper/211:03:282
27835996947,17cyclictest0-21swapper/211:03:282
27835996944,9cyclictest181rcu_preempt12:22:352
27835996944,9cyclictest181rcu_preempt12:22:352
27823996940,10cyclictest181rcu_preempt08:47:510
27823996939,10cyclictest181rcu_preempt07:38:080
27823996939,10cyclictest181rcu_preempt07:38:080
2782399692,55cyclictest352-21runrttasks10:28:310
2782399692,55cyclictest352-21runrttasks10:28:310
27835996850,12cyclictest17552-21kernelversion08:08:162
27835996845,15cyclictest2374-21systemd-journal10:48:542
27835996839,7cyclictest181rcu_preempt12:29:022
27835996826,13cyclictest181rcu_preempt07:58:152
2782399683,35cyclictest23893-21/usr/sbin/munin08:28:130
2782399683,32cyclictest20308-21timerwakeupswit11:18:280
2783599673,36cyclictest11480-21mailstats07:49:262
27835996637,23cyclictest2276-21latency_hist07:22:542
27823996646,14cyclictest10347-21apt-key07:47:540
189762660,7sleep30-21swapper/312:10:553
27823996542,12cyclictest0-21swapper/007:03:280
27823996542,12cyclictest0-21swapper/007:03:280
2783599643,46cyclictest7848-21/usr/sbin/munin07:38:242
2783599643,46cyclictest7848-21/usr/sbin/munin07:38:242
2784199594,38cyclictest9890-21chrt10:08:163
2784199594,38cyclictest9890-21chrt10:08:163
2784199553,39cyclictest20789-21chrt09:32:543
2784199543,40cyclictest7300-21taskset10:03:263
2784199543,39cyclictest3004-21chrt07:23:233
2784199543,39cyclictest12134-21chrt11:05:183
2784199543,39cyclictest12134-21chrt11:05:183
2784199543,34cyclictest23881-21chrt11:24:473
2784199543,34cyclictest23881-21chrt11:24:473
2651825415,22sleep30-21swapper/306:58:173
2784199533,37cyclictest13625-21chrt11:08:143
2784199533,35cyclictest25849-21chrt12:22:543
2784199533,35cyclictest25849-21chrt12:22:543
2784199532,38cyclictest31965-21taskset07:13:143
2784199526,30cyclictest0-21swapper/310:13:123
2784199524,33cyclictest18881-21chrt08:13:083
2784199523,35cyclictest31119-21taskset08:48:203
2784199523,32cyclictest1798-21taskset11:42:553
2784199523,32cyclictest1798-21taskset11:42:553
2784199522,36cyclictest6498-21chrt09:08:463
2784199522,36cyclictest6498-21chrt09:08:463
2784199522,36cyclictest25938-21chrt10:33:533
2784199522,35cyclictest7336-21chrt07:38:093
2784199522,35cyclictest7336-21chrt07:38:093
2784199513,38cyclictest2808-21chrt09:02:543
2784199513,38cyclictest1167-21chrt09:53:213
2784199513,36cyclictest3714-21chrt09:03:233
2784199513,36cyclictest3714-21chrt09:03:233
2784199513,36cyclictest21709-21chrt10:28:063
2784199513,36cyclictest21709-21chrt10:28:063
2784199513,35cyclictest29560-21chrt12:28:223
2784199512,38cyclictest16002-21chrt12:05:573
2784199512,38cyclictest14283-21chrt09:21:393
2784199512,38cyclictest14283-21chrt09:21:393
2784199512,33cyclictest16382-21chrt11:13:043
2784199512,33cyclictest16382-21chrt11:13:043
2784199503,38cyclictest25687-21chrt11:28:163
2784199503,37cyclictest30012-21chrt07:08:043
2784199503,35cyclictest10205-21chrt11:57:013
2784199503,35cyclictest10205-21chrt11:57:013
2784199503,34cyclictest4774-21chrt09:58:523
2784199503,34cyclictest4774-21chrt09:58:523
2784199503,34cyclictest1954-21chrt10:48:243
2784199502,36cyclictest9712-21taskset11:01:543
2784199502,36cyclictest12092-21taskset07:52:513
2784199502,34cyclictest22595-21chrt08:23:373
2784199502,34cyclictest10470-21chrt09:14:173
2784199502,32cyclictest30907-21chrt10:43:133
2784199499,26cyclictest0-21swapper/311:58:073
2784199499,26cyclictest0-21swapper/311:58:073
2784199493,36cyclictest1558-21taskset11:42:513
2784199493,35cyclictest15762-21chrt10:18:093
2784199493,33cyclictest20452-21chrt08:18:023
2784199492,35cyclictest26371-21taskset09:42:023
2784199492,34cyclictest20532-21chrt11:18:443
2784199483,33cyclictest23227-21chrt09:36:353
2784199474,29cyclictest28810-21chrt08:42:533
2784199474,29cyclictest28810-21chrt08:42:533
2784199473,34cyclictest12885-21chrt07:53:203
2784199463,34cyclictest28833-21chrt11:33:183
2784199463,34cyclictest28833-21chrt11:33:183
2784199463,32cyclictest15321-21taskset08:02:513
2784199463,31cyclictest19578-21chrt10:23:283
2784199463,28cyclictest4218-21chrt07:28:133
2784199462,32cyclictest4817-21chrt11:48:013
2784199462,29cyclictest1076-21taskset07:18:123
2784199453,31cyclictest5065-21chrt10:53:263
2784199453,31cyclictest5065-21chrt10:53:263
2784199443,31cyclictest30608-21chrt09:48:223
2784199443,31cyclictest30608-21chrt09:48:223
2784199443,31cyclictest29283-21taskset08:43:143
2784199443,28cyclictest15750-21chrt09:23:253
2784199434,26cyclictest29539-21chrt07:07:553
2784199434,26cyclictest29539-21chrt07:07:553
2784199433,30cyclictest28444-21taskset12:27:203
2784199433,29cyclictest32254-21taskset08:53:063
2784199433,28cyclictest20045-21chrt12:13:063
2784199423,29cyclictest23910-21chrt08:28:033
2784199422,30cyclictest28874-21taskset09:45:393
2784199413,28cyclictest27857-21taskset10:38:113
2784199413,27cyclictest26098-21taskset08:33:213
2784199413,25cyclictest5654-21taskset07:32:583
2784199413,25cyclictest5654-21taskset07:32:583
2784199412,29cyclictest8894-21taskset07:43:033
2784199393,26cyclictest16795-21taskset08:06:123
2784199392,28cyclictest18210-21taskset08:08:433
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional