You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 17:10
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Fri Jul 24, 2026 12:44:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5952214310,94sleep06035-21unixbench_singl06:52:100
65499911933,53cyclictest15350irq/295-2188000.ethernet09:21:370
65499911933,53cyclictest15350irq/295-2188000.ethernet09:21:370
6549991183,91cyclictest29724-21apt-get10:16:410
6549991172,27cyclictest9002-21apt-get07:01:500
6549991153,27cyclictest26191-21ssh10:09:520
6549991148,24cyclictest12969-21sh11:35:390
6549991143,27cyclictest1030-21apt-get10:21:320
6549991143,27cyclictest1030-21apt-get10:21:320
6549991143,27cyclictest1030-21apt-get10:21:320
6549991122,28cyclictest5165-21ssh09:35:300
6549991113,29cyclictest16082-21sh10:46:350
6549991113,29cyclictest16082-21sh10:46:350
6549991112,28cyclictest30499-21ssh11:10:470
6549991112,28cyclictest30499-21ssh11:10:470
6549991102,29cyclictest22410-21latency_hist11:51:240
6549991093,24cyclictest32741-21sh11:13:400
6549991092,29cyclictest20315-21taskset07:36:370
6549991092,29cyclictest20315-21taskset07:36:370
6549991083,24cyclictest28939-21/usr/sbin/munin12:01:460
6549991082,28cyclictest25026-21ssh09:13:530
6549991082,25cyclictest1314-21ssh09:28:050
6549991074,30cyclictest17362-21cstates09:01:390
6549991073,24cyclictest28924-21ssh09:21:210
6549991073,24cyclictest28924-21ssh09:21:210
6549991072,28cyclictest25233-21taskset11:01:420
6549991063,30cyclictest12472-21ssh10:40:410
6549991063,30cyclictest12472-21ssh10:40:410
6549991042,27cyclictest3179-21sh11:18:240
6549991042,27cyclictest3179-21sh11:18:240
6549991042,22cyclictest24554-21ssh11:00:560
6549991042,22cyclictest24554-21ssh11:00:560
65499910341,54cyclictest0-21swapper/007:51:590
6549991033,26cyclictest13902-21diskmemload10:34:520
6549991033,11cyclictest0-21swapper/010:41:340
6549991032,27cyclictest23347-21ssh10:05:220
6549991032,25cyclictest4178-21ssh12:14:480
65689910264,14cyclictest181rcu_preempt09:41:362
6549991024,27cyclictest12938-21/usr/sbin/munin09:47:010
6549991024,27cyclictest12938-21/usr/sbin/munin09:47:010
6549991023,32cyclictest8131-21unin-run07:01:230
6549991023,23cyclictest29065-21ssh10:14:340
6549991023,23cyclictest10468-21rm12:25:020
6549991023,23cyclictest10468-21rm12:25:020
6549991022,25cyclictest18407-21ssh09:57:090
6549991022,23cyclictest15034-21ssh11:38:020
65499910216,68cyclictest0-21swapper/009:10:580
65499910216,68cyclictest0-21swapper/009:10:580
6549991012,28cyclictest10300-21taskset08:42:010
6549991012,28cyclictest10300-21taskset08:42:010
65499910117,53cyclictest0-21swapper/011:56:530
65499910117,53cyclictest0-21swapper/011:56:530
6549991003,31cyclictest352-21runrttasks12:21:050
6549991003,25cyclictest6267-21cut08:31:430
654999995,64cyclictest0-21swapper/011:43:330
654999995,64cyclictest0-21swapper/011:43:330
654999994,7cyclictest0-21swapper/011:21:320
654999992,25cyclictest11950-21taskset08:47:050
654999992,21cyclictest5579-21ssh10:28:090
63522996,18sleep10-21swapper/106:55:541
654999983,9cyclictest0-21swapper/011:26:320
654999983,10cyclictest31664-21cstates12:06:400
654999983,10cyclictest31664-21cstates12:06:400
654999982,22cyclictest24227-21taskset11:53:350
656899978,63cyclictest18201-21ssh09:56:582
654999975,7cyclictest0-21swapper/008:17:090
654999973,23cyclictest29365-21taskset08:02:060
654999973,23cyclictest29365-21taskset08:02:060
654999972,23cyclictest13588-21taskset08:52:250
654999972,22cyclictest18281-21taskset07:29:370
654999964,7cyclictest0-21swapper/008:56:340
654999964,7cyclictest0-21swapper/008:56:340
654999963,6cyclictest0-21swapper/007:41:380
654999962,30cyclictest19241-21sshd10:51:400
654999954,30cyclictest23526-21/usr/sbin/munin07:46:530
6568999432,11cyclictest0-21swapper/207:51:342
654999944,21cyclictest16647-21taskset07:24:350
6549999437,50cyclictest0-21swapper/009:46:230
654999943,32cyclictest17934-21sed09:56:250
654999943,26cyclictest12252-21apt-get07:11:340
654999942,23cyclictest27782-21taskset07:57:230
6568999363,24cyclictest13902-21diskmemload11:56:042
654999933,32cyclictest14814-21head07:16:590
654999933,28cyclictest556-21unin-run08:16:230
654999933,27cyclictest9142-21latency_hist08:41:240
654999923,18cyclictest6070-21fschecks_count09:36:430
654999923,18cyclictest6070-21fschecks_count09:36:430
654999922,25cyclictest4786-21fschecks_time08:26:590
654999922,24cyclictest11176-21ntpq07:06:520
6568999171,13cyclictest29726-21cron10:16:212
654999913,30cyclictest4212-21latency_hist08:26:240
654999913,30cyclictest4212-21latency_hist08:26:240
6568998972,10cyclictest12727-21kworker/u19:0+events_unbound09:16:212
6560998969,12cyclictest300-21rs:main1
6560998969,12cyclictest300-21rs:main1
656099893,68cyclictest30268-21apt-get10:16:431
654999894,25cyclictest18680-21/usr/sbin/munin07:31:440
6568998846,13cyclictest181rcu_preempt07:01:342
6560998847,12cyclictest181rcu_preempt10:21:361
6560998847,12cyclictest181rcu_preempt10:21:361
6560998847,12cyclictest181rcu_preempt10:21:361
654999883,23cyclictest29992-21/usr/sbin/munin08:06:530
654999883,23cyclictest29992-21/usr/sbin/munin08:06:530
6568998762,16cyclictest28554-21ssh12:01:372
656899846,55cyclictest5340-21ssh12:16:402
6568998443,13cyclictest181rcu_preempt11:16:562
6568998443,13cyclictest181rcu_preempt11:16:562
656899836,55cyclictest17460-21sshd10:16:462
6560998368,10cyclictest23717-21sh10:06:211
6568998253,8cyclictest181rcu_preempt08:31:252
6568998157,13cyclictest18908-21ssh10:51:252
6568998157,13cyclictest18908-21ssh10:51:252
6560998164,10cyclictest31982-21modprobe09:26:221
6560998164,10cyclictest31982-21modprobe09:26:221
6560998161,14cyclictest19062-21processes09:57:521
6568998035,32cyclictest548-21systemd-run08:16:232
6568997960,13cyclictest13902-21diskmemload11:31:402
6568997922,31cyclictest12146-21cat07:11:242
6560997959,14cyclictest22633-21sh11:51:391
6568997852,16cyclictest170-21jbd2/mmcblk2p2-807:21:402
6568997849,20cyclictest12834-21grep07:11:552
656899783,63cyclictest352-21runrttasks08:17:202
656899783,44cyclictest22850-21grep07:42:012
6560997840,8cyclictest181rcu_preempt10:11:421
656099782,48cyclictest2392-21fschecks_time12:11:451
6568997756,14cyclictest24900-21idleruntime-cro07:51:212
6568997756,13cyclictest6960-21latency_hist10:31:242
6568997740,26cyclictest0-21swapper/206:56:502
656099773,36cyclictest31-21ksoftirqd/108:31:221
656099772,37cyclictest31-21ksoftirqd/111:07:071
656099772,37cyclictest31-21ksoftirqd/111:07:071
6568997657,9cyclictest22440-21ssh11:51:252
6568997651,12cyclictest181rcu_preempt07:36:252
6568997645,13cyclictest181rcu_preempt07:56:392
656899763,44cyclictest39-21ksoftirqd/209:46:592
656899763,44cyclictest39-21ksoftirqd/209:46:592
6568997557,12cyclictest8505-21grep09:41:212
6568997557,12cyclictest8505-21grep09:41:212
6568997546,13cyclictest181rcu_preempt10:02:042
6568997544,13cyclictest181rcu_preempt10:51:322
6568997544,12cyclictest181rcu_preempt10:56:322
6568997544,12cyclictest181rcu_preempt10:56:322
6568997541,10cyclictest181rcu_preempt11:11:542
656899752,10cyclictest24088-21ssh10:06:372
6560997560,11cyclictest6540-21cyclictest11:16:391
6560997560,11cyclictest6540-21cyclictest11:16:391
6560997557,13cyclictest13902-21diskmemload09:38:171
6560997557,13cyclictest13902-21diskmemload09:38:171
6560997532,24cyclictest31-21ksoftirqd/111:42:141
6560997532,24cyclictest31-21ksoftirqd/111:42:141
656099752,40cyclictest31-21ksoftirqd/109:05:531
6568997456,12cyclictest1-21systemd12:16:232
656899742,23cyclictest8457-21sshd12:21:392
656899742,23cyclictest8457-21sshd12:21:392
6560997459,9cyclictest19208-21sh10:51:381
656099743,53cyclictest5285-21apt-get12:16:511
606327410,39sleep26072-21vmstat06:52:122
6568997355,12cyclictest1902-21ssh10:22:302
6568997355,12cyclictest1902-21ssh10:22:302
6568997355,12cyclictest1902-21ssh10:22:302
6568997353,12cyclictest0-21swapper/212:01:242
6568997353,12cyclictest0-21swapper/212:01:242
6568997348,18cyclictest10496-21latency10:36:482
6568997348,18cyclictest10496-21latency10:36:482
6568997348,13cyclictest0-21swapper/210:41:592
6568997347,16cyclictest27717-21kworker/u17:3-events_unbound09:21:382
6568997347,16cyclictest27717-21kworker/u17:3-events_unbound09:21:382
6568997347,11cyclictest181rcu_preempt09:08:302
6568997347,11cyclictest181rcu_preempt09:08:302
6560997356,12cyclictest3951-21latency_hist08:26:251
6560997356,12cyclictest3951-21latency_hist08:26:251
656099733,38cyclictest31-21ksoftirqd/111:56:511
656099733,38cyclictest31-21ksoftirqd/111:56:511
656099732,29cyclictest31-21ksoftirqd/111:22:571
657499723,57cyclictest18038-21chrt07:26:583
657499723,56cyclictest8661-21chrt12:21:453
657499723,56cyclictest8661-21chrt12:21:453
656899723,28cyclictest30326-21/usr/sbin/munin08:06:452
656899723,28cyclictest30326-21/usr/sbin/munin08:06:452
6560997234,23cyclictest31-21ksoftirqd/107:21:591
656099723,35cyclictest31-21ksoftirqd/111:39:301
657499713,55cyclictest2830-21chrt09:31:353
657499712,37cyclictest31728-21chrt12:06:413
657499712,37cyclictest31728-21chrt12:06:413
656899712,44cyclictest32509-21timerwakeupswit12:06:592
656899712,44cyclictest32509-21timerwakeupswit12:06:592
6568997123,33cyclictest29132-21sh09:21:252
6568997123,33cyclictest29132-21sh09:21:252
6560997155,10cyclictest8259-21apt-key07:01:251
6560997150,15cyclictest352-21runrttasks11:13:411
656099714,48cyclictest3526-21sendmail_mailqu09:31:541
656099714,30cyclictest31-21ksoftirqd/108:17:481
656099714,23cyclictest31-21ksoftirqd/110:36:581
656099714,23cyclictest31-21ksoftirqd/110:36:581
6560997140,22cyclictest11279-21find11:31:561
6560997135,11cyclictest181rcu_preempt11:30:001
6560997123,23cyclictest0-21swapper/107:31:521
656099712,28cyclictest31-21ksoftirqd/108:06:441
656099712,28cyclictest31-21ksoftirqd/108:06:441
657499703,55cyclictest9847-21chrt10:36:243
6574997021,33cyclictest0-21swapper/312:16:373
656899705,41cyclictest8071-21tail10:31:572
656899704,46cyclictest32574-21fschecks_time09:26:442
6568997043,21cyclictest7531-21latency_hist08:36:242
6568997038,16cyclictest18726-21ssh09:03:052
656899703,49cyclictest15202-21timerwakeupswit08:56:582
656899703,49cyclictest15202-21timerwakeupswit08:56:582
6568997026,32cyclictest27583-21sh11:06:032
6560997051,14cyclictest12664-21cstates08:51:391
6560997049,15cyclictest544-21cron09:01:221
6560997049,15cyclictest544-21cron09:01:221
6560997047,17cyclictest22980-21latency_hist09:11:241
6560997047,17cyclictest22980-21latency_hist09:11:241
6560997024,19cyclictest0-21swapper/107:06:401
657499693,33cyclictest32464-21chrt09:26:413
656899697,35cyclictest298-21in:imuxsock09:32:402
6568996942,8cyclictest181rcu_preempt08:49:462
6568996940,10cyclictest181rcu_preempt08:26:252
6568996940,10cyclictest181rcu_preempt08:26:252
6568996937,8cyclictest181rcu_preempt11:41:472
6568996937,8cyclictest181rcu_preempt11:41:472
6568996936,10cyclictest4805-21kworker/u17:0-nfsiod09:54:032
656899692,37cyclictest7318-21latency_hist11:26:232
6560996955,10cyclictest6540-21cyclictest10:27:041
6560996953,10cyclictest28860-21ntp_kernel_pll_08:01:501
6560996953,10cyclictest28860-21ntp_kernel_pll_08:01:501
656099695,28cyclictest31-21ksoftirqd/110:50:341
656099695,28cyclictest31-21ksoftirqd/110:50:341
6560996947,9cyclictest181rcu_preempt07:04:511
6560996947,12cyclictest0-21swapper/107:56:251
656099694,37cyclictest31818-21fschecks_count08:11:441
656099693,33cyclictest31-21ksoftirqd/112:26:081
656099693,33cyclictest31-21ksoftirqd/112:26:081
656099692,46cyclictest11718-21wc08:46:551
6560996922,19cyclictest291rcuc/112:01:541
657499687,29cyclictest23184-21chrt11:51:543
657499683,52cyclictest9151-21chrt07:02:243
657499682,34cyclictest20463-21taskset07:36:423
657499682,34cyclictest20463-21taskset07:36:423
657499682,33cyclictest17542-21taskset09:01:443
657499682,31cyclictest15230-21chrt07:21:233
6568996836,12cyclictest21195-21/usr/sbin/munin07:36:592
6568996836,12cyclictest21195-21/usr/sbin/munin07:36:592
656899683,39cyclictest15192-21sh07:21:212
656899683,36cyclictest13775-21sort11:36:432
6568996832,8cyclictest181rcu_preempt11:06:402
6568996832,8cyclictest181rcu_preempt11:06:402
6568996829,12cyclictest181rcu_preempt08:41:452
6568996829,12cyclictest181rcu_preempt08:41:452
6560996853,10cyclictest6540-21cyclictest08:33:381
6560996849,13cyclictest0-21swapper/107:11:481
656099682,44cyclictest31-21ksoftirqd/110:07:181
656099682,41cyclictest31-21ksoftirqd/110:32:491
656099682,38cyclictest31-21ksoftirqd/109:28:161
656099682,28cyclictest31-21ksoftirqd/111:47:491
656099682,19cyclictest31-21ksoftirqd/109:41:461
657499677,48cyclictest0-21swapper/310:21:363
657499677,48cyclictest0-21swapper/310:21:363
657499677,48cyclictest0-21swapper/310:21:363
657499673,31cyclictest11187-21chrt07:06:533
657499673,28cyclictest4236-21chrt08:26:243
657499673,28cyclictest4236-21chrt08:26:243
657499672,52cyclictest14420-21chrt09:51:093
657499672,52cyclictest14420-21chrt09:51:093
657499672,34cyclictest7081-21taskset09:37:273
657499672,34cyclictest7081-21taskset09:37:273
657499672,34cyclictest15458-21chrt10:45:403
656899674,36cyclictest29857-21latency_hist08:06:252
656899674,36cyclictest29857-21latency_hist08:06:252
6560996747,8cyclictest181rcu_preempt09:15:581
6560996744,13cyclictest0-21swapper/107:56:391
6560996723,8cyclictest0-21swapper/111:01:351
6560996723,24cyclictest31-21ksoftirqd/109:17:131
6560996723,24cyclictest31-21ksoftirqd/109:17:131
656099672,25cyclictest24063-21ssh10:59:321
656099672,25cyclictest24063-21ssh10:59:321
657499663,49cyclictest25966-21taskset11:56:483
657499663,49cyclictest25966-21taskset11:56:483
657499663,32cyclictest20951-21chrt09:06:553
657499663,32cyclictest20951-21chrt09:06:553
657499662,33cyclictest16088-21taskset10:46:373
657499662,33cyclictest16088-21taskset10:46:373
657499662,32cyclictest11305-21chrt10:37:253
657499662,32cyclictest11305-21chrt10:37:253
656899663,43cyclictest8249-21sh11:26:542
6568996623,32cyclictest9259-21latency_hist08:41:252
656099662,21cyclictest31-21ksoftirqd/107:31:251
657499653,45cyclictest0-21swapper/310:16:463
657499653,32cyclictest28149-21chrt10:12:003
657499653,31cyclictest9328-21chrt08:41:393
657499653,31cyclictest9328-21chrt08:41:393
657499653,31cyclictest20825-21chrt10:53:503
657499653,30cyclictest30676-21taskset11:11:173
657499653,30cyclictest30676-21taskset11:11:173
657499653,30cyclictest22420-21chrt07:41:533
657499652,33cyclictest32618-21chrt11:13:203
657499652,31cyclictest18312-21chrt11:43:513
657499652,31cyclictest18312-21chrt11:43:513
657499652,30cyclictest306-21chrt08:14:133
657499652,29cyclictest15479-21taskset11:39:183
657499652,28cyclictest2689-21chrt12:11:523
6568996545,8cyclictest181rcu_preempt08:51:482
6568996537,20cyclictest18424-21modprobe07:31:222
6560996549,10cyclictest13629-21sh10:41:511
6560996534,9cyclictest181rcu_preempt09:53:071
657499644,29cyclictest26154-21chrt07:52:063
657499643,30cyclictest13583-21chrt08:52:203
657499643,29cyclictest17637-21chrt09:56:233
657499643,29cyclictest10442-21chrt09:43:273
657499642,31cyclictest30156-21chrt08:06:403
657499642,31cyclictest30156-21chrt08:06:403
657499642,31cyclictest29412-21chrt08:02:433
657499642,31cyclictest29412-21chrt08:02:433
657499642,31cyclictest22456-21chrt10:56:463
657499642,31cyclictest22456-21chrt10:56:463
657499642,31cyclictest13105-21chrt11:36:013
657499642,30cyclictest29630-21chrt12:01:593
657499642,30cyclictest19922-21chrt07:35:043
657499642,27cyclictest26050-21chrt09:16:233
6560996445,7cyclictest181rcu_preempt07:20:591
656099643,43cyclictest9407-21sshd08:41:401
656099643,43cyclictest9407-21sshd08:41:401
657499634,28cyclictest12332-21chrt07:11:393
657499633,29cyclictest22615-21chrt10:03:213
657499632,31cyclictest6116-21chrt10:29:393
657499632,31cyclictest26297-21chrt09:16:363
657499632,31cyclictest26297-21chrt09:16:363
657499632,31cyclictest15387-21taskset08:57:063
657499632,31cyclictest15387-21taskset08:57:063
657499632,31cyclictest1006-21chrt08:16:363
657499632,30cyclictest9329-21chrt11:28:483
657499632,29cyclictest8953-21chrt08:40:173
657499632,29cyclictest12233-21chrt08:50:423
657499632,24cyclictest19661-21chrt11:46:393
656099636,32cyclictest31-21ksoftirqd/112:06:261
656099636,32cyclictest31-21ksoftirqd/112:06:261
656099633,25cyclictest20367-21/usr/sbin/munin07:36:391
656099633,25cyclictest20367-21/usr/sbin/munin07:36:391
6560996326,20cyclictest0-21swapper/107:41:351
657499624,28cyclictest24780-21chrt07:50:073
657499623,29cyclictest25348-21chrt10:07:303
657499623,28cyclictest28032-21taskset08:00:403
657499612,30cyclictest3654-21chrt11:19:443
657499612,30cyclictest3654-21chrt11:19:443
657499612,30cyclictest27120-21chrt11:04:453
657499612,29cyclictest5583-21chrt08:28:233
657499612,28cyclictest7966-21chrt07:00:173
657499612,28cyclictest7247-21chrt08:34:023
657499612,28cyclictest16637-21chrt07:24:273
657499612,22cyclictest19005-21taskset09:57:493
656099613,29cyclictest9240-21latency_hist08:41:251
656099612,40cyclictest24468-21unixbench_multi07:46:581
657499603,28cyclictest31742-21chrt09:25:523
657499603,28cyclictest31742-21chrt09:25:523
657499587,42cyclictest0-21swapper/311:21:493
657499493,38cyclictest0-21swapper/306:56:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional